One hundred eighty-five years in, Deere’s agricultural initiatives have gone from plows to near-full tractor autonomy.

In 1837, John Deere debuted the first commercially successful self-scouring steel plow. Today, the global company fertilized by his invention seeks to again transform farming with the John Deere Autonomous 8R Tractor. Combining a flagship tractor with its TruSet™-enabled chisel plow and technologies from GPS guidance to image stitching, AI to machine learning to real-time data, the Deere-exclusive system and its sub-inch accuracy marks a major step out of the cab. A Deere spokesperson put the system’s autonomy at Level 4 with some characteristics of Level 5.

Joe Liefer, senior product manager for autonomy, John Deere Intelligent Solutions Group, enthusiastically endorses using autonomy to increase productivity, profitability and sustainability for its customers. A 17-year Deere veteran, Liefer defines his work as “discovering what farmers will need in their first autonomous products and then working with engineering to develop these solutions.”

IUS: WHEN DID DEERE GET INTO AUTONOMY?

LIEFER: We’re leveraging 20 years of self-driving with GPS on tractors. In 2019, we started forming our Autonomy Development Team, which I’m a part of now, to bring autonomy to fall tillage. We talked to farmers—what are you looking for? Why do you want it?

IUS: WHAT DOES AUTONOMY BRING TO PRECISION AGRICULTURE?

LIEFER: We truly believe that the next pivot point for agriculture is full autonomy. By leveraging computer vision, machine learning and advanced software, precision has taken this from a field level to managing every individual plant. How do we make sure it’s got the best growing environment, it’s protected, it’s nourished, and, ultimately, we’re going to grow the most food to feed the world? Precision autonomy is going to allow farmers to grow more food, help get the job done on time, help with their quality of life and ultimately be more profitable.

IUS: HOW DOES THIS AUTONOMY INTERSECT WITH THE MACRO FORCES FARMERS FACE?

LIEFER: Farmers have an incredibly rewarding job. They’re also under incredible time pressure to get the job done at the right time for the perfect growing conditions. And in many cases, multiple jobs need to happen at the same time. There are labor shortages in rural America, and they can’t work 24/7. Some of these growing seasons, it’s a very, very intense 6 to 8 weeks, 12 to 18 hours a day, and they can’t get every job on the farm done. That’s why they’re asking us for autonomy.

IUS: HOW DOES THIS WORK FROM THE FARMER’S STANDPOINT?

LIEFER: We’ve approached autonomy as a retrofit solution. It’s a bolt-on to the stock John Deere 8R tractor that’s in their machine shed today, so that a farmer is not going to be required to go buy a brand new machine, or some specialty purpose-built vehicle. In under eight hours, that John Deere dealer can install our camera pods, the wiring harnesses, the new tractor software and our precision ag technology to make that piece of equipment fully autonomous. It’s going to lower the barrier to entry.

IUS: AND IN THE FIELD?

LIEFER: It’s manually driven to the field. Then they’ll basically jump out of the cab. They’re going to grab their cell phone, open the John Deere Operations Center Mobile connected to the cloud and to the tractor as well. They’ll swipe to start motion and then they’re free to go do a second, value-added job, whether taking care of animals or being a seed salesman or an agronomist. Or spend time with family or in the community.

IUS: YOU’VE USED AN INTERESTING PHRASE: “THE CONFIDENCE TO LEAVE THE FIELD.” HOW DOES AUTONOMOUS TECHNOLOGY EMPOWER THIS?

LIEFER: Farmers have been creating field boundaries with our GPS technology for years. The tractor already knows where it’s at, so it’s going to automatically plan the most optimal, optimized way to farm that field. We have three pairs of stereo cameras on the front of the tractor, and three pairs on the back around the top of the cab, placed to give us 360-degree visibility. Access to a live video feed from any one of the cameras lets farmers monitor job quality. Farmers can make remote adjustments, speed or depth, directly from their cell phones, and the system adapts to the new setting.

On each camera pod, we have an NVIDIA Jetson GPU that’s processing the images. We’re looking at the world at three frames per second, and it takes about 100 milliseconds to process a frame. We’re looking for changes in the environment, objects that might be on the path of travel, and also anyone approaching the machine from the side. If we see an object, the GPU is going to request to bring the tractor to a stop immediately. We’re confident we can perceive the world around us.

IUS: WHAT’S THE ROLE OF DATA IN THIS SOLUTION?

LIEFER: We have telematics and machine data, where we know there’s many times where a farmer’s ultimate productivity isn’t 100%. As we optimize guidance plans, we can increase job efficiency.

In 2019, we started collecting images of farm fields. The actual models trained off of about 150,000 images, but we have 50 million images in our data set and that’s growing every day. We’re using machine learning to classify the world. We’re looking at the world with our stereo cameras from a red/green/blue image classification to perceive and categorize what’s around us. We’re also—which we think is super-novel—using a depth channel to improve predictions from stereo cameras because we need to be able to geospatially locate things out in the field—power poles, tree lines. Additionally, there’s another layer of computer vision model we’ve trained to look at job quality—residue plugging, making sure ground-engaging components aren’t broken. We also have a model checking on machine health. We’re leveraging different sensor modalities, a lot of computer vision machine learning, to replace the eyes and ears of the farmer.

IUS: WHAT WERE SOME DEVELOPMENTAL CHALLENGES YOU HAD TO OVERCOME?

LIEFER: We have extreme temperature ranges. It’s dusty. It’s outdoors. So we had to ruggedize the hardware and the components. The NVIDIA Jetson chip wasn’t necessarily designed to be bounced around in a farmer’s field all day, so we had to work on an enclosure for that. We developed a model to recognize the unexpected—something blowing into a field, a tree breaking where a farmer wasn’t aware it was down—to make sure we stop.

IUS: HOW DID YOU INCLUDE FARMERS IN THE DEVELOPMENT LOOP?

LIEFER: We’ve got development partners. First collecting data on their farms, then, as we created the product, giving us feedback. Is it meeting their expectations? What do they need so they can adopt it? We’re staging some actions in the field to make sure we’ve got a safe system.

IUS: HOW HAVE DEERE’S ACQUISITIONS FIT INTO THIS PROCESS?

LIEFER: In 2017, we acquired Blue River Technology, a computer vision machine learning company. Recently, we acquired Bear Flag Robotics, an autonomy company. We believe the opportunity for autonomy is vast, and so, while our initial entry here is with stereo cameras, Bear Flag Robotics has approached autonomy with LiDAR and radar. As we get into new jobs, we’ll have some additional sensor modalities to make sure we can do every job a farmer expects us to do.

IUS: WHAT’S THE TIMETABLE FOR GETTING THIS SYSTEM TO THE FARMER?

LIEFER: In 2022, it will be available in limited release; we’re starting with fall tillage. I’ve already demoed it to farmers; I’m already sold out for fall ‘22 of the units we’ll have. As we step into 2023, we will qualify more tractor and implement models capable of doing jobs autonomously for farmers.

IUS: FARMS ARE WIDELY VARIABLE IN CROPS AND SIZE, LOCATION AND SOIL. WHERE WOULD YOU SEE THIS POSITIONED?

LIEFER: Think about a farmer that has a full-time day job but still might be farming 1,500 acres on nights and weekends, missing out on family time and sporting events. This part-time farmer will be able to start a job in the morning, go to a day job, maybe come back, move the tractor to another job, let it run all night. For them, it’s going to be a game changer for quality of life. Full-time farmers—they’ll be able to leverage autonomy and still focus on other ways they’re generating income. Then let’s move to the ultra-large farms. They’ve got dozens of machines, dozens of employees, which is a challenge, finding, training, retaining labor. Let’s run with autonomy to replace that need.

IUS: AND PRICING?

LIEFER: We haven’t released pricing, but this will be a different business model for Deere. As we think about how to make autonomy more accessible to more growers, we’re going to look at a pay-for-usage type model, to allow farmers to continually get access to the latest and most capable software.

IUS: SOME COVERAGE HAS VOICED WORRIES THAT AUTONOMY WILL WIPE OUT FARMERS. HOW DO YOU TALK ABOUT THAT?

LIEFER: It’s not that farmers think this is going to replace them. It’s really that force multiplier that they’re excited about. We’re not trying to take farmers out of the farm; we’re trying to give them opportunities for what, how and where they want to spend their time.

Farmers in the U.S. are 55-or-so years old, and that average continues to increase. And there are fewer and fewer people wanting to work in rural America. This will hopefully excite a younger generation that’s very willing to adopt new tech to still be a farmer.

IUS: WHAT OTHER AUTONOMOUS PROJECTS ARE YOU TRYING TO DO?

LIEFER: We’re starting in, I’ll call it row crop agriculture—that’s where our bread and butter is. But farmers are asking us to help automate the complete production system, so we totally foresee that this tech goes to a self-propelled sprayer, a combine, and we will incorporate what sensing technology we need. And not only in farming; there’s construction for job sites. We serve customers in many different aspects of raising food, fuel and fiber, where we see autonomy opportunities.

IUS: FINALLY, WHAT WOULD JOHN DEERE SAY IF HE CAME BACK AND SAW THIS?

LIEFER: He was bringing innovation to the farm in the form of a steel plow. We’ve gone from horses to having a robot that can do the job with the farmer. It’s really, really cool to be part of a brand that has this history of innovation. Ant there’s our partnership with farmers as we develop technology to help feed the world.