Abstract: The calibration system of the farming machine receives images from each camera of the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate a pair of cameras including a first camera and second camera, the calibration system determines a relative pose between the pair of cameras by extracting relative position and orientation characteristics from visual information in both an image received from the first camera and an image received from the second camera. The calibration system identifies a calibration error for the pair of cameras based on a comparison of the relative pose with an expected pose between the first pair of cameras. The calibration system transmits a notification to an operator of the farming machine that describes the calibration error and instructions for remedying the calibration error.

Abstract: As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Abstract: A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

Abstract: A farming machine is configured to autonomously operate in a field to accomplish a farming objective. While doing so the farming machine can dynamically modify treatment plans and/or generate experimental treatment plans. Dynamically modifying treatment plans includes modifying, delaying, accelerating, adding, and/or removing farming actions in a treatment plan. Generating an experimental treatment plan includes intentionally modifying a treatment plan to induce a variance in a result, measuring the result, and determining whether the result from the experimental treatment plan is better than the unmodified treatment plan. The farming machine is also configured to generate user interactable feedback, and receive user generated instruction, to dynamically modify treatment plans and/or generate experimental treatment plans.

Abstract: A farming machine identifies and treats a plant as the farming machine travels through a field. The farming machine includes a pair of image sensors for capturing images of a plant. The image sensors are different, and their output images are used to generate a depth map to improve the plant identification process. A control system identifies a plant using the depth map. The control system captures images, identifies a plant, and actuates a treatment mechanism in real time.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on depth information in in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.

Abstract: A modular system includes a hub and a set of modules removably coupled to the hub. The modules are physically coupled to the frame relative to each other so that each module can operate with respect to a different row of a field. An individual module includes a sensor for capturing field measurement data of individual plants along a row as the modular system moves through the geographic region. An individual module further includes a treatment mechanism for applying a treatment to the individual plants of the row based on the field measurement data before the modular system passes by the individual plants. An individual module further includes a computing device that determines the treatment based on the field measurement data and communicates data to the hub. The hub is communicatively coupled to the modules, so that it may exchange data between the modules and with a remote computing system.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on depth information in in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.

Abstract: A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

Abstract: The calibration system of the farming machine receives images from each camera of the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate a pair of cameras including a first camera and second camera, the calibration system determines a relative pose between the pair of cameras by extracting relative position and orientation characteristics from visual information in both an image received from the first camera and an image received from the second camera. The calibration system identifies a calibration error for the pair of cameras based on a comparison of the relative pose with an expected pose between the first pair of cameras. The calibration system transmits a notification to an operator of the farming machine that describes the calibration error and instructions for remedying the calibration error.

Abstract: As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Abstract: As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Abstract: As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Abstract: A modular system includes a hub and a set of modules removably coupled to the hub. The modules are physically coupled to the frame relative to each other so that each module can operate with respect to a different row of a field. An individual module includes a sensor for capturing field measurement data of individual plants along a row as the modular system moves through the geographic region. An individual module further includes a treatment mechanism for applying a treatment to the individual plants of the row based on the field measurement data before the modular system passes by the individual plants. An individual module further includes a computing device that determines the treatment based on the field measurement data and communicates data to the hub. The hub is communicatively coupled to the modules, so that it may exchange data between the modules and with a remote computing system.

Abstract: A farming machine including a number of treatment mechanisms treats plants according to a treatment plan as the farming machine moves through the field. The control system of the farming machine executes a plant identification model configured to identify plants in the field for treatment. The control system generates a treatment map identifying which treatment mechanisms to actuate to treat the plants in the field. To generate a treatment map, the farming machine captures an image of plants, processes the image to identify plants, and generates a treatment map. The plant identification model can be a convolutional neural network having an input layer, an identification layer, and an output layer. The input layer has the dimensionality of the image, the identification layer has a greatly reduced dimensionality, and the output layer has the dimensionality of the treatment mechanisms.

Abstract: A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. Portions of the environment may include moisture, such as puddles or mud patches. A control system associated with the vehicle may include a traversability model or a moisture model to help the vehicle operate in the environment with the moisture. In particular, the control system may employ the traversability model to reduce the likelihood of the vehicle attempting to traverse an untraversable portion of the environment, and the control system may employ the moisture model to reduce the likelihood of the vehicle performing an action that will damage a portion of the environment.

Abstract: A farming machine including a number of treatment mechanisms treats plants according to a treatment plan as the farming machine moves through the field. The control system of the farming machine executes a plant identification model configured to identify plants in the field for treatment. The control system generates a treatment map identifying which treatment mechanisms to actuate to treat the plants in the field. To generate a treatment map, the farming machine captures an image of plants, processes the image to identify plants, and generates a treatment map. The plant identification model can be a convolutional neural network having an input layer, an identification layer, and an output layer. The input layer has the dimensionality of the image, the identification layer has a greatly reduced dimensionality, and the output layer has the dimensionality of the treatment mechanisms.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on the visual information in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses the image(s) and creates a labelled three-dimensional point cloud representing the field. The control system identifies and treats plants based on the labelled point cloud. To do so, the control system applies pre-processing functions to the labelled point cloud to determine characteristics of the field and/or modify labels in the point cloud. Point clusters in the point cloud are identified as plants, crops, weeds, ground, etc., using the determined characteristics and modified labels. The control system derives feature values for the plants based on the determined characteristics and labels. If the feature value indicates that the plant should be treated, the farming machine actuates a treatment mechanism to treat the plant.

Abstract: A method including: recording a first image of a first field region; automatically treating a plant within the first region in-situ based on the first image; automatically verifying the plant treatment with a second image of the first region; and automatically treating a second region concurrently with treatment verification.