Abstract: A vehicle moves through a construction environment and performs one or more construction actions in the environment. A master coordinator associated with the construction environment updates a global state of the construction environment based on the performed construction actions. The master coordinator may determine a plan of construction actions for machines in the environment using a generative language model and send instruction to perform the construction actions or capture data from an associated viewpoint based on the global state to the machines. Each machine may have a local coordinator that determines its local state based on sensor data and may alter actions within alteration constraints provided by the master coordinator. The master coordinator receives local states from the machines over time, updates the global state accordingly, and sends instructions to the one or more machines based on the updated global state.

Abstract: A control system deploys a virtual safety bubble for autonomous operation of a construction vehicle in a worksite. The control system obtains a work schedule for the construction vehicle to perform one construction action. The control system identifies one or more objects to be interacted with by the construction vehicle while performing the one construction action. The control system deploys a virtual safety bubble for the construction vehicle based on the one construction action. Breach of the virtual safety bubble triggers remedial actions; however, the virtual safety bubble permits breach by the one or more identified objects during the construction action. The control system detects breach of the virtual safety bubble by the one or more identified objects during the construction action. Responsive to the breach of the virtual safety bubble by the one or more identified objects during the construction action, the control system withholds the remedial actions.

Abstract: A vehicle moves through a construction environment and performs one or more construction actions in the environment. A master coordinator associated with the construction environment updates a global state of the construction environment based on the performed construction actions. The master coordinator may determine a plan of construction actions for machines in the environment using a generative language model and send instruction to perform the construction actions or capture data from an associated viewpoint based on the global state to the machines. Each machine may have a local coordinator that determines its local state based on sensor data and may alter actions within alteration constraints provided by the master coordinator. The master coordinator receives local states from the machines over time, updates the global state accordingly, and sends instructions to the one or more machines based on the updated global state.

Abstract: A vehicle moves through a landscaping environment and performs one or more landscaping actions in the environment. A master coordinator associated with the landscaping environment updates a global state of the landscaping environment based on the performed landscaping actions. The master coordinator may determine a plan of landscaping actions for machines in the environment using a generative language model and send instruction to perform the landscaping actions or capture data from an associated viewpoint based on the global state to the machines. Each machine may have a local coordinator that determines its local state based on sensor data and may alter actions within alteration constraints provided by the master coordinator. The master coordinator receives local states from the machines over time, updates the global state accordingly, and sends instructions to the one or more machines based on the updated global state.

Abstract: A construction vehicle may move along a route in a construction environment towards a portion of the construction environment. The construction vehicle may access an image of the portion of the construction environment in front of the construction vehicle. The construction vehicle may apply a rut identification model to the image, where the rut identification model is configured to identify a rut in the portion of the construction environment using pixels of the image. The construction vehicle may, subsequent to identifying the rut in the portion of the construction environment, navigate the construction vehicle along the route based on the identified rut.

Abstract: A modular system includes a hub and a set of modules removably coupled to the hub. The modules are physically coupled to the frame relative to each other so that each module can operate with respect to a different row of a field. An individual module includes a sensor for capturing field measurement data of individual plants along a row as the modular system moves through the geographic region. An individual module further includes a treatment mechanism for applying a treatment to the individual plants of the row based on the field measurement data before the modular system passes by the individual plants. An individual module further includes a computing device that determines the treatment based on the field measurement data and communicates data to the hub. The hub is communicatively coupled to the modules, so that it may exchange data between the modules and with a remote computing system.

Abstract: A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

Abstract: A method including: recording a first image of a first field region; automatically treating a plant within the first region in-situ based on the first image: automatically verifying the plant treatment with a second image of the first region; and automatically treating a second region concurrently with treatment verification.

Abstract: A modular system includes a hub and a set of modules removably coupled to the hub. The modules are physically coupled to the frame relative to each other so that each module can operate with respect to a different row of a field. An individual module includes a sensor for capturing field measurement data of individual plants along a row as the modular system moves through the geographic region. An individual module further includes a treatment mechanism for applying a treatment to the individual plants of the row based on the field measurement data before the modular system passes by the individual plants. An individual module further includes a computing device that determines the treatment based on the field measurement data and communicates data to the hub. The hub is communicatively coupled to the modules, so that it may exchange data between the modules and with a remote computing system.

Abstract: A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

Abstract: The calibration system of the farming machine receives images from the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate the camera array, the system determines a relative pose between pairs of cameras by extracting relative position and orientation characteristics from visual information in images captured by the camera pairs. The calibration system can determine that a pair of cameras is in a swapped state by comparing the relative pose of the pair of cameras to an expected pose of the pair of cameras. The calibration system adjusts the pair to remedy the swapped state.

Abstract: A farming machine including a number of treatment mechanisms treats plants according to a treatment plan as the farming machine moves through the field. The control system of the farming machine executes a plant identification model configured to identify plants in the field for treatment. The control system generates a treatment map identifying which treatment mechanisms to actuate to treat the plants in the field. To generate a treatment map, the farming machine captures an image of plants, processes the image to identify plants, and generates a treatment map. The plant identification model can be a convolutional neural network having an input layer, an identification layer, and an output layer. The input layer has the dimensionality of the image, the identification layer has a greatly reduced dimensionality, and the output layer has the dimensionality of the treatment mechanisms.

Abstract: A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. Portions of the environment may include moisture, such as puddles or mud patches. A control system associated with the vehicle may include a traversability model or a moisture model to help the vehicle operate in the environment with the moisture. In particular, the control system may employ the traversability model to reduce the likelihood of the vehicle attempting to traverse an untraversable portion of the environment, and the control system may employ the moisture model to reduce the likelihood of the vehicle performing an action that will damage a portion of the environment.

Abstract: A method including: recording a first image of a first field region; automatically treating a plant within the first region in-situ based on the first image; automatically verifying the plant treatment with a second image of the first region; and automatically treating a second region concurrently with treatment verification.

Abstract: A farming machine including a number of treatment mechanisms treats plants according to a treatment plan as the farming machine moves through the field. The control system of the farming machine executes a plant identification model configured to identify plants in the field for treatment. The control system generates a treatment map identifying which treatment mechanisms to actuate to treat the plants in the field. To generate a treatment map, the farming machine captures an image of plants, processes the image to identify plants, and generates a treatment map. The plant identification model can be a convolutional neural network having an input layer, an identification layer, and an output layer. The input layer has the dimensionality of the image, the identification layer has a greatly reduced dimensionality, and the output layer has the dimensionality of the treatment mechanisms.

Abstract: The calibration system of the farming machine receives images from the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate the camera array, the system determines a relative pose between pairs of cameras by extracting relative position and orientation characteristics from visual information in images captured by the camera pairs. The calibration system determines a calibration error in part by propagating the relative poses between camera pairs. The calibration system may perform automated self-calibration by adjusting one or more of the cameras in the camera array, or may transmit remedial instructions to an operator to adjust the one or more cameras in the camera array.

Abstract: A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. Portions of the environment may include moisture, such as puddles or mud patches. A control system associated with the vehicle may include a traversability model or a moisture model to help the vehicle operate in the environment with the moisture. In particular, the control system may employ the traversability model to reduce the likelihood of the vehicle attempting to traverse an untraversable portion of the environment, and the control system may employ the moisture model to reduce the likelihood of the vehicle performing an action that will damage a portion of the environment.

Abstract: As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on depth information in in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.

Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on depth information in in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.