Abstract: A vehicle moves through a construction environment and performs one or more construction actions in the environment. A master coordinator associated with the construction environment updates a global state of the construction environment based on the performed construction actions. The master coordinator may determine a plan of construction actions for machines in the environment using a generative language model and send instruction to perform the construction actions or capture data from an associated viewpoint based on the global state to the machines. Each machine may have a local coordinator that determines its local state based on sensor data and may alter actions within alteration constraints provided by the master coordinator. The master coordinator receives local states from the machines over time, updates the global state accordingly, and sends instructions to the one or more machines based on the updated global state.

Abstract: A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. A control system associated with the vehicle may include a mode selection module and a transportation determination module. In particular, the control system may employ a machine vision model to identify images captured for each region in the environment. The model identifies viable unloading locations in an unloading zone of the environment, and then navigates the vehicle to one or more unloading locations based on pixels of the image representing terrain features and existing cargo dump piles. Control signals for loading or unloading cargo are determined in alignment with a transportation protocol set by a site manager. A loading or unloading mechanism is then actuated using these control signals to unload cargo in the appropriate location.

Abstract: A control system deploys a virtual safety bubble for autonomous operation of a construction vehicle in a worksite. The control system obtains a work schedule for the construction vehicle to perform one construction action. The control system identifies one or more objects to be interacted with by the construction vehicle while performing the one construction action. The control system deploys a virtual safety bubble for the construction vehicle based on the one construction action. Breach of the virtual safety bubble triggers remedial actions; however, the virtual safety bubble permits breach by the one or more identified objects during the construction action. The control system detects breach of the virtual safety bubble by the one or more identified objects during the construction action. Responsive to the breach of the virtual safety bubble by the one or more identified objects during the construction action, the control system withholds the remedial actions.

Abstract: A vehicle moves through a construction environment and performs one or more construction actions in the environment. A master coordinator associated with the construction environment updates a global state of the construction environment based on the performed construction actions. The master coordinator may determine a plan of construction actions for machines in the environment using a generative language model and send instruction to perform the construction actions or capture data from an associated viewpoint based on the global state to the machines. Each machine may have a local coordinator that determines its local state based on sensor data and may alter actions within alteration constraints provided by the master coordinator. The master coordinator receives local states from the machines over time, updates the global state accordingly, and sends instructions to the one or more machines based on the updated global state.

Abstract: A construction vehicle may move along a route in a construction environment towards a portion of the construction environment. The construction vehicle may access an image of the portion of the construction environment in front of the construction vehicle. The construction vehicle may apply a rut identification model to the image, where the rut identification model is configured to identify a rut in the portion of the construction environment using pixels of the image. The construction vehicle may, subsequent to identifying the rut in the portion of the construction environment, navigate the construction vehicle along the route based on the identified rut.