Abstract: The present disclosure relates generally to systems for facilitating the use of autonomous vehicles (AVs), and more particularly to automated artificial intelligence (AI)-based techniques for determining an insurance premium for an AV ride based upon various factors including the evaluation of risk associated with the AV ride. An automated AI-based infrastructure is provided that uses automated machine-learning (ML) based techniques for evaluating a level of risk for any particular AV ride and then determining an insurance premium for the AV ride based on the level of risk. The insurance premium determination incorporates Usage Based Insurance Pricing (UBIP) that has been customized for autonomous driving, whereby the level of risk is predicted based on information associated with the expected usage of the AV during the ride. Thus, the insurance premium is customized for each ride and can be determined as part of calculating upfront the total price of the ride.

Abstract: Techniques are described herein for determining one or more actions for an autonomous vehicle to perform, based on simulation of at least one possible scenario. A possible scenario may involve, for example, the autonomous vehicle interacting with an object in the environment. The possible scenario may be simulated by modifying a first internal map containing information about the autonomous vehicle and the environment. As part of the simulation, one or more parameters of the first internal map can be modified in order to, for example, determine the state of the object at a particular point in the future. Based on the modification of the one or more parameters, a second internal map representing a possible scenario is generated from the first internal map. Both the first internal map and the second internal map can be evaluated to decide which action to take.

Abstract: The present disclosure relates generally to autonomous machines (AMs) and more particularly to techniques for intelligently planning, managing and performing various tasks using AMs. A control system (referred to as a fleet management system or FMS) is disclosed for managing a set of resources at a site, which may include AMs. The FMS is configured to control and manage the AMs at the site such that tasks are performed autonomously by the AMs. An AM may directly communicate with another AM located on the site to complete a task without requiring to be in constant communication with the FMS during the performance of the task. The FMS is configured to use various optimization techniques to allocate resources (e.g., AMs) for performing tasks at the site. The resource allocation is performed so as to maximize the use of available AMs while ensuring that the tasks get performed in a timely manner.

Abstract: Techniques are described herein for determining one or more actions for an autonomous vehicle to perform, based on simulation of at least one possible scenario. A possible scenario may involve, for example, the autonomous vehicle interacting with an object in the environment. The possible scenario may be simulated by modifying a first internal map containing information about the autonomous vehicle and the environment. As part of the simulation, one or more parameters of the first internal map can be modified in order to, for example, determine the state of the object at a particular point in the future. Based on the modification of the one or more parameters, a second internal map representing a possible scenario is generated from the first internal map. Both the first internal map and the second internal map can be evaluated to decide which action to take.

Abstract: The present disclosure relates techniques for autonomously controlling heavy equipment and vehicles using task hierarchies. Particularly, aspects of the present disclosure are directed to obtaining a task to be performed by an autonomous vehicle, determining subtasks to be performed to perform the task, obtaining sensor data providing a representation of operation of the autonomous vehicle in a worksite environment and situational context of the worksite environment, determining a task context for a subtask based on the sensor data, identifying a predictive model from a library of predictive models based on the task context, estimating, by the predictive model, a set of output data based on sensor data, and controlling operations of the autonomous vehicle in the worksite environment to perform the subtask using a set of input data derived from the sensor data and the set of output data.

Abstract: Systems and methods enable an autonomous vehicle to perform an iterative task of transferring material from a source location to a destination location, such as moving dirt from a pile, in a more efficient manner, using a combination of reinforcement learning techniques to select a motion path for a particular iteration and visual servo control to guide the motion of the vehicle along the selected path. Lifting, carrying, and depositing of material by the autonomous vehicle can also be managed using similar techniques.

Abstract: The present disclosure relates generally to identification of drivable surfaces in connection with autonomously performing various tasks at industrial work sites and, more particularly, to techniques for distinguishing drivable surfaces from non-drivable surfaces based on sensor data. A framework for the identification of drivable surfaces is provided for an autonomous machine to facilitate it to autonomously detect the presence of a drivable surface and to estimate, based on sensor data, attributes of the drivable surface such as road condition, road curvature, degree of inclination or declination, and the like. In certain embodiments, at least one camera image is processed to extract a set features from which surfaces and objects in a physical environment are identified, and to generate additional images for further processing. The additional images are combined with a 3D representation, derived from LIDAR or radar data, to generate an output representation indicating a drivable surface.

Abstract: Systems and methods for object detection in a dusty environment can enhance the ability of autonomous machines to distinguish dust clouds from solid obstacles and proceed appropriately. A library of dust classifiers can be provided, where each dust classifier is separately trained to distinguish airborne dust from objects in the environment. Different dust classifiers can correspond to different categories of dusty environments. Based on current conditions, control logic in an autonomous machine can categorize its environment and select a corresponding dust classifier. The dust classifier output can be used to alter a behavior of the autonomous machine, including a behavior of the control logic. For instance, the control logic can apply a consistency check to the output of the dust classifier and an output of an AI-based object classifier to detect instances where the object classifier misidentifies dust as an object.

Abstract: An infrastructure is provided for improving the safety of autonomous systems. An autonomous vehicle management system (AVMS) controls one or more autonomous functions or operations performed by a vehicle or machine such that the autonomous operations are performed in a safe manner. The AVMS is capable of dynamically controlling the behavior of sensors associated with a vehicle. For example, for a sensor, the AVMS can dynamically change and control what sensor data is captured by the sensor and/or communicated from the sensor to the AVMS (e.g., granularity/resolution, field of view, control zoom), when the data is captured by the sensor and/or communicated by the sensor to the AVMS (e.g., on-demand, according to a schedule), and how the data is captured by the sensor and/or communicated from the sensor to the AVMS (e.g., communication format, communication protocol, rate of data communication).

Abstract: The present disclosure relates generally to techniques for the kinematic estimation and dynamic behavior estimation of autonomous heavy equipment or vehicles to improve navigation, digging and material carrying tasks at various industrial work sites. Particularly, aspects of the present disclosure are directed to obtaining a set of sensor data providing a representation of operation of an autonomous vehicle in a worksite environment, estimating, by a trained model comprising a Gaussian process, a set of output data based on the set of sensor data, controlling an operation of the autonomous vehicle in the worksite environment using input data derived from the set of sensor data and the set of output data, obtaining actual output data from the operation of the autonomous vehicle in the worksite environment, and updating the trained model with the input data and the actual output data.

Abstract: The present disclosure relates generally to the operation of autonomous machinery for performing various tasks at various industrial work sites, and more particularly to the volumetric estimation and dimensional estimation of a pile of material or other object, and the use of multiple sensors for the volumetric estimation and dimensional estimation of a pile of material or other object at such work sites. An application and a framework is disclosed for volumetric estimation and dimensional estimation of a pile of material or other object using at least one sensor, preferably a plurality of sensors, on an autonomous machine (e.g., robotic machines or autonomous vehicles) in various work-site environments applicable to various industries such as, construction, mining, manufacturing, warehousing, logistics, sorting, packaging, agriculture, etc.

Abstract: The present disclosure relates techniques for autonomously controlling heavy equipment and vehicles using task hierarchies. Particularly, aspects of the present disclosure are directed to obtaining a task to be performed by an autonomous vehicle, determining subtasks to be performed to perform the task, obtaining sensor data providing a representation of operation of the autonomous vehicle in a worksite environment and situational context of the worksite environment, determining a task context for a subtask based on the sensor data, identifying a predictive model from a library of predictive models based on the task context, estimating, by the predictive model, a set of output data based on sensor data, and controlling operations of the autonomous vehicle in the worksite environment to perform the subtask using a set of input data derived from the sensor data and the set of output data.

Abstract: An infrastructure is provided for improving the safety of autonomous systems. An autonomous vehicle management system (AVMS) controls one or more autonomous functions or operations performed by a vehicle or machine such that the autonomous operations are performed in a safe manner. The AVMS uses various artificial intelligence (AI) based techniques (e.g., neural networks, reinforcement learning (RL) techniques, etc.) and models as part of its processing. For an inferring data point, for which a prediction is made by AVMS using an AI model, the AVMS checks how statistically similar (or dissimilar) the inferring data point is to the distribution of the training dataset. A score (confidence score) is generated indicative of how similar or dissimilar the inferring data point is to the training dataset. The AVMS uses this confidence score to decide how the prediction made by the AI model is to be used.

Abstract: In one aspect, In one aspect, a method for data transfer and processing communications is provided. The method includes the step of providing a machine-to-everything (M2X) application layer on each machine of the plurality of machines. The method includes the step of providing a plurality of communication nodes on each machine for communication between the plurality of machines with every other machine, the plurality of machines and any infrastructure at a work site, and a plurality of communication nodes communicating using the at least one application layer. The method includes the step of providing a communication processing system for receiving a data transfer and processing communications. The communication processing system includes a plurality of processing stations, one or more multiple data management protocols, a plurality of network protocols, a plurality of databases and plurality of data processing network architectures.

Abstract: Techniques are described herein for determining one or more actions for an autonomous vehicle to perform, based on simulation of at least one possible scenario. A possible scenario may involve, for example, the autonomous vehicle interacting with an object in the environment. The possible scenario may be simulated by modifying a first internal map containing information about the autonomous vehicle and the environment. As part of the simulation, one or more parameters of the first internal map can be modified in order to, for example, determine the state of the object at a particular point in the future. Based on the modification of the one or more parameters, a second internal map representing a possible scenario is generated from the first internal map. Both the first internal map and the second internal map can be evaluated to decide which action to take.

Abstract: The present disclosure relates generally to identification of drivable surfaces in connection with autonomously performing various tasks at industrial work sites and, more particularly, to techniques for distinguishing drivable surfaces from non-drivable surfaces based on sensor data. A framework for the identification of drivable surfaces is provided for an autonomous machine to facilitate it to autonomously detect the presence of a drivable surface and to estimate, based on sensor data, attributes of the drivable surface such as road condition, road curvature, degree of inclination or declination, and the like. In certain embodiments, at least one camera image is processed to extract a set features from which surfaces and objects in a physical environment are identified, and to generate additional images for further processing. The additional images are combined with a 3D representation, derived from LIDAR or radar data, to generate an output representation indicating a drivable surface.

Abstract: Techniques are described herein for determining one or more actions for an autonomous vehicle to perform, based on simulation of at least one possible scenario. A possible scenario may involve, for example, the autonomous vehicle interacting with an object in the environment. The possible scenario may be simulated by modifying a first internal map containing information about the autonomous vehicle and the environment. As part of the simulation, one or more parameters of the first internal map can be modified in order to, for example, determine the state of the object at a particular point in the future. Based on the modification of the one or more parameters, a second internal map representing a possible scenario is generated from the first internal map. Both the first internal map and the second internal map can be evaluated to decide which action to take.

Abstract: Systems and methods for object detection in a dusty environment can enhance the ability of autonomous machines to distinguish dust clouds from solid obstacles and proceed appropriately. A library of dust classifiers can be provided, where each dust classifier is separately trained to distinguish airborne dust from objects in the environment. Different dust classifiers can correspond to different categories of dusty environments. Based on current conditions, control logic in an autonomous machine can categorize its environment and select a corresponding dust classifier. The dust classifier output can be used to alter a behavior of the autonomous machine, including a behavior of the control logic. For instance, the control logic can apply a consistency check to the output of the dust classifier and an output of an AI-based object classifier to detect instances where the object classifier misidentifies dust as an object.

Abstract: The present disclosure relates generally to the operation of autonomous machinery for performing various tasks at various industrial work sites, and more particularly to the volumetric estimation and dimensional estimation of a pile of material or other object, and the use of multiple sensors for the volumetric estimation and dimensional estimation of a pile of material or other object at such work sites. An application and a framework is disclosed for volumetric estimation and dimensional estimation of a pile of material or other object using at least one sensor, preferably a plurality of sensors, on an autonomous machine (e.g., robotic machines or autonomous vehicles) in various work-site environments applicable to various industries such as, construction, mining, manufacturing, warehousing, logistics, sorting, packaging, agriculture, etc.

Abstract: In one aspect, a computer-implemented method useful for managing autonomous vehicle application operations with reinforcement learning (RL) methods, the method includes the step of providing an autonomous vehicle application of an autonomous vehicle, wherein the autonomous vehicle application manages a final action of a specified operation of the autonomous vehicle. The method includes the step of generating an RL model-agent for the specified operation. The RL model-agent learns by a maximizing rewards function related to the specified operation. The method includes the step of generating and managing a Safety Sanity Index (SSI) that monitors the safety performance of RL model. The method includes the step of obtaining an observed state of the autonomous vehicle, and generating an interruptible command based on the SSI and the observed state.