Abstract: Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Abstract: Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Abstract: Disclosed are autonomous vehicles that may autonomously navigate at least a portion of a route defined by a service request allocator. The autonomous vehicle may, at a certain portion of the route, request remote assistance. In response to the request, an operator may provide input to a console that indicates control positions for one or more vehicle controls such as steering position, brake position, and/or accelerator position. A command is sent to the autonomous vehicle indicating how the vehicle should proceed along the route. When the vehicle reaches a location where remote assistance is no longer required, the autonomous vehicle is released from manual control and may then continue executing the route under autonomous control.

Abstract: The present disclosure is directed to using anomaly data detected in traffic data to efficiently initiate remote assistance sessions. In particular, a computing system can receive, from a computing device associated with a human-driven vehicle, travel data for the human-driven vehicle. The computer system can identify a navigation anomaly associated with the human-driven vehicle based on the travel data. The computer system can generate, based on the identified navigation anomaly, an anomaly entry for storage in an anomaly database, the anomaly entry comprising geofence data describing a geographic area associated with the navigation anomaly. The computer system can determine, based on location data received from an autonomous vehicle and the geofence data, that the autonomous vehicle is entering the geographic area associated with the navigation anomaly. The computer system can initiate a remote assistance session with the autonomous vehicle.

Abstract: Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Abstract: Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Abstract: Systems and methods for vehicle motion control with interactive object annotation are provided. A method can include obtaining data indicative of a plurality of objects within a surrounding environment of the autonomous vehicle. For example, the plurality of objects can include at least at one problem object encountered by the autonomous vehicle while navigating a planned route. The method can include determining a group of objects of the plurality of objects. For example, the group of objects can include the problem object and one or more other objects in proximity to the problem object. The method can include determining a classification update to be applied to the group of objects. The method can include applying the classification update to the group of objects. The method can include providing data indicative of the classification update for the group of objects to the autonomous vehicle for use in motion planning.

Abstract: Systems and methods for improving remote operator assistance for an autonomous vehicle are provided. In particular, a computing system can obtain a plurality of sensor sweeps. The computing system can receive a request for remote assistance from an autonomous vehicle. The computing system can receive real-time video data from the autonomous vehicle. The computing system can receive real-time telemetry data from the autonomous vehicle. The computing system can present the real-time video data in a remote assistance interface. The computing system can generate a graphical overlay based on the telemetry data and display it in a remote assistance user interface over the real-time video data. The computing system can display one or more remote assistance options in the remote assistance user interface. The computing system can transmit instructions to perform one or more of the remote assistance options based on input from a remote assistance operator.

Abstract: The present disclosure is directed to a system for generating customized command toolboxes for remote operators in a service system that includes autonomous vehicles. The system receives a request for remote assistance from an autonomous vehicle. The system determines, from a local storage location, vehicle data associated with the autonomous vehicle. The system selects a subset of remote assistance actions from a predetermined set of remote assistance actions. The system displays, in a remote assistance user interface, one or more user interface elements indicative of the subset of remote assistance actions. The system determines one or more remote assistance actions from the subset of remote assistance actions based at least in part on a user input associated with the one or more user interface elements. The system transmits one or more control signals associated with the one or more remote assistance actions.

Abstract: The present disclosure is directed to using anomaly data detected in traffic data to efficiently initiate remote assistance sessions. In particular, a computing system can receive, from a computing device associated with a human-driven vehicle, travel data for the human-driven vehicle. The computer system can identify a navigation anomaly associated with the human-driven vehicle based on the travel data. The computer system can generate, based on the identified navigation anomaly, an anomaly entry for storage in an anomaly database, the anomaly entry comprising geofence data describing a geographic area associated with the navigation anomaly. The computer system can determine, based on location data received from an autonomous vehicle and the geofence data, that the autonomous vehicle is entering the geographic area associated with the navigation anomaly. The computer system can initiate a remote assistance session with the autonomous vehicle.

Abstract: The present disclosure is directed to autonomous vehicle service assignment simulation using simulated remote operators. In particular, a computing system comprising one or more computing devices can obtain sensor data associated with an interior of an autonomous vehicle. The computing system can determine using the sensor data that the interior of the autonomous vehicle contains one or more passengers. In response to determining that the interior of the autonomous vehicle contains one or more passengers, the computing system can analyze the sensor data to determine whether the one or more passengers are violating one or more passenger policies. In response to determining that the one or more passengers are violating one or more passenger policies, the computing system can automatically initiate a remote assistance session with a remote operator.

Abstract: The present disclosure is directed to autonomous vehicle service assignment simulations using simulated remote operators. In particular, a computing system comprising one or more computing devices can obtain data associated with a simulated autonomous vehicle to use within a simulation environment based at least in part on a service assignment. The computing system can generate one or more simulated remote assistance operators. The computing system can initiate a simulation of a service assignment using the data associated with the simulated autonomous vehicle to perform the service assignment within the simulation environment. The computing system can provide one or more simulated events from the one or more simulated remote assistance operators to the simulated autonomous vehicle. The computing system can determine whether the simulated autonomous vehicle has successfully completed the service assignment based at least in part on the current state of the simulation.

Abstract: The present disclosure is directed to a system for generating customized command toolboxes for remote operators in a service system that includes autonomous vehicles. The system receives a request for remote assistance from an autonomous vehicle. The system determines, from a local storage location, vehicle data associated with the autonomous vehicle. The system selects a subset of remote assistance actions from a predetermined set of remote assistance actions. The system displays, in a remote assistance user interface, one or more user interface elements indicative of the subset of remote assistance actions. The system determines one or more remote assistance actions from the subset of remote assistance actions based at least in part on a user input associated with the one or more user interface elements. The system transmits one or more control signals associated with the one or more remote assistance actions.

Abstract: Systems and methods for vehicle motion control with interactive object annotation are provided. A method can include obtaining data indicative of a plurality of objects within a surrounding environment of the autonomous vehicle. For example, the plurality of objects can include at least at one problem object encountered by the autonomous vehicle while navigating a planned route. The method can include determining a group of objects of the plurality of objects. For example, the group of objects can include the problem object and one or more other objects in proximity to the problem object. The method can include determining a classification update to be applied to the group of objects. The method can include applying the classification update to the group of objects. The method can include providing data indicative of the classification update for the group of objects to the autonomous vehicle for use in motion planning.

Abstract: Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Abstract: Disclosed are autonomous vehicles that may autonomously navigate at least a portion of a route defined by a service request allocator. The autonomous vehicle may, at a certain portion of the route, request remote assistance. In response to the request, an operator may provide input to a console that indicates control positions for one or more vehicle controls such as steering position, brake position, and/or accelerator position. A command is sent to the autonomous vehicle indicating how the vehicle should proceed along the route. When the vehicle reaches a location where remote assistance is no longer required, the autonomous vehicle is released from manual control and may then continue executing the route under autonomous control.