Abstract: Techniques and systems for testing, evaluating and/or otherwise assessing the operation of vehicles are discussed herein. Aspects relate to a system and method comprising receiving video data comprising a video stream derived from a camera during a vehicle run and being in a first format; storing part of the video stream in the first format; transcoding part of the video stream from the first format into a second format; storing, in a cache, the part of the video stream as video segments in the second format; receiving a request for video data associated with a point in time, wherein: if video segments in the second format do not correspond to the point in time, transcoding part of the video stream in the first format to the second format to create further second video segments in the second format; transmitting and storing, in the cache, the further second video segments.

Abstract: Techniques are discussed herein for controlling autonomous vehicles within a driving environment, including generating and using bounding contours associated with objects detected in the environment. Image data may be captured and analyzed to identify and/or classify objects within the environment. Image-based and/or lidar-based techniques may be used to determine depth data associated with the objects, and a bounding contour may be determined based on the object boundaries and associated depth data. An autonomous vehicle may use the bounding contours of objects within the environment to classify the objects, predict the positions, poses, and trajectories of the objects, and determine trajectories and perform other vehicle control actions while safely navigating the environment.

Abstract: Techniques related to generating map data explicitly indicating a total drivable surface, which may include multiple types of drivable surfaces, are described herein. For instance, a given portion of a map may include map data indicating a combination of various drivable surfaces, such as road segments, lane properties, intersections, parking areas, shoulders, driveways, etc. Examples include joining these different types of drivable surfaces into combined map data that explicitly indicates a total drivable surface, such as a perimeter boundary indicating or representing a transition from a drivable surface to a non-drivable surface. The map data indicating the total drivable surface may be searched to determine information related to a drivable surface boundary, such as location and type. This boundary information may be used in various contexts, such as when planning a trajectory or remotely controlling a vehicle.

Abstract: Techniques for determining gaps for performing lane change operations are described. A first region in an environment of a vehicle can be determined. The first region can be associated with a first time period through which the vehicle is unable to travel and can correspond to a constraint space. A second region of the environment can be determined. The second region can be associated with a second time period and can correspond to a configuration space. A gap in the environment can be determined based on a portion of the configuration space that is exclusive of the constraint space. A trajectory can be determined based on the gap. The trajectory can be associated with performing a lane change operation and can be associated with a cost. The vehicle can be controlled to perform the lane change operation based at least in part on the trajectory and the cost.

Abstract: A vehicle computing system may implement techniques to control a vehicle to avoid collisions between the vehicle and an object in an environment in which a vehicle path and an object path merge. The techniques may include determining an initial merge location associated with the vehicle path and the object path and a final merge location. The final merge location may represent a location at which the vehicle is proximate to and ahead of the object. The vehicle computing system may determine whether the vehicle may proceed to the final merge location and merge with the object without the occurrence of a collision. The vehicle computing system may determine to maintain a vehicle trajectory or modify the vehicle to trajectory to yield to the object based on a determination of whether the vehicle may proceed without the occurrence of the collision.

Abstract: Techniques for enabling operation of a vehicle after detecting a fault associated with a component of the vehicle are described herein. A vehicle computing system can monitor components of the vehicle and identify a fault associated with a component. Based on the fault, the vehicle computing system can cause the vehicle to cease (e.g., stop) or limit operation in an environment. The vehicle computing system can receive a request to override the fault and continue operation of the vehicle in a recovery mode with limited parameters (e.g., limited speed, acceleration, etc.). Based on a determination that the request to override the fault is valid, the vehicle computing system can cause the vehicle to be controlled, by an operator or an autonomous controller, according to parameters of the recovery mode.

Abstract: A door interface system for a vehicle door includes a sensor and a visual indicator. The sensor may be a proximity sensor and may be positioned proximate to the visual indicator such that it will detect an object proximate to the proximity sensor. The visual indicator may convey the position of the sensor and/or a status of the vehicle door. The door interface system is configured to control the vehicle door based at least in part on detecting an object proximate the visual indicator.

Abstract: A computer implemented method. Includes: obtaining, from a sensor system onboard a vehicle, a plurality of hypotheses for an orientation of an object in a vicinity of the vehicle, relative to a coordinate system of the vehicle; estimating, based on the plurality of hypotheses, values of a probability distribution for the orientation of the object at a plurality of candidate orientations; and estimating, based at least in part on the estimated values of the probability distribution at the plurality of candidate orientations, the orientation of the object relative to the coordinate system.

Abstract: Techniques are disclosed for component verification for complex systems. The techniques may include receiving log data, obtaining ground truth data based on the log data and determining an outcome at least in part by simulating a prediction by a prediction component based on the log data and the ground truth data. The techniques may further include simulating a second prediction by the prediction component based on the ground truth data, determining whether the second prediction resulted in the negative outcome of the scenario and determining the disengagement event is attributable to a perception component of the autonomous operation system at least partly in response to determining the second prediction based on the ground truth data did not result in the negative outcome.

Abstract: Tracking a current and/or previous position, velocity, acceleration, and/or heading of an object using sensor data may comprise determining whether to associate a current object detection generated from recently received (e.g., current) sensor data with a previous object detection generated from formerly received sensor data. In other words, a track may identify that an object detected in former sensor data is the same object detected in current sensor data. However, multiple types of sensor data may be used to detect objects and some objects may not be detected by different sensor types or may be detected differently, which may confound attempts to track an object. An ML model may be trained to receive outputs associated with different sensor types and/or a track associated with an object, and determine a data structure comprising a region of interest, object classification, and/or a pose associated with the object.

Abstract: Techniques for aggregating costs associated with one or more heat maps to control a vehicle in an environment are discussed herein. A vehicle computing device can implement a model to determine heat maps and respective cost information for different features of the environment based on sensor data. The vehicle computing device can output a planned trajectory for the vehicle based on combining the heat maps. The techniques can also include determining a rationalization or root cause detailing reasons why the planned trajectory was determined.

Abstract: Techniques for determining a location of a vehicle in an environment using sensors and determining calibration information associated with the sensors are discussed herein. A vehicle can use map data to traverse an environment. The map data can include semantic map objects such as traffic lights, lane markings, etc. The vehicle can use a sensor, such as an image sensor, to capture sensor data. Semantic map objects can be projected into the sensor data and matched with object(s) in the sensor data. Such semantic objects can be represented as a center point and covariance data. A distance or likelihood associated with the projected semantic map object and the sensed object can be optimized to determine a location of the vehicle. Sensed objects can be determined to be the same based on matching with the semantic map object. Epipolar geometry can be used to determine if sensors are capturing consistent data.

Abstract: An autonomous vehicle may modify scene context map data based on specific object types and/or features perceived within an environment, and may use the modified map data with prediction machine learning models to predict the behaviors of other dynamic objects in the environment. In some examples, the vehicle may receive sensor data of the environment, as well as map data representing the static context of the environment. The vehicle may analyze the sensor data to determine combinations of object types, features, and/or events, and may use predefined heuristics to determine modifications to existing map features based on the specific combinations of object data. A multi-channel representation of the environment based on the modified map data may be provided to prediction models to predict the behavior of dynamic objects and control the vehicle.

Abstract: Techniques for image data protection using cyclic redundancy checks are disclosed herein. Some of the techniques may include, at a processor, receiving image data that includes multiple lines of pixel data. The processor may also determine at least a first hash value representing a first line of pixel data of the multiple lines of pixel data and a second hash value representing a second line of pixel data of the multiple lines of pixel data. The processor may also send the image data to a computing device that is configured to determine, based at least in part on the first hash value and the second hash value, whether the image data is corrupt.

Abstract: A teleoperations system that collaboratively works with an autonomous vehicle planning component to generate a path for controlling the autonomous vehicle to pass a situation where the vehicle is unable to identify a vehicle option to proceed and may comprise presenting one or more paths to a teleoperator (e.g., a human user, machine-learned model, and/or artificial intelligence component), such paths being generated either at the vehicle or remote system. The teleoperations system may receive input from the teleoperator indicating a vehicle option to select for the vehicle to proceed in the environment. The teleoperations system may generate a guidance path based on the vehicle options and the input and transmit the guidance path to the autonomous vehicle. Based at least in part on the guidance path, the autonomous vehicle may generate a control trajectory to use to navigate around the obstacle.

Abstract: Perception sensors of a vehicle can be used for various operating functions of the vehicle. A computing device may receive sensor data from the perception sensors, and may calibrate the perception sensors using the sensor data, to enable effective operation of the vehicle. To calibrate the sensors, the computing device may project the sensor data into a voxel space, and determine a voxel score comprising an occupancy score and a residual value for each voxel. The computing device may then adjust an estimated position and/or orientation of the sensors, and associated sensor data, from at least one perception sensor to minimize the voxel score. The computing device may calibrate the sensor using the adjustments corresponding to the minimized voxel score. Additionally, the computing device may be configured to calculate an error in a position associated with the vehicle by calibrating data corresponding to a same point captured at different times.

Abstract: Techniques for cleaning and calibrating a microphone are discussed herein. For example, a computing device can implement a microphone component to operate a speaker adjacent to a microphone to cause removal of an obstruction (e.g., rain, mud, dirt, dust, snow, ice, animal droppings, etc.) on or near the microphone. The microphone component can also or instead cause the speaker to output a frequency for testing operation of the microphone. By implementing the cleaning and/or calibrating techniques described herein, foreign particle(s) of varying size and type can be dislodged from an area near the microphone to improve performance of the microphone.

Abstract: Techniques for determining detection boxes representing objects in an environment using pixel clustering are disclosed herein. Autonomous vehicle sensors can capture data in an environment that may include separate objects, such as large and/or articulated vehicles. In an example, the data can include a plurality of pixels that are associated with a large object. A vehicle computing system can generate a detection box for each pixel of the plurality of pixels and can determine at least one averaged detection box representing the large object. In an example, the vehicle computing system can control the vehicle based in part on the at least one averaged detection boxes.

Abstract: Techniques are discussed for generating and optimizing trajectories for controlling autonomous vehicles in performing on-route and off-route actions within a driving environment. A planning component of an autonomous vehicle can receive or generate time-discretized (or temporal) trajectories for the autonomous vehicle to traverse an environment. Trajectories can be optimized, for example, based on the lateral and longitudinal dynamics of the vehicle, using loss functions and/or costs. In some examples, the temporal optimization of a trajectory may include resampling a previous trajectory based on the differences in the time sequences of the temporal trajectories, to ensure temporal consistency of trajectories across planning cycles. Constraints also may be applied during temporal optimization in some examples, to control or restrict driving maneuvers that are not supported by the autonomous vehicle.

Abstract: A door assembly includes a door, a door actuator mechanism configured to transition the door between an open position and a closed position, a first latch configured to hold the door in the closed position, and a second latch engaging with a portion of the door actuator mechanism and being configured to hold the door in the closed position. The second latch includes a cam and a biasing member coupled to the cam. The biasing member is configured to transition the cam between a first position in which the cam engages the portion of the door actuator mechanism, and a second position in which the cam disengages the portion of the door actuator mechanism.