Abstract: Provided are methods for track segment cleaning of tracked objects using neural networks, which can include detecting a first track segment and a second track segment. The method includes applying a machine learning model trained to determine if the first track segment and second track segment capture real objects and if the first track segment and the second track segment are representative of an identical object exterior to a vehicle. The method further includes combining the first track segment and the second track segment to form a single track segment having a single trajectory in response to the first track segment and the second track segment being determined to be representative of the identical object. Systems and computer program products are also provided.

Abstract: Among other things, we describe techniques for estimating a speed profile for a proposed trajectory for a vehicle and operating the vehicle along the proposed trajectory according to the speed profile, including a method for: obtaining, by a planning circuit on a vehicle, a proposed trajectory for the vehicle in response to a driving scenario; obtaining, by the planning circuit, an estimated speed profile, and a confidence score, wherein the confidence score represents a similarity of the estimated speed profile to an actual speed profile that would be generated by a control circuit for the proposed trajectory; determining whether the confidence score meets a confidence threshold; and in accordance with a determination that the confidence score exceeds the confidence threshold, operating, by a control circuit on the vehicle, the vehicle along the proposed trajectory.

Abstract: Provided are methods for managing traffic light detections, which can include: deriving a first state of a traffic light at an intersection a vehicle is approaching, according to first detection data acquired by a first traffic light detection (TLD) system; deriving a second state of the traffic light at the intersection, according to second detection data acquired by a second TLD system that is independent from the first TLD system; determining traffic light information at the intersection based on at least one of (i) the first state or (ii) a result of checking whether the first state is same as the second state; and causing the vehicle to operate in accordance with the determined traffic light information at the intersection. Systems and computer program products are also provided.

Abstract: Provided are methods for forming redundant node configurations in a multi-systems-on-a-chip environment. Each system-on-a-chip can include one or processors and memories independent of other systems-on-a-chip and in communication via a cache coherent fabric. To facilitate rapid and extensible reconfiguration, various systems-on-a-chip from the multi-systems-on-a-chip environment can be configured into a redundant node configuration, with each logical node implemented by one or more of the systems-on-a-chip. Each system-on-a-chip within a logical node can be configured to utilize a shared memory space, such as by transparent mirroring of logical memory addresses. Across nodes, systems-on-a-chip can communicate using a device-to-device protocol such as a non-transparent bridge. My reconfiguration of communication between systems-on-a-chip, the multi-systems-on-a-chip environment can be reconfigured to represent a variety of redundant configurations.

Abstract: Techniques are provided for autonomous vehicle operation using linear temporal logic. The techniques include using one or more processors of a vehicle to store a linear temporal logic expression defining an operating constraint for operating the vehicle. The vehicle is located at a first spatiotemporal location. The one or more processors are used to receive a second spatiotemporal location for the vehicle. The one or more processors are used to identify a motion segment for operating the vehicle from the first spatiotemporal location to the second spatiotemporal location. The one or more processors are used to determine a value of the linear temporal logic expression based on the motion segment. The one or more processors are used to generate an operational metric for operating the vehicle in accordance with the motion segment based on the determined value of the linear temporal logic expression.

Abstract: Provided are methods and apparatuses for testing imaging devices, which can include a testing apparatus a testing apparatus including a test chamber; a device receptacle inside the test chamber for holding a device under test; a target receptacle inside the test chamber for holding an optical target within a field of view of the device under test; a temperature controller configured to adjust a temperature proximate to the device receptacle; and an air curtain controller configured to generate an air curtain inside the test chamber between a location of the device receptacle and a location of the target receptacle, the air curtain thermally isolating the device receptacle from the target receptacle.

Abstract: A method may include applying a first machine learning model trained to perform an open-set detection by at least identifying, based at least on image data indicative of one or more objects present in at least one environment in which one or more vehicles operate, at least one new class of objects. A dataset including a plurality of classes of objects may be updated to include the at least one new class of objects. In some cases, the dataset may be further updated to include a label associated with the at least one new class of objects. A second machine learning model may be trained, or in some cases updated, based at least on the updated dataset including the at least one new class of objects. Related systems and computer program products are also provided.

Abstract: Among other things, we describe techniques for implementing a vehicle response to sensor failure. In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include receiving information from a plurality of sensors coupled to a vehicle, determining that a level of confidence of the received information from at least one sensor of a first subset of sensors of the plurality of sensors is less than a first threshold, comparing a number of sensors in the first subset of sensors to a second threshold, and adjusting the driving capability of the vehicle to rely on information received from a second subset of sensors of the plurality of sensors, wherein the second subset of sensors excludes the at least one sensor of the first subset of sensors.

Abstract: Among other things, techniques are described for controlling, using a control circuit, motion of a vehicle based objects identified using LiDAR. For example, respective classes of points of a point cloud are determined, and based on the determined respective classes of the points of the point cloud, objects in the vicinity of the vehicle are identified.

Abstract: Provided are methods for prediction error scenario mining for machine learning methods, which can include determining a prediction error indicative of a difference between a planned decision of an autonomous vehicle and an ideal decision of the autonomous vehicle. The prediction error is associated with an error-prone scenario for which a machine learning model of an autonomous vehicle is to make planned movements. The method includes searching a scenario database for the error-prone scenario based on the prediction error. The scenario database includes a plurality of datasets representative of data received from an autonomous vehicle sensor system in which the plurality of datasets is marked with at least one attribute of the set of attributes. The method further includes obtaining the error-prone scenario from the scenario database for inputting into the machine learning model for training the machine learning model. Systems and computer program products are also provided.

Abstract: Using a planning circuit of a vehicle, a map is accessed that includes information identifying at least one lane on which vehicles can travel. Using the planning circuit and from the map, a graph representing a driving environment of the vehicle is generated. The graph includes multiple trajectories. At least one trajectory includes a lane change. Each trajectory is a path for the vehicle to move from a first spatiotemporal location to a second spatiotemporal location. The trajectory includes at least one lane alone which the vehicle can move. Using the planning circuit, a trajectory of the multiple trajectories for the vehicle to travel is selected based on an initial vehicle trajectory of the vehicle. The selected trajectory includes a stem. The stem is a portion of the selected trajectory to which the vehicle is configured to adhere. Using the control circuit, the vehicle is moved along the selected trajectory.

Abstract: Some methods described include: receiving, with at least one processor, a point cloud from a pose graph; receiving, with the at least one processor, an image from a driving log of a vehicle, the image corresponding to the point cloud from the pose graph; obtaining, with the at least one processor, image pixel labels for the image; projecting, with the at least one processor, the point cloud in a point cloud coordinate system to an image coordinate system based on the image pixel labels; generating, with the at least one processor, a six-dimensional colored point cloud by combining the point cloud and color information from the image; and transforming, with the at least one processor, the six-dimensional colored point cloud into five-dimensional map tiles to form a colored map layer. Systems and computer program products are also provided.

Abstract: Provided are methods for location based parameters for an image sensor, which can include determining the geographic location of the vehicle, adjusting the parameters of the image sensor of the vehicle from a first setting to a second setting based on the geographic location of the vehicle, receiving sensor data associated with the image sensor based on the second setting, and processing the sensor data to generate an image. Systems and computer program products are also provided.

Abstract: Techniques are provided for monocular 3D object detection from an image semantics network. An image semantics network (ISN) is a single stage, single image object detection network that is based on single shot detection (SSD). In an embodiment, the ISN augments the SSD outputs to provide encoded 3D properties of the object along with a 2D bounding box and classification scores. For each priorbox, a 3D bounding box is generated for the object using the dimensions and location of the priorbox, the encoded 3D properties and camera intrinsic parameters.

Abstract: Among other things, we describe techniques for automatic annotation of environmental features in a map during navigation of a vehicle. The techniques include receiving, by the vehicle located within an environment, a map of the environment. Sensors of the vehicle receive sensor data and semantic data. The sensor data includes a plurality of features of the environment. A geometric model of a feature of the plurality of features is generated. The feature is associated with a drivable area within the environment. A drivable segment is extracted from the drivable area. The drivable segment is segregated into a plurality of geometric blocks, wherein each geometric block corresponds to a characteristic of the drivable area and the geometric model of the feature includes the plurality of geometric blocks. The geometric model is annotated using the semantic data. The annotated geometric model is embedded within the map.

Abstract: Among other things, a determination is made that intervention in an operation of one or more autonomous driving capabilities of a vehicle is appropriate. Based on the determination, a person is enabled to provide information for an intervention. The intervention is caused in the operation of the one or more autonomous driving capabilities of the vehicle.

Abstract: Provided are methods for enhanced semantic labeling in mapping with a semantic labeling system, which can include receiving, from a LiDAR sensor of a vehicle, LiDAR point cloud information including at least one raw point feature for a point, receiving, from a camera of the vehicle, image data associated with an image captured using the camera, generating at least one rich point feature for the point based on the image data, predicting, using a LiDAR segmentation neural network and based on the at least one raw point feature and the at least one rich point feature, a point-level semantic label for the point, and providing the point-level semantic label to a mapping engine to generate a map based on the point-level semantic label Systems and computer program products are also provided.

Abstract: Embodiments are disclosed for ground plane estimation (GPE) using a LiDAR semantic network. In an embodiment, a method comprises: obtaining a point cloud from a depth sensor of a vehicle operating in an environment; encoding the point cloud; estimating, using a deep learning network with the encoded point cloud as input, a ground plane in the environment; planning a path through the environment based on a drivable area of the estimated ground plane; and operating the vehicle, the vehicle along the path. The deep learning network includes a two-dimensional (2D) convolutional backbone, a detection head for detecting objects and a GPE head for estimating the ground plane. In an embodiment, point pillars are used to encode the point cloud.