Abstract: This application is directed to predicting vehicle actions according to a hierarchy of interconnected vehicle actions. The hierarchy of interconnected vehicle actions includes a plurality of predefined vehicle actions that are organized to define a plurality of vehicle action sequences. A first vehicle obtains one or more images of a road and a second vehicle, and predicts a sequence of vehicle actions of the second vehicle through the hierarchy of interconnected vehicle actions using the one or more images. The first vehicle is controlled to drive at least partially autonomously based on the predicted sequence of vehicle actions of the second vehicle. In some embodiments, the hierarchy of interconnected vehicle actions includes a first action level that is defined according to a stage of a trip and corresponds to three predefined vehicle actions of: “start a trip,” “move in a trip,” and “complete a trip.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining parameters for an environment based on a regenerable elevation index; generating a speed profile for the environment based on the parameters; and generating control actions for control of a vehicle in the environment based on the speed profile.

Abstract: Techniques for assembling an integrated assembly are provided. An example method can include injecting a polymer into a mold of a housing base. The method can further include cooling the mold to form the housing base, the housing base forming a compartment. The method can further include arranging a first sensor in the compartment, the first sensor to be connected to the housing base. The method can further include arranging a light source in the compartment inward from the first sensor and the first opening. The method can further include sealing the first opening of the housing base with a cover plate, the cover plate comprising a second opening for permitting a light from the light source to be emitted to form an integrated sensor assembly.

Abstract: An integrated sensor assembly is described. The integrated sensor assembly can include a housing arranged on a side surface of a cabin of an autonomous vehicle and a threshold elevation from a reference surface. The integrated sensor assembly can further include a first sensor arranged in a compartment of the housing. The integrated sensor assembly can further include an arm connected at a first end to the housing and connected at a second end to the side surface of the cabin. The integrated sensor assembly can further include an actuator for rotating the arm between a first state and a second state, the arm and the housing arranged externally to the side surface in the first state and the second state. The integrated sensor assembly can further include a light source arranged in the compartment.

Abstract: This application is directed to augmenting training images used for generating a model for monitoring vehicle drivers. A computer system obtains a first image of a first driver in an interior of a first vehicle and separates, from the first image, a first driver image from a first background image of the interior of the first vehicle. The computer system obtains a second background image and generates a second image by overlaying the first driver image on the second background image. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for monitoring vehicle drivers. In some embodiments, at least one of the first driver image and the second background image is adjusted to match lighting conditions, average intensities, and sizes of the first driver image and the second background image.

Abstract: Techniques are described for determining failover to a redundant braking system. An example method can include determining a fault occurring at a first braking system or a second braking system of a primary braking system of an autonomous vehicle operating in an autonomous mode, and the autonomous vehicle using the first braking system. The method can further include switching from using the first braking system in the autonomous mode if the fault is determined at the first braking system. The method can further include transmitting to an alert system of the autonomous vehicle, a request to generate an alert for switching to a manual control mode of the autonomous vehicle based at least in part on determining the fault. The method can further include deactivating the autonomous mode of the autonomous vehicle upon determining that the autonomous vehicle is in the manual control mode.

Abstract: The present teaching relates to different configurations for facilitating calibration of multiple sensors of different types. A plurality of fiducial markers are arranged in space for simultaneously calibrating multiple sensors of different types. Each of the plurality of fiducial markers has a feature point thereon and is provided to enable the multiple sensors to calibrate by detecting the features points and estimating their corresponding 3D coordinates with respect to respective coordinate systems of the multiple sensors.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising detecting an object in sensor data captured in an environment; detecting a marker in the sensor data captured in the environment; and determining a first location of the object in the environment based on a relative location of the object to the marker and a second location of the marker.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining first speeds of first objects in a first lane and second speeds of second objects in a second lane; determining a first speed of traffic for the first lane based on the first speeds and a second speed of traffic for the second lane based on the second speeds; and generating a speed limit for a third lane based on the first speed of traffic and the second speed of traffic.

Abstract: A method comprises: receiving, at a processor, a first image from a first camera from a stereo camera pair and a second image from a second camera from the stereo camera pair. The method also includes determining, at the processor using a machine learning model, a first set of objects in the first image. The processor determines an object type. The processor identifies a second set of objects in the second image associated with the first plurality of objects. The method also includes calculating, at the processor, a set of disparity values between the first image and the second image based on (1) an object from the first set of objects, (2) an object from the second set of objects and associated with the object from the first set of objects, and (3) an object type of the object from the first set of objects.

Abstract: The present teaching relates to method, system, medium, and implementations for sensor calibration. An ego vehicle determines whether a sensor deployed on the ego vehicle to facilitate autonomous driving of the ego vehicle needs to be calibrated and sends, if it is determined that the sensor needs to be calibrated, a request for assistance in collaborative calibration of the sensor, with a first position of the ego vehicle or a first configuration of the sensor with respect to the ego vehicle. When a response of the request is received, an assisting vehicle is indicated to travel to be near the ego vehicle to facilitate the calibration of the sensor by collaborating with the moving ego vehicle and the ego vehicle coordinates with the assisting vehicle to enable the sensor to acquire information of a target present on the assisting vehicle for the collaborative calibration of the sensor.

Abstract: This application is directed to aerial view generation for vehicle control. A vehicle obtains a forward-facing view of a road captured by a front-facing camera of a vehicle and applies a machine learning model to process the forward-facing view to predict determine a trajectory of the vehicle and a road layout based on a Frenet-Serret coordinate system of the road for the vehicle. The trajectory of the vehicle is combined with the road layout to predict an aerial view of the road, and the aerial view of the road is used to at least partially autonomously drive the vehicle. In some embodiments, the machine learning model is applied to process the forward-facing view to determine a first location of an obstacle vehicle in the Frenet-Serret coordinate system. The obstacle vehicle is placed on the aerial map based on the first location of the obstacle vehicle.

Abstract: Techniques are described for determining failover to a redundant braking system. An example method can include a system determining a fault occurring in a braking system of an autonomous vehicle (AV) operating in an autonomous mode, the braking system comprising a first braking system and a second braking system, the fault occurring in at least one of the first braking system and the second braking system, and the AV using the first braking system. The system can further determine whether to switch from using the first braking system based at least in part on determining the fault. The system can further transmit to an alert system of the AV, a request to generate an alert for manual control of the AV based at least in part on determining the fault. The system can further deactivate the autonomous mode of the AV based at least in part on determining the fault.

Abstract: An assembly is described. An assembly can include a housing comprising a first compartment and a plurality of vents letting air flow into and out. The assembly can further include a first ECU positioned within the first compartment, the first ECU comprising a first inlet letting air flow into the first ECU. The assembly can further include a second ECU positioned within the first compartment, the second ECU comprising a second inlet letting air flow into the second ECU. The assembly can further include a first plenum positioned between the first ECU and the second ECU and comprising a second compartment. The assembly can further include a first air duct forming a first channel letting air from a first vent of the plurality of vents into the second compartment, wherein the first plenum further comprises a second vent positioned to align with the first inlet of the first ECU.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining an image of an environment. A drivable area and boundary information associated with the environment are determined based on the image. At least one boundary for navigation of the environment is generated based on the drivable area and boundary information.

Abstract: A safety case discovery system includes a scenario framework and safety protocols for edge cases. The safety case discovery system receives sensor data generated by at least one sensor during operation of a vehicle and stores the sensor data in a data warehouse. The data warehouse can be queried based on a predefined scenario description to produce a subset of records which are ranked based on a relevancy of the records to the predefined scenario description. The safety case discovery system deduplicates the ranked results to produce edge cases and updates the safety protocol. The safety case discovery system can train a machine learning model vehicle based on the edge cases, to produce a trained machine learning model for optimizing the driving system.

Abstract: This application is directed to augmenting training data used for vehicle driving modelling. A computer system obtains a first image of a road and identifies a drivable area of the road within the first image. The computer system obtains an image of an object and generates a second image from the first image by overlaying the image of the object over the drivable area. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for facilitating driving of a vehicle (e.g., at least partial autonomously). In some embodiments, the computer system applies machine learning to train a model using the corpus of training images and distributes the model to one or more vehicles. In use, the model processes road images captured by the one or more vehicles to facilitate vehicle driving.

Abstract: A computer system obtains a plurality of road images captured by one or more cameras attached to one or more vehicles. The one or more vehicles execute a model that facilitates driving of the one or more vehicles. For each road image of the plurality of road images, the computer system determines, in the road image, a fraction of pixels having an ambiguous lane marker classification. Based on the fraction of pixels, the computer system determines whether the road image is an ambiguous image for lane marker classification. In accordance with a determination that the road image is an ambiguous image for lane marker classification, the computer system enables labeling of the image and adds the labeled image into a corpus of training images for retraining the model.

Abstract: Techniques are described for providing a hands-off steering wheel detection warning. An example method can include a vehicle computer determining a real-time level of fatigue of a driver of an autonomous vehicle. The vehicle computer can determine an operating parameter associated with an environment in which the autonomous vehicle is traveling. The vehicle computer can generate, using a machine learning model, a predicted driving pattern of a second vehicle traveling in the environment. The vehicle computer can determine a time interval for providing a hands-off steering wheel detection warning based at least in part on the real-time level of fatigue of the driver, the operating parameter, and the predicted driving pattern of the second vehicle. The vehicle computer can identify a final time interval for providing a hands-off steering wheel detection warning. The vehicle computer can output the hands-off steering wheel detection warning after the final time interval has elapsed.

Abstract: Embodiments provide a vehicle computer coupled to a vehicle. The vehicle computer may be configured to receive sensor data from a plurality of sensors of the autonomous vehicle, analyze the sensor data, predict a need for a steering torque overlay based on analyzing the sensor data, determine a steering torque overlay output based on at least the sensor data, provide the steering torque overlay output to a steering system of the vehicle, and control the steering system in view of the steering torque overlay output. The steering torque overlay output is provided to the steering system while the vehicle is transitioning from an autonomous mode or a manual mode to a hybrid mode where the steering system is controlled simultaneously by the vehicle computer and a driver when the steering torque overlay output is applied.