Abstract: A method for multi-view dataset formation from fleet data is described. The method includes detecting at least a pair of vehicles within a vicinity of one another, and having overlapping viewing frustums of a scene. The method also includes triggering a capture of sensor data from the pair of vehicles. The method further includes synchronizing the sensor data captured by the pair of vehicles. The method also includes registering the sensor data captured by the pair of vehicles within a shared coordinate system to form a multi-view dataset of the scene.

Abstract: Systems and methods for training a policy are disclosed. In one example, a system includes a processor and a memory with instructions that cause the processor to train the policy using a training data set with training scenes to generate an identification policy and perform a closed-loop simulation on the identification policy to collect closed-loop metrics. Based on the closed-loop metrics, the instructions cause the processor to construct an error set of the training scenes and construct an upsampled training set by upsampling the error set. After that, the policy is trained using the upsampled training set to generate a final policy.

Abstract: Systems and methods for learning and managing robot user interfaces are disclosed herein. One embodiment generates, based on input data including information about past interactions of a particular user with a robot and with existing HMIs of the robot, a latent space using one or more encoder neural networks, wherein the latent space is a reduced-dimensionality representation of learned behavior and characteristics of the particular user, and uses the latent space as input to train a decoder neural network associated with (1) a new HMI distinct from the existing HMIs or (2) a particular HMI among the existing HMIs to alter operation of the particular HMI. The trained first decoder neural network is deployed in the robot to control, at least in part, operation of the new HMI or the particular HMI in accordance with the learned behavior and characteristics of the particular user.

Abstract: Systems and methods for use of operator state conditions to generate, adapt or otherwise produce visual signals relaying information to the operator are disclosed. A monitoring system may observe and analyze a vehicle operator to determine an operator state. The monitoring system may transmit the observed driver state to an operator alert system that generates, conditions and controls the transmission of signals to the operator.

Abstract: A control system, computer-readable storage medium and method of preventing occlusion of and minimizing shadows on the driver's face for driver monitoring. The system includes a steering wheel, a plurality of fiberscopes arranged evenly spaced around the steering wheel, and one or more video cameras arranged at remote ends of the plurality of fiberscopes. Distal ends of the fiberscopes emerge to a surface of the steering wheel through holes that are perpendicular to an axis of rotation of the steering wheel. Each of the distal ends of the fiberscopes includes a lens. The system includes a plurality of light sources and an electronic control unit connected to the one or more video cameras and the light sources.

Abstract: Disclosed are systems and methods for training at least one policy using a framework for encoding human behaviors and preferences in a driving environment. In one example, the method includes the steps of setting parameters of rewards and a Markov Decision Process (MDP) of the at least one policy that models a simulated human driver of a simulated vehicle and an adaptive human-machine interface (HMI) system configured to interact with each other and training the at least one policy to maximize a total reward based on the parameters of the rewards of the at least one policy.

Abstract: Embodiments described herein include systems and methods for predicting a transfer of control of a vehicle to a driver. A method includes receiving information about an environment of the vehicle, identifying at least one condition represented in the information about the environment of the vehicle that corresponds to at least one of one or more known conditions that lead to a handback of operational control of the vehicle to the driver, and predicting the transfer of control of the vehicle to the driver based on the at least one condition identified from the information about the environment of the vehicle.

Abstract: Systems and methods for detecting occluded objects are disclosed. In one embodiment, a method of determining a shape and pose of an object occluded by an occlusion object includes receiving, by a generative model, a latent vector, and iteratively performing an optimization routine until a loss is less than a loss threshold. The optimization routine includes generating, by the generative model, a predicted object having a shape and a pose from the latent vector, generating a predicted shadow cast by the predicted object, calculating the loss by comparing the predicted shadow with an observed shadow, and modifying the latent vector when the loss is greater than the loss threshold. The method further includes selecting the predicted object as the object when the loss is less than the loss threshold.

Abstract: A vehicular warning system control system can include a processor and a memory in communication with the processor. The memory can include a warning system control module having instructions that, when executed by the processor, cause the processor to detect, using sensor data having information about a gaze of each eye of a driver of a vehicle, an abnormality of a gaze of the driver. The instructions further cause the processor to modify, using the sensor data, a signal emitted by the vehicle when the abnormality is detected.

Abstract: Systems and methods are provided for an advanced driver-assistance system (ADAS) that obtains data from a plurality of sensors. In some embodiments, the system can retrieve data regarding a user’s past interactions and analyze the data with the sensor data to determine the user’s behavior. In some embodiments, the ADAS can determine whether a user is unaware of an ADAS feature based on this behavior and a prompt that recommends the ADAS feature. The user’s response to this prompt may be incorporated into the user’s behavior for future recommendations.

Abstract: Extended reality content in a video can be varied based on a risk level of an external environment of a vehicle. The video can be of an external environment of a vehicle can be presented in real-time on a display located within the vehicle. The display can be a video pass through display. The display can be an in-vehicle display, or it can be a part of a video pass-through extended reality headset. The video can present a view of an external environment of the vehicle as well as extended reality content. A risk level of the external environment of the vehicle can be determined. An amount of the extended reality content presented in the video can be varied based on the risk level.

Abstract: Extended reality content in a video can be varied based on driver attentiveness. The video can be of an external environment of a vehicle and can be presented in real-time on a display located within the vehicle. The display can be a video pass through display. The display can be an in-vehicle display, or it can be a part of a video pass-through extended reality headset. The video can present a view of an external environment of the vehicle as well as extended reality content. A level of attentiveness of a driver of the vehicle can be determined. An amount of the extended reality content presented in the video can be varied based on the level of attentiveness.

Abstract: Systems and methods for training a model are described herein. In one example, a system for training the model includes a processor and a memory in communication with the processor having a training module. The training module has instructions that cause the processor to determine a contrastive loss using a self-supervised contrastive loss function, adjust, based on the contrastive loss, model weights a visual backbone that generated feature maps and/or a textual backbone that generated feature vectors. The training module also has instructions that cause the processor to determine a localized loss using a supervised loss function that compares an image-caption attention map with visual identifiers and adjust, based on the localized loss, the model weights the visual backbone and/or the textual backbone.

Abstract: A system for predicting a hazardous event from road-scene data includes an electronic control unit configured to implement a neural network and a camera communicatively coupled to the electronic control unit, wherein the camera generates the road-scene data. The electronic control unit is configured to receive the road-scene data from the camera, and predict, with the neural network, an occurrence of the hazardous event within the road-scene data from the camera.

Abstract: A control system, computer-readable storage medium and method of preventing occlusion of and minimizing shadows on the driver's face for driver monitoring. The system includes a steering wheel, a plurality of fiberscopes arranged evenly spaced around the steering wheel, and one or more video cameras arranged at remote ends of the plurality of fiberscopes. Distal ends of the fiberscopes emerge to a surface of the steering wheel through holes that are perpendicular to an axis of rotation of the steering wheel. Each of the distal ends of the fiberscopes includes a lens. The system includes a plurality of light sources and an electronic control unit connected to the one or more video cameras and the light sources.

Abstract: A method for scenario-based event triggers is described. The method includes generating, by a first machine-learning (ML) model, feature vectors encoding driving scenarios surrounding an ego vehicle. The method also includes detecting, by a second machine-learning (ML) model, a unique driving scenario outside of pre-programmed event triggers corresponding to one of the feature vectors encoding driving scenarios surrounding the ego vehicle. The method further includes triggering uploading of the unique driving scenario outside of pre-programmed event triggers to a central scenario-based event control server.

Abstract: Systems and methods for an intelligent camera system are provided. A method includes receiving, from a first camera in a vehicle, view data corresponding to an area from a vantage point of the vehicle. The method further includes detecting a region of interest from the view data provided by the first camera. The method also includes providing the region of interest to a second camera in the vehicle. The method further includes receiving, from the second camera, zoom view data corresponding to a zoom view of the region of interest.

Abstract: Systems, vehicles and methods for determining wrong direction driving are disclosed. In one embodiment, a system for determining a vehicle traveling in a wrong direction includes one or more sensors that produce sensor data, one or more processors, and one or more non-transitory computer-readable medium storing computer readable-instructions. When the computer-readable instructions are executed by the one or more processors, the computer-readable instructions cause the one or more processors to determine one or more lanes within a roadway using the sensor data, determine a direction of travel of the one or more lanes using the sensor data, and identify a non-compliant vehicle traveling in a direction in the one or more lanes that is different from the determined direction of travel in the one or more lanes.

Abstract: A method may include presenting an image capture user interface on a display device of a multipurpose device including a live view portion configured to display a live view of image data currently sensed by an image capture device of the multipurpose device and a recall portion configured to display a thumbnail preview of stored image data most recently captured by the image capture device; receiving an indication of a first touch input on the display device, the first touch input starting at a first position on or near the recall portion of the user interface; receiving an indication of a swiping gesture from the recall portion to a second position with contact being maintained of the first touch input; and in response to the first touch input, displaying an enlarged preview of the stored image data, the enlarged preview being larger in size than the thumbnail preview.

Abstract: A control system, computer-readable storage medium and method of preventing occlusion of and minimizing shadows on the driver's face for driver monitoring. The system includes a steering wheel, a plurality of fiberscopes arranged evenly spaced around the steering wheel, and one or more video cameras arranged at remote ends of the plurality of fiberscopes. Distal ends of the fiberscopes emerge to a surface of the steering wheel through holes that are perpendicular to an axis of rotation of the steering wheel. Each of the distal ends of the fiberscopes includes a lens. The system includes a plurality of light sources and an electronic control unit connected to the one or more video cameras and the light sources.