Abstract: Localization of a vehicle based at least in part on a map of reflective objects having specular reflectivity. A specular constellation map generation system and method are used to create a mapping of location, orientation, and reflectivity values of reflective objects in a driving surface. A specular constellation localization system and method collect reflection data associated with a driving surface and generate a constellation of specular highlights associated with the reflection data. The constellation of specular highlights can be used in conjunction with a location of a light source and the mapping of reflective object values to determine a pose of the vehicle.

Abstract: Systems and techniques for enabling interaction with physical objects as proxy objects representing virtual objects are provided herein. Virtual reality application data associated with a virtual reality application executed on a virtual reality device, a first virtual reality object data associated with a first virtual reality object from the virtual reality application, and virtual reality event data associated with one or more events from the virtual reality application may be received. Robotic arms including a robotic hand may grasp a first physical object which corresponds to the first virtual reality object of the virtual reality application. Sensors may detect a user interaction with the first physical object. Force feedback instructions commanding the robotic arms to move while maintaining grasp of the first physical object may be generated and executed based on detecting the user interaction with the first physical object and based on the virtual reality event data.

Abstract: A system and method for remapping surface areas of vehicle environment that include receiving a plurality of images of the surface areas of the vehicle environment from a computing device. The system and method also include evaluating the plurality of images to determine an optical flow value associated with at least one pixel of at least two images of the plurality of images. The system and method additionally include determining at least one surface area that includes an external environment of the vehicle. The system and method further include remapping the surface areas by filtering the at least one surface area from which the external environment is captured and rendering at least one surface area graphical augmentation on at least one remapped surface area of the vehicle environment.

Abstract: A computer-implemented method for rendering views to an output device and controlling a vehicle. The method includes determining a maneuver path for the vehicle within a spatial environment around the vehicle based on vehicle data from one or more vehicle systems of the vehicle. The method includes updating a view based on the spatial environment and the maneuver path, by augmenting one or more components of a model to provide a representation of the maneuver path virtually in the view as an available maneuver path or an unavailable maneuver path. The view is rendered to an output device and a vehicle maneuver request is generated based on the maneuver path. Further, the one or more vehicle systems are controlled to execute the vehicle maneuver request.

Abstract: A method for rendering views to an output device in a vehicle, including receiving vehicle data from a portable device, and determining vehicle occupant action data based on the vehicle data associated with a vehicle occupant of the vehicle. The method including generating a view based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes modifying one or more components of the model according to the vehicle occupant action data. The method including rendering the view to the output device by controlling the output device to update display of the view according to the modified one more components of the model.

Abstract: A method for rendering views to an output device in a vehicle, including receiving vehicle data from a first frame of reference. The first frame of reference is defined by the vehicle relative to an exterior environment of the vehicle. The method includes determining vehicle occupant action data based on the vehicle data associated with a vehicle occupant of the vehicle in a second frame of reference different than the first frame of reference. The method including generating a view based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes modifying one or more components of the model according to the vehicle occupant action data. The method including rendering the view to the output device by controlling the output device to update display of the view according to the modified one more components of the model.

Abstract: A system and method providing a shared audio experience that include analyzing at least one audio stream that is associated with at least one application that is executed on at least one portable device. The system and method also include determining a plurality of audio elements associated with the at least one audio stream based on the analysis of the at least one audio stream. The system and method further include controlling at least one audio source to provide the shared audio experience.

Abstract: According to one aspect, driver management is provided. One or more speech segments of a driver of an autonomous vehicle may be recorded. A position, a destination, or a previous destination of a driver or an autonomous vehicle may be tracked. An estimated state of the driver may be determined based on one or more of the speech segments, the position, the destination, or the previous destination, and a calendar event associated with the driver or a passenger of the autonomous vehicle. Autonomous driving features may be automatically enabled, disabled, or operation of the autonomous vehicle may be enabled or disabled in different modes based on the estimated state of the driver. Additionally, notifications may be displayed, rideshare applications may be launched, or warnings may be sent based on the estimated state of the driver.

Abstract: A method for rendering views to an output device, including receiving vehicle data from a portable device. The vehicle data is from a frame of reference of an exterior environment of a vehicle. The method also includes receiving user data from a portable device. The user data includes a health data about a user from a frame of reference of an exterior environment of the vehicle. The method also includes generating a view based on the vehicle data, the user data and a model. The model includes one or more components that define the view. The method further includes determining a difference between the user data and the vehicle data. The method additionally includes rendering the view at a rendering speed to the output device by controlling the output device to update display of the view according to the determined difference. The rendering speed is based on the health data.

Abstract: A method for rendering views to an output device, including receiving vehicle data, the vehicle data associated with a vehicle, and receiving user data including a health data about a user. The method including generating a view based on the vehicle data and the user data. The method including determining a difference between the user data and the vehicle data indicative of a perspective based on motion of the vehicle. The method including rendering the view at a rendering speed to the output device by controlling the output device to update display of the view according to the determined difference, wherein the rendering speed is based on the health data.

Abstract: A method for rendering views to an output device in a vehicle, including receiving vehicle data from one or more vehicle systems of the vehicle, wherein the vehicle data includes predictive vehicle data. The method including generating a view based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes augmenting one or more components of the model according to the vehicle data. The method including rendering the view to the output device by controlling the output device to update display of the view according to the augmented one or more components of the model.

Abstract: A method for virtual reality rendering views to an output device is described. In one embodiment, the method includes receiving vehicle data from a portable device. The vehicle data including a velocity, an orientation, and a location of a vehicle. The method including receiving user data from the portable device. The user data including a velocity, an orientation and a location of a user. The method including generating a view based on the vehicle data, the user data, and the model including one or more components that define the view. The method including determining a difference between the user data and the vehicle data. The method including rendering the view to the output device by controlling the output device to update display of the view according to the determined difference.

Abstract: A method for rendering views to an output device in a vehicle, including receiving vehicle data from a portable device. The vehicle data includes vehicle dynamics data and predictive vehicle data within a frame of reference of an exterior environment of the vehicle. The method also includes generating a view based on the vehicle data and a model. The model includes one or more components that define the view. The method also includes determining a difference between the vehicle dynamics data and the predictive vehicle data indicative of a perspective based on motion of the vehicle within the frame of reference of the exterior environment of the vehicle. The method further includes rendering the view to the output device by controlling the output device to update display of the view according to the model and the determined difference.

Abstract: A system and method providing a shared audio experience that include analyzing at least one audio stream that is associated with at least one application that is executed on at least one portable device. The system and method also include determining a plurality of audio elements associated with the at least one audio stream based on the analysis of the at least one audio stream. The system and method further include controlling at least one audio source to provide the shared audio experience.

Abstract: According to one or more aspects, systems and methods for providing different virtual environment perspectives based on a common virtual environment are described herein. A virtual environment model associated with a virtual environment, including a first virtual object associated with virtual environment coordinates, a first set of functions corresponding to a first virtual environment mode, and a second set of functions corresponding to a second virtual environment mode may be stored on a database. A virtual environment engine may generate a first virtual environment view of the first virtual object based on the first virtual environment mode and generate a second virtual environment view of the first virtual object based on the second virtual environment mode. The first virtual environment view and the second virtual environment view are respectively rendered on different devices in a concurrent fashion.

Abstract: According to one aspect, driver management is provided. One or more speech segments of a driver of an autonomous vehicle may be recorded. A position, a destination, or a previous destination of a driver or an autonomous vehicle may be tracked. An estimated state of the driver may be determined based on one or more of the speech segments, the position, the destination, or the previous destination, and a calendar event associated with the driver or a passenger of the autonomous vehicle. Autonomous driving features may be automatically enabled, disabled, or operation of the autonomous vehicle may be enabled or disabled in different modes based on the estimated state of the driver. Additionally, notifications may be displayed, rideshare applications may be launched, or warnings may be sent based on the estimated state of the driver.

Abstract: The system includes a memory that stores instructions for executing processes converting line drawings to rendered images. The system also includes a processor configured to execute the instructions. The instructions cause the processor to: train a neural network to account for irregularities in the line drawings by introducing noise data into training data of the neural network; receive a first line drawing from an input device; generate a first rendered image based on features identified in the first line drawing; and display the first rendered image on an output device.

Abstract: A conversation recognition system on board a vehicle includes an acoustic sensor component that detects sound in a cabin of the vehicle, a voice recognition component coupled to the acoustic sensor component that analyzes the sound detected by the acoustic sensor component and identifies a plurality of utterances, and a conversation threading unit coupled to the voice recognition component that analyzes the utterances identified by the voice recognition component and identifies a plurality of conversations between a plurality of occupants of the vehicle. The conversation recognition system enables multiple conversations in an environment to be recognized and distinguished from each other.

Abstract: A computer-implemented method for rendering views to an output device in a vehicle that is restricted to a fixed track that has a predetermined fixed path. Vehicle data about the vehicle is received from an off-vehicle device that is located remotely from the vehicle and along the predetermined fixed path. The vehicle data includes a velocity, an orientation, and a location of the vehicle. User data about a user in a vehicle is received that includes a velocity, an orientation, and a location. The method includes generating a view based on the vehicle data, the user data, and a model. The model includes one or more components that define the view. The method includes determining a difference between the vehicle data and the user data, and rendering the view to the output device by controlling the output device to display the view according to the determined difference.

Abstract: Systems and methods for designing a virtual platform based on user inputs. The system includes a memory that stores instructions for executing processes for designing a virtual platform based on user inputs. The system also includes a processor configured to execute the instructions. The instructions cause the processor to: receive, via an input device, a plurality of user inputs; generate the virtual platform, wherein successful inputs of the virtual platform are based on the plurality of user inputs; and store the virtual platform on a storage device such that a user may utilize the virtual platform on a computing device.