Abstract: The disclosure includes a system and method for wireless data sharing between a mobile client device and a three-dimensional heads-up display unit. A system may include a three-dimensional heads-up display unit (“3D HUD”) installed in a vehicle. The system may include a memory storing instructions that, when executed, cause the system to: establish a peer-to-peer video stream between a mobile client device and the vehicle; generate live video data for providing a video stream for causing a screen of the mobile client device to display visual content of the 3D HUD that includes substantially live images depicting what the driver of the vehicle sees when looking at the 3D HUD; and stream the live video data to the mobile client device to cause the screen of the mobile client device to display the video stream that depicts what the driver of the vehicle sees when looking at the 3D HUD.

Abstract: This disclosure describes, according to some implementations, a system and method for adapting object handover from robot to human using perceptual affordances. In an example method, upon receiving sensor data describing surroundings and/or operational state of a robot unit from a sensor, the method calculates a probability of a perceptual classification based on the sensor data. The perceptual classification may be one or more of an environment classification, an object classification, a human action classification, or an electro-mechanical state classification. The method further calculates an affordance of the probability of the perceptual classifier using a preference model, determines a handover action based on the affordance, executes the handover action, and updates the preference model based on feedback.

Abstract: The disclosure includes a method that includes assigning a classification to a travel route followed by a first client device based on data associated with when the first client device followed the travel route. The method may further include recommending the travel route to a second client device based on a request from the second client device for a desired travel route with the classification.

Abstract: The disclosure includes methods for determining a current location for a user in an environment; detecting obstacles within the environment; estimating one or more physical capabilities of the user based on an EHR associated with the user; generating, with a processor-based device that is programmed to perform the generating, instructions for a robot to perform a task based on the obstacles within the environment and one or more physical capabilities of the user; and instructing the robot to perform the task.

Abstract: The disclosure includes a method and a robot for projecting the one or more graphics on the surface of a user environment. The method may include determining robot velocity data describing one or more of an angular velocity and a linear velocity of the robot. The method may include determining graphical data describing one or more graphics for projection on the surface of a user environment based on the angular velocity and the linear velocity of the robot. The one or more graphics may include one or more arrows describing a past behavior of the robot and a pending behavior of the robot. The method may include projecting the one or more graphics on the surface of the user environment so that a user present in the user environment with the robot may have access to information describing the past and pending behavior of the robot.

Abstract: Technology for localized guidance of a body part of a user to specific objects within a physical environment using a vibration interface is described. An example system may include a vibration interface wearable on an extremity by a user. The vibration interface includes a plurality of motors. The system includes sensor(s) coupled to the vibrotactile system and a sensing system coupled to the sensor(s) and the vibration interface. The sensing system is configured to analyze a physical environment in which the user is located for a tangible object using the sensor(s), to generate a trajectory for navigating the extremity of the user to the tangible object based on a relative position of the extremity of the user bearing the vibration interface to a position of the tangible object within the physical environment, and to guide the extremity of the user along the trajectory by vibrating the vibration interface.

Abstract: The disclosure includes a system and method for providing navigation guidance. The system includes a tracking application and a user. The tracking application determines a location of a user, determines a position of at least one object in proximity to the user, determines a path for the user to avoid the at least one object, generates linear and angular velocities that correspond to the path and generates directions that correspond to the linear and angular velocities. The user can receive the directions as audio instructions or via a tactile belt with motors that vibrate to convey the directions.

Abstract: The disclosure includes a system and method for using a robot to simulate user motions by detecting a user, estimating a first position and orientation of the user, determining a current position of the robot, generating an initial path from the current position of the robot to a goal position based on whether the robot affects the user while travelling on the initial path, determining whether the user will traverse in response to the robot when the robot travels on the initial path, responsive to the user traversing in response to the robot, estimating a second position where the user will move when the robot is near, and generating a new path from the current position of the robot to the goal position.

Abstract: The disclosure includes a system and method for determining posture data for a user based on observation of the user, generating user model data describing one or more user models, determining user preferences data describing the preferences of the user for receiving an object, generating two or more reach simulations that simulate the user reaching for different points inside a user environment to receive the object, each reach simulation resulting in a hand of the user arriving at a three-dimensional point in the user environment, analyzing the reach simulations to assign a value to each three-dimensional point in the user environment, grouping the three-dimensional points into one or more clusters of similar points to form candidate zones including one or more points inside the user environment where a robot can place the object for handover to the user, and ranking the candidate zones.

Abstract: The disclosure includes a system and method for providing a heads-up display in a vehicle for alerting users to roadway objects. The system includes a processor and a memory storing instructions that, when executed, cause the system to: monitor, with the one or more processors, for a roadway object; detect, with the one or more processors, the roadway object; generate graphical data configured to emphasize the roadway object on a heads-up display; determine whether a user sees the roadway object; responsive to the user failing to see the roadway object, determine whether the roadway object is critical; and responsive to the roadway object being critical, enhance a graphic output on the heads-up display to get the user's attention.

Abstract: The disclosure includes a method and a robot for projecting the one or more graphics on the surface of a user environment. The method may include determining robot velocity data describing one or more of an angular velocity and a linear velocity of the robot. The method may include determining graphical data describing one or more graphics for projection on the surface of a user environment based on the angular velocity and the linear velocity of the robot. The one or more graphics may include one or more arrows describing a past behavior of the robot and a pending behavior of the robot. The method may include projecting the one or more graphics on the surface of the user environment so that a user present in the user environment with the robot may have access to information describing the past and pending behavior of the robot.

Abstract: The disclosure includes a system and method for determining a robot path based on user motion by determining a current position of a robot with a processor-based computing device programmed to perform the determining, receiving sensor readings on positions, directions, and velocities of a visually-impaired user and other users, generating a model of the motions of the visually-impaired user and the other users, the model including a user path for the visually-impaired user and a robot path for the robot, generating a collision prediction map to predict collisions between at least one of the robot, the visually-impaired user, and the other users, determining whether there is a risk of collision for either the visually-impaired user or the robot, and responsive to the risk of collision, updating at least one of the user path and the robot path.

Abstract: Technology for localized guidance of a body part of a user to specific objects within a physical environment using a vibration interface is described. An example system may include a vibration interface wearable on an extremity by a user. The vibration interface includes a plurality of motors. The system includes sensor(s) coupled to the vibrotactile system and a sensing system coupled to the sensor(s) and the vibration interface. The sensing system is configured to analyze a physical environment in which the user is located for a tangible object using the sensor(s), to generate a trajectory for navigating the extremity of the user to the tangible object based on a relative position of the extremity of the user bearing the vibration interface to a position of the tangible object within the physical environment, and to guide the extremity of the user along the trajectory by vibrating the vibration interface.

Abstract: The disclosure includes methods for determining a current location for a user in an environment; detecting obstacles within the environment; estimating one or more physical capabilities of the user based on an EHR associated with the user; generating, with a processor-based device that is programmed to perform the generating, instructions for a robot to perform a task based on the obstacles within the environment and one or more physical capabilities of the user; and instructing the robot to perform the task.

Abstract: The disclosure includes a system and method for spatial information for a heads-up display. The system includes a processor and a memory storing instructions that, when executed, cause the system to: receive sensor data about an entity, assign the entity to a category, estimate a danger index for the entity based on vehicle data, category data, and a position of the entity, generate entity data that includes the danger index, identify a graphic that is a representation of the entity based on the entity data, determine a display modality for the graphic based on the danger index, and position the graphic to correspond to a user's eye frame.

Abstract: The disclosure includes a speech-enabled device and method to share speech dialog capabilities of the speech-enabled device with a dumb device. The speech-enabled device includes a processor and a memory storing instructions that, when executed by the processor, cause the speech-enabled device to: receive speech dialog data of the dumb device that indicates a function of the dumb device; receive speech input; determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data; generate a command effective to invoke the function of the dumb device based on the speech dialog data; and send the command to the dumb device to invoke the function of the dumb device.

Abstract: The disclosure includes a system, method and tangible memory for providing a lane quality service. The system includes a back-up camera mounted to the exterior of a vehicle. The system also includes one or more vibration sensors mounted configured to measure sensor data describing a roadway impairment for a current lane of a roadway. The system also includes an onboard vehicle computer including a lane quality application stored in a tangible memory and a processor. The lane quality application, responsive to being executed by the processor, causes the processor to execute steps including: determining a capture time for executing the back-up camera; executing the back-up camera at the capture time to capture image data describing an image of the roadway impairment; determining location data describing a current geographic location of the roadway impairment; and creating a log that describes the sensor data, the image data and the location data.

Abstract: The disclosure includes a system and method for using a robot to prevent risk to a user by identifying a point of interest, determining whether the point of interest is a risk for the user with visual impairments based on the user's position, responsive to a first predetermined threshold for distance being met, determining where to move the robot to mitigate the risk to the user, and instructing the robot to move to a new location. The robot may move between the user and the point of interest to prevent the user from colliding with the point of interest. The robot may use an auditory or tactile warning to warn the user. The tactile warning may be transmitted to a walking stick or a belt worn by the user.

Abstract: The disclosure includes a method that includes assigning a classification to a travel route followed by a first client device based on data associated with when the first client device followed the travel route. The method may further include recommending the travel route to a second client device based on a request from the second client device for a desired travel route with the classification.

Abstract: The disclosure includes a system and method for providing a heads-up display in a vehicle for alerting users to roadway objects. The system includes a processor and a memory storing instructions that, when executed, cause the system to: monitor, with the one or more processors, for a roadway object; detect, with the one or more processors, the roadway object; generate graphical data configured to emphasize the roadway object on a heads-up display; determine whether a user sees the roadway object; responsive to the user failing to see the roadway object, determine whether the roadway object is critical; and responsive to the roadway object being critical, enhance a graphic output on the heads-up display to get the user's attention.