Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: An artificial reality system is described that implements adaptive degrees-of-freedom (DOF) selection when tracking frames of reference and rendering artificial reality content. In one example, the artificial reality system comprises a head mounted display (HMD) that outputs rendered artificial reality content. A performance monitor determines one or more performance indicators associated with the artificial reality system. A degree-of-freedom (DOF) selector applies one or more policies to the performance indicators to select between a first mode in which a pose tracker computes one or more poses of the HMD within the 3D environment using 6DOF and a second mode in which the pose tracker computes the one or more poses using 3DOF. The pose tracker computes the one or more poses for the HMD within the 3D environment in accordance with the selected mode. A rendering engine renders the content for the artificial reality application based on the computed pose.

Abstract: An artificial reality system is described that implements adaptive degrees-of-freedom (DOF) selection when tracking frames of reference and rendering artificial reality content. In one example, the artificial reality system comprises a head mounted display (HMD) that outputs rendered artificial reality content. A performance monitor determines one or more performance indicators associated with the artificial reality system. A degree-of-freedom (DOF) selector applies one or more policies to the performance indicators to select between a first mode in which a pose tracker computes one or more poses of the HMD within the 3D environment using 6DOF and a second mode in which the pose tracker computes the one or more poses using 3DOF. The pose tracker computes the one or more poses for the HMD within the 3D environment in accordance with the selected mode. A rendering engine renders the content for the artificial reality application based on the computed pose.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: Systems and techniques for sign based localization are provided herein. Sign based localization may be achieved by capturing an image of an operating environment around a vehicle, extracting one or more text candidates from the image, detecting one or more line segments within the image, defining one or more quadrilateral candidates based on one or more of the text candidates, one or more of the line segments, and one or more intersections of respective line segments, determining one or more sign candidates for the image based on one or more of the quadrilateral candidates and one or more of the text candidates, matching one or more of the sign candidates against one or more reference images, and determining a location of the vehicle based on a match between one or more of the sign candidates and one or more of the reference images.

Abstract: Systems and techniques for sign based localization are provided herein. Sign based localization may be achieved by capturing an image of an operating environment around a vehicle, extracting one or more text candidates from the image, detecting one or more line segments within the image, defining one or more quadrilateral candidates based on one or more of the text candidates, one or more of the line segments, and one or more intersections of respective line segments, determining one or more sign candidates for the image based on one or more of the quadrilateral candidates and one or more of the text candidates, matching one or more of the sign candidates against one or more reference images, and determining a location of the vehicle based on a match between one or more of the sign candidates and one or more of the reference images.

Abstract: One or more embodiments of techniques or systems for creating a road marking classification template and vehicle localization using road markings are provided herein. A road marking classification template database includes templates of training images taken from different navigation environments. A training image with a road marking can be rectified and enhanced. Corners can be calculated for the road marking using the rectified or enhanced image. Locations can be determined for the corners and stored as part of a template. Similarly, a runtime image can also be rectified, enhanced, boosted, etc. Additionally, corners can be calculated for the runtime image and matched against corners of templates from the template database. A location or a pose of a vehicle can be determined using the match. In this way, vehicle localization is provided such that drift or other issues associated with GPS, such as occlusion, are mitigated, for example.

Abstract: A system and method for mapping, localization and pose correction including, determining a current position of a vehicle along a travel route and a set of currently observable landmarks along the travel route relative to the current position, the set of currently observable landmarks extracted from one or more stereo images obtained from an imaging device, and querying a survey landmark database to identify a subset of surveyed landmarks relative to the current position of the vehicle. The method including determining one or more two-dimensional transform estimates between the set of currently observable landmarks and the subset of surveyed landmarks and identifying a best transform estimate from the one or more two-dimensional transform estimates that minimizes distances between the set of currently observable landmarks and the subset of surveyed landmarks. The method including correcting a pose of the vehicle based on the best transform estimate.

Abstract: A system and method for mapping, localization and pose correction including, determining a current position of a vehicle along a travel route and a set of currently observable landmarks along the travel route relative to the current position, the set of currently observable landmarks extracted from one or more stereo images obtained from an imaging device, and querying a survey landmark database to identify a subset of surveyed landmarks relative to the current position of the vehicle. The method including determining one or more two-dimensional transform estimates between the set of currently observable landmarks and the subset of surveyed landmarks and identifying a best transform estimate from the one or more two-dimensional transform estimates that minimizes distances between the set of currently observable landmarks and the subset of surveyed landmarks. The method including correcting a pose of the vehicle based on the best transform estimate.

Abstract: A system and method for mapping, localization and pose correction including, determining a current position of a vehicle along a travel route and a set of currently observable landmarks along the travel route relative to the current position, the set of currently observable landmarks extracted from one or more stereo images obtained from an imaging device, and querying a survey landmark database to identify a subset of surveyed landmarks relative to the current position of the vehicle. The method including determining one or more two-dimensional transform estimates between the set of currently observable landmarks and the subset of surveyed landmarks and identifying a best transform estimate from the one or more two-dimensional transform estimates that minimizes distances between the set of currently observable landmarks and the subset of surveyed landmarks. The method including correcting a pose of the vehicle based on the best transform estimate.

Abstract: A system and method are provided for learning part-based object models during a learning phase from training images and applying the learned object models to an input image during runtime. The learned part-based object models are augmented by appearance-based models of the objects. The part-based object models correspond to the shapes of the parts of an object. The appearance-based models provide additional appearance cues to the object models for object classification. The approach to learning part-based object models has the capability of learning object models without using viewpoint labels of the objects. The learning is also invariant to scale and in-plane rotation of the objects.

Abstract: A system and method for mapping, localization and pose correction including, determining a current position of a vehicle along a travel route and a set of currently observable landmarks along the travel route relative to the current position, the set of currently observable landmarks extracted from one or more stereo images obtained from an imaging device, and querying a survey landmark database to identify a subset of surveyed landmarks relative to the current position of the vehicle. The method including determining one or more two-dimensional transform estimates between the set of currently observable landmarks and the subset of surveyed landmarks and identifying a best transform estimate from the one or more two-dimensional transform estimates that minimizes distances between the set of currently observable landmarks and the subset of surveyed landmarks. The method including correcting a pose of the vehicle based on the best transform estimate.

Abstract: The prevention of vehicle accidents is targeted. A road texture model is created based on a vehicle camera image. An initial vehicle location estimate is determined, and map imagery is obtained based on this location estimate. A refined vehicle location is determined using visual egomotion. In particular, 3D features of the vehicle image and the retrieved map imagery are identified and aligned. A map image is selected based on this alignment, and the location associated with the map image is modified by a displacement between the selected map image and the vehicle image to produce a refined vehicle location. A road boundary model is created based on the road texture model and the refined vehicle location, and a road departure model is created based on the road boundary model and vehicle odometry information. The operator of the vehicle is warned of a road departure based on the road departure model.

Abstract: A system and method are disclosed for detecting road marking in a video using learned road marking templates. The system comprises a template learning module configured to learn the feature-based road marking templates from a set of training images. The template learning module is configured to rectify each training image, detect multiple regions of interest, and for each detected region of interest, detect multiple key points. The template learning module extracts feature vectors for the detected key points and builds the road marking templates from the feature vectors. The system also includes a road marking detection module for detecting road markings in a video at runtime using the learned road marking templates. During runtime, these templates are matched using a two-step process of first selecting promising feature matches and subsequently performing a structural matching to account for the shape of the road markings.

Abstract: One or more embodiments of techniques or systems for creating a road marking classification template and vehicle localization using road markings are provided herein. A road marking classification template database includes templates of training images taken from different navigation environments. A training image with a road marking can be rectified and enhanced. Corners can be calculated for the road marking using the rectified or enhanced image. Locations can be determined for the corners and stored as part of a template. Similarly, a runtime image can also be rectified, enhanced, boosted, etc. Additionally, corners can be calculated for the runtime image and matched against corners of templates from the template database. A location or a pose of a vehicle can be determined using the match. In this way, vehicle localization is provided such that drift or other issues associated with GPS, such as occlusion, are mitigated, for example.

Abstract: A system and method are disclosed for detecting road marking in a video using learned road marking templates. The system comprises a template learning module configured to learn the feature-based road marking templates from a set of training images. The template learning module is configured to rectify each training image, detect multiple regions of interest, and for each detected region of interest, detect multiple key points. The template learning module extracts feature vectors for the detected key points and builds the road marking templates from the feature vectors. The system also includes a road marking detection module for detecting road markings in a video at runtime using the learned road marking templates. During runtime, these templates are matched using a two-step process of first selecting promising feature matches and subsequently performing a structural matching to account for the shape of the road markings.

Abstract: A system and method are disclosed for learning part-based object models during a learning phase from training images and applying the learned object models to an input image during runtime. The learned part-based object models are augmented by appearance-based models of the objects. The part-based object models correspond to the shapes of the parts of an object. The appearance-based models provide additional appearance cues to the object models for object classification. The approach to learning part-based object models has the capability of learning object models without using viewpoint labels of the objects. The learning is also invariant to scale and in-plane rotation of the objects.

Abstract: A system and method are disclosed for detecting and labeling places recognized in a video stream using change-points detection. The system includes a segmentation module and a label learning module. The segmentation module is configured to receive a video stream comprising multiple digital representations of images. The video stream is represented by a measurement stream comprising one or more image histograms of the video stream. The segmentation module segments the measurement stream into multiple segments corresponding to place recognized in the videos stream. The segmentation module detects change-points of the measurement stream and computes probability distributions of the segments over multiple pre-learned place models. The label generation module is configured to generate place labels for the places recognized by the place models.