Abstract: A system and method can support image based determination of mobile device location through recognition of surface features for a previously scanned physical environment. The system and method can also support authoring and positioning of augmented reality features in an authoring interface using the same images and positions of surface features that are to be used for subsequent mobile device localization. As a result, mobile devices leveraging those same images and positions of surface features for localization will be more likely to obtain a localization that is consistent with the positioning displayed in the authoring interface. Augmented reality features authored using the same scan of the environment can be reliably displayed to an end user of an augmented reality application in a position consistent with their authoring in a common coordinate system, even though the authoring may have been performed remotely, away from the actual situs of the physical environment.

Abstract: Augmented reality (AR) or virtual reality (VR) systems described herein can be configured to record images, video, and/or annotations for concurrent communication to a remote system for display or subsequent access. A communication between a user and an expert user using the system can include an audio communication (unidirectional or bidirectional), a video communication from the user to the expert user (allowing the expert user to see, in real time, the same environment as the user), and a data communication (via which content overlaid over the video communication, such as annotations, may be displayed for both the expert user and the user). The systems can be configured to communicate concurrently while also creating a recording for later review based on a “live” or current issue being faced by a user assisted by an expert where the communication session is recorded for later playback by other users.

Abstract: A shared augmented reality system can support the sharing of video captured by a local user, using a head mounted display (HMD), with a remote user. The remote user may add augmented reality annotations (markings, notes, drawings) to certain objects within the environment captured within the video, where the annotations track the movement of those objects within the shared video. An HMD may not, however, provide a convenient interface for performing certain user-interface intensive tasks, which might be better performed on an additional device such as a mobile phone, tablet, or computer. During a shared augmented reality session, the additional device can be configured to communicate with a HMD such that certain tasks can be performed by the user through the additional device, and other tasks can be performed or experienced through the HMD. The additional device, the HMD and the remote user's device can communicatively coordinate during the session.

Abstract: An example method includes: obtaining, from computer memory, information from a three-dimensional (3D) graphical model that represents an object; identifying, based on the information, a first part of the object having an attribute; performing a recognition process on the object based on features of the object, where the recognition process attaches more importance to a second part of the object than to the first part, with the second part either not having the attribute or having less of the attribute than the first part; and providing data for rendering content on a graphical user interface based, at least in part, on recognition of the object performed by the recognition process.

Abstract: A shared augmented reality system can support the sharing of video captured by a local user, using a head mounted display (HMD), with a remote user. The remote user may add augmented reality annotations (markings, notes, drawings) to certain objects within the environment captured within the video, where the annotations track the movement of those objects within the shared video. An HMD may not, however, provide a convenient interface for performing certain user-interface intensive tasks, which might be better performed on an additional device such as a mobile phone, tablet, or computer. During a shared augmented reality session, the additional device can be configured to communicate with a HMD such that certain tasks can be performed by the user through the additional device, and other tasks can be performed or experienced through the HMD. The additional device, the HMD and the remote user's device can communicatively coordinate during the session.

Abstract: In a template design application, a user creates a template for a machine-readable image target that supports encoded data and augmented reality homography. The design application enforces design constraints that ensure reliable detection of the target by a detection application while supporting creative freedom in designing the template. The design constraints can include specification of a detection contour outlining a transition between a perimeter of a border shape and an adjacent area of a contrasting color. Data to be encoded is applied to the template to create printable image instances of the target. The template is also loaded into a detection application, which scans and recognizes printed instances of the target and decodes the encoded data. The detection application uses the decoded data to create a synthesized image target matching the scanned target and which is used to execute further tracking of the target within a moving field of view.

Abstract: An example method includes: obtaining, from computer memory, information from a three-dimensional (3D) graphical model that represents an object; identifying, based on the information, a first part of the object having an attribute; performing a recognition process on the object based on features of the object, where the recognition process attaches more importance to a second part of the object than to the first part, with the second part either not having the attribute or having less of the attribute than the first part; and providing data for rendering content on a graphical user interface based, at least in part, on recognition of the object performed by the recognition process.

Abstract: An example method includes: obtaining, from computer memory, information from a three-dimensional (3D) graphical model that represents an object; identifying, based on the information, a first part of the object having an attribute; performing a recognition process on the object based on features of the object, where the recognition process attaches more importance to a second part of the object than to the first part, with the second part either not having the attribute or having less of the attribute than the first part; and providing data for rendering content on a graphical user interface based, at least in part, on recognition of the object performed by the recognition process.

Abstract: An example method includes: obtaining, from computer memory, information from a three-dimensional (3D) graphical model that represents an object; identifying, based on the information, a first part of the object having an attribute; performing a recognition process on the object based on features of the object, where the recognition process attaches more importance to a second part of the object than to the first part, with the second part either not having the attribute or having less of the attribute than the first part; and providing data for rendering content on a graphical user interface based, at least in part, on recognition of the object performed by the recognition process.

Abstract: Disclosed is a method and apparatus for using color measurement features at multiple scales for a Color Transfer technique. In one embodiment, the functions implemented include: resizing a ground truth image target frame to a plurality of different scales; selecting one or more color measurement features from the ground truth image target frame at each of the plurality of different scales; making a color measurement for each color measurement feature in the ground truth image target frame; and adjusting colors of a virtual object in an augmented frame based at least in part on the color measurements.

Abstract: In a template design application, a user creates a template for a machine-readable image target that supports encoded data and augmented reality homography. The design application enforces design constraints that ensure reliable detection of the target by a detection application while supporting creative freedom in designing the template. The design constraints can include specification of a detection contour outlining a transition between a perimeter of a border shape and an adjacent area of a contrasting color. Data to be encoded is applied to the template to create printable image instances of the target. The template is also loaded into a detection application, which scans and recognizes printed instances of the target and decodes the encoded data. The detection application uses the decoded data to create a synthesized image target matching the scanned target and which is used to execute further tracking of the target within a moving field of view.

Abstract: A mobile platform detects and tracks a three-dimensional (3D) object using occlusions of a two-dimensional (2D) surface. To detect and track the 3D object, an image of the 2D surface with the 3D object is captured and displayed and the 2D surface is detected and tracked. Occlusion of a region assigned as an area of interest on the 2D surface is detected. The shape of the 3D object is determined based on a predefined shape or by using the shape of the area of the 2D surface that is occluded along with the position of the camera with respect to the 2D to calculate the shape. Any desired action with respect to the position of the 3D object on the 2D surface may be performed, such as rendering and displaying a graphical object on or near the displayed 3D object.

Abstract: Systems, methods, and devices are described for constructing a digital representation of a physical scene by obtaining information about the physical scene. Based on the information, an initial portion of a planar surface within the physical scene may be identified. In one aspect of the disclosure, constructing a digital representation of a physical scene may include obtaining information about the physical scene, identifying a planar surface within the physical scene, selecting a physical object within the physical scene, placed above the planar surface, detecting properties associated with the physical object, generating a three-dimensional (3D) reconstructed object using the properties associated with the physical object, and representing the planar surface as an augmented reality (AR) plane in an augmented reality environment, wherein the AR plane in the AR environment is capable of supporting 3D reconstructed objects on top of it.

Abstract: Techniques for managing applications associated with a mobile device are provided. The techniques disclosed herein include techniques for obtaining an image of an object in the view of a camera associated with a mobile device, identifying the object in the image based on attributes of the object extracted from the image, and determining whether one or more applications are associated with the object. If there are one or more applications associated with the real-world object, an application associated with the object can be automatically launched on the mobile device. The association between a real-world object and an application may be identified by a visual indicator, such as an icon, symbol, or other markings on the object that indicates that the object is associated with one or more applications.

Abstract: Techniques are presented for expanding a digital representation of a physical plane from a physical scene. In some aspects, a method may include determining an orientation and an initial portion of a physical plane in the scene, and subdividing a rectified image for the scene into a plurality of grid cells. For the grid cells, an image signature may be generated. A grid cell contiguous to the obtained initial portion of the plane is determined to include part of the plane. An iterative process may be performed for each neighboring grid cell from the grid cell contiguous to at least part of the obtained initial portion, determining whether the neighboring grid cell is to be included as part of the plane if the image signature of said neighboring grid cell is similar to the image signature of a grid cell already determined to be included as part of the plane.

Abstract: A computer-implemented method of tracking a target object in an object recognition system includes acquiring a plurality of images with a camera. The method further includes simultaneously tracking the target object and dynamically building environment map data from the plurality of images. The tracking of the target object includes attempting to estimate a target pose of the target object with respect to the camera based on at least one of the plurality of images and based on target map data. Next, the method determines whether the tracking of the target object with respect to the camera is successful. If not, then the method includes inferring the target pose with respect to the camera based on the dynamically built environment map data. In one aspect the method includes fusing the inferred target pose with the actual target pose even if tracking is successful to improve robustness.

Abstract: Method, computer program product, and apparatus for providing interactions of tangible and augmented reality objects are disclosed. In one embodiment, a method of controlling a real object using a device having a camera comprises receiving a selection of at least one object, tracking the at least one object in a plurality of images captured by the camera, and causing control signals to be transmitted from the device to the real object via a machine interface based at least in part on the tracking.

Abstract: Disclosed is a method and apparatus for selecting a part of SLAM map information of a first device for transmission to a second device in a collaborative SLAM environment. In one embodiment, the functions implemented include: determining whether a 3D registration transformation between a map of the first device and a map of the second device is available; and transmitting a part of map information of the first device to the second device according to one of a first strategy or a second strategy based on whether or not a 3D registration transformation between the map of the first device and the map of the second device is available.

Abstract: Disclosed is a method and apparatus for using color measurement features at multiple scales for a Color Transfer technique. In one embodiment, the functions implemented include: resizing a ground truth image target frame to a plurality of different scales; selecting one or more color measurement features from the ground truth image target frame at each of the plurality of different scales; making a color measurement for each color measurement feature in the ground truth image target frame; and adjusting colors of a virtual object in an augmented frame based at least in part on the color measurements.

Abstract: Method, computer program product, and apparatus for providing interactions of tangible and augmented reality objects are disclosed. In one embodiment, a method for use with an augmented reality enabled device (ARD) comprises performing 3-dimensional tracking of one or more objects based at least in part on captured images of the one or more objects, detecting a state change of at least one object of the one or more objects based at least in part on the captured images, and causing an augmentation to be rendered in response to the state change of the at least one object, where a type of the augmentation is based at least in part on the state change of the at least one object.