Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising receiving an image that depicts a person, identifying a set of skeletal joints of the person and identifying a pose of the person depicted in the image based on positioning of the set of skeletal joints. The operations also include receiving speech input comprising a request to perform an AR operation and an ambiguous intent, discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image and in response to receiving the speech input, performing the AR operation based on discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image.

Abstract: A multi-dimensional fabric user interface is described herein. A remote server executes an operating system that provides a multi-dimensional fabric user interface for storing content for a particular user. The user can access content by manipulating the multi-dimensional fabric user interface through a graphical user interface displayed on a personal mobile computing device. The remote server also enables the user to enter a portal and travel to a virtual representation a second actual physical location from a virtual representation of a first actual physical location, and transfer an object or content from the virtual representation of the first actual physical location to the virtual representation of the second actual physical location. This initiates an operation relating to the one or more of objects and content, in response to the user picking up, transferring, and placing the one or more of objects and content.

Abstract: A system and method for providing a weather effect in an image includes selecting at least one weather texture image indicating weather, and providing a weather effect in the image by overlapping the selected weather texture image on the image.

Abstract: Systems and techniques for aggregation, display, and sharing of data. Graphic items representing data objects identified by a data package may be displayed on timelines. Each timeline may be associated with a respective class of data, and each graphic item displayed on a respective timeline may represent one or more of the data objects in the class associated with the respective timeline.

Abstract: A system and method builds triangles from vertices of polygons and stores the pixels overlayed at least in part by each triangle. The system and method then takes pixels overlayed by an even number of times for each polygon and renders them. Antialiasing information may be generated and provided by oring or exclusive oring bits representing each polygon for each of several pixel divisions.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: Systems and methods are provided for retail environments. In one implementation, a non transitory computer-readable medium may include instructions that when executed by a processor cause the processor to perform a method. The method may include receiving camera location information associated with a plurality of cameras deployed in a retail store, the camera location information being indicative of an actual position of each camera of the plurality of cameras; receiving, for each camera of the plurality of cameras, at least one camera status indicator; using the received camera location information and the received camera status indicators to generate a digital signal; and providing the generated digital signal to an extended reality device to thereby cause the extended reality device to display a visual representation of the at least one status indicator for each camera of the plurality of cameras relative to the actual position of the camera in the retail store.

Abstract: An immersive content generation system can capture a plurality of images of an object in the performance area using a camera. The object is at least partially surrounded by one or more displays presenting images of a virtual environment. The system can determine attributes of a lens of the taking camera, a first distance from the taking camera to the physical object, and a second distance between a LED display presenting a virtual image of the physical object and the taking camera. The system can determine the depth of field blur for the virtual objects based on the attributes of the taking camera and the virtual depth of each of the virtual objects. The system can apply the depth of field blur to the virtual objects and generate content based on the plurality of captured images and the determined amount of depth of field blur to the virtual objects.

Abstract: An augmented reality device generates an updated position and orientation (pose) value by initially determining, using image-based processing, a pose estimate from a current image frame, a previous image frame and a previous pose. An IMU pose is then obtained from data generated by an inertial measurement unit including motion sensors, and transformation between the pose estimate and the IMU pose is determined. The transformation is applied to the pose estimate to generate a pose update. Media content is applied to a position within an image, the position being based on the pose update.

Abstract: Devices, systems, and methods are provided for real-time object placement guidance in augmented reality experience. An example method may include receiving, by a device having a sensor, an indication of an object to be viewed in an physical environment of the device. The example method may also include determining a 3D model of the physical environment using data of the physical environment captured by the sensor. The example method may also include determining that a first surface in the 3D model of the environment is a first floor space, and a second surface in the 3D model of the environment is a first wall space. The example method may also include determining that a portion of the first surface is unoccupied and sized to fit the object.

Abstract: Disclosed herein are various embodiments for a headset-based interface and menu system. An embodiment operates by determining a first position of a headset, configured to display and enable interactions with an interface of a computing system, is less than a threshold. A second position of the headset is detected and it is determined that the second position of the headset is greater than the threshold. Responsive to the determining that the second position of the headset is greater than the threshold, a menu is provided for display in the interface visible via the headset overlaying the interface.

Abstract: Viewport transformation modules for use in a three-dimensional rendering system wherein vertices are received from an application in a strip. The viewport transformation modules include a fetch module configured to read from a vertex buffer: untransformed coordinate data for a vertex in a strip; information identifying a viewport associated with the vertex; and information identifying a viewport associated with one or more other vertices in the strip. The one or more other vertices in the strip are selected based on a provoking vertex of a primitive to be formed by the vertices in the strip and a number of vertices in the primitive. The viewport transformation modules also include a processing module that performs a viewport transformation on the untransformed coordinate data based on each of the identified viewports to generate transformed coordinate data for each identified viewport; and a write module that writes the transformed coordinate data for each identified viewport to the vertex buffer.

Abstract: The present specification discloses a method for displaying an AR navigation screen, and an AR navigation system. The method for displaying an AR navigation screen according to the present specification may comprise the steps of: overlapping and displaying images of the surroundings of a vehicle and a plurality of AR images corresponding to the images of the surroundings; generating a window for separately displaying a first area including AR images overlapping at least one of the plurality of AR images; and overlapping and displaying the window and the images of the surroundings. The invention of the present specification has the effect of improving user convenience and promoting safe vehicle operation by using a window when AR images and the like are difficult to recognize due to being overlapped.

Abstract: An augmented reality (AR) overlay augments traditional menu items with corresponding photos, thereby facilitating a decision-making process of a user ordering from the menu. In addition to providing imagery of the menu items listed, other information may also be supplied, such as ratings, reviews etc. In this regard, users can visualize what to expect before ordering, and can order with a greater degree of confidence that they will enjoy the menu item they select.

Abstract: In one embodiment, a computer implemented method for rendering virtual environments is disclosed. The method includes associating by a computing system, an object with a container effect, by receiving information regarding an object category for the object and matching the object category to a category associated with the container effect, where the container effect defines virtual effects for objects associated therewith. The method also includes generating by the computing system a virtual environment including the object by retrieving a model of the object an utilizing the model and the container effect to render a virtual object.

Abstract: An editing terminal includes a simple display data acquisition unit that acquires simple display data from an item management server, an item selection processing unit that receives selection of an item from a plurality of items displayed using the simple display data, a three-dimensional data acquisition unit that acquires three-dimensional data of a selected item from the item management server, and an editing processing unit that displays an editing space on an editing screen on the basis of editing space information, receives an input of operation information regarding editing of the editing space using the three-dimensional data of the selected item, transmits the operation information to an editing server, and displays the editing space after editing on the editing screen.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable faster and more efficient real-time physical object recognition, information retrieval, and updating of a CGR environment. In some implementations, the CGR environment is provided at a first device based on a classification of the physical object, image or video data including the physical object is transmitted by the first device to a second device, and the CGR environment is updated by the first device based on a response associated with the physical object received from the second device.

Abstract: Systems and methods for providing an AR visual display to save are described. A camera of a computing device scans an area and an AR engine of the computing device extrapolates flat portions of the area. A digital representation of the area is displayed via a GUI of the computing device to allow a user to place digitally created content. The digitally created content is textual content, 2D content, 3D content, video content, and/or audio content. The AR engine receives, from a user, an action executed on a surface and at a location of the digital representation to add a tag or an anchor. The AR engine utilizes planes as indicators to save the placement of the tag or the anchor and transmits the placement of the tag or the anchor to a server. The placement of the tag or the anchor allows for viewing from other computing devices.

Abstract: A virtual local presence display apparatus, system and method is disclosed.

Abstract: A split-architecture for rendering and warping world-locked AR elements, such as graphics in a navigation application, for display on augmented reality (AR) glasses is disclosed. The split-architecture can help to alleviate a resource burden on the AR glasses by performing the more complex processes associated with the rendering and warping on a computing device, while performing the less complex processes associated with the rendering and warping on the AR glasses. The AR glasses and the computing device are coupled via wireless communication, and the disclosed systems and methods address the large and variable latencies associated with the wireless communication that could otherwise make splitting these processes impractical.