Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving, from a client device of a first user, a request from the first user to engage in an AR shopping experience curated by a store; identifying a first real-world product available for purchase from the store; receiving an image of a real-world environment of the first user; generating a first AR item that represents the first real-world product; comparing visual attributes of the first AR item to physical layouts of a plurality of real-world objects depicted in the image of the real-world environment; and overlaying the first AR item on a first real-world object of the plurality of real-world objects in the image responsive to comparing the visual attributes of the first AR item to the physical layouts of the plurality of real-world objects.

Abstract: The present invention discloses a three-dimensional model and material merging method, device, terminal, storage medium, and rendering method, which belongs to the technical field of graphic rendering. The method comprises obtaining three-dimensional model files, material files and texture maps; determining the layout of the texture map; de-duplicating the material, merging the texture map of each channel according to the determined texture layout to obtain the texture map puzzle of each channel; optimizing the storage size of the three-dimensional models and texture map puzzles of each channel to generate and store the configuration file. The present method improves the speed of three-dimensional model and material merging, reduces the amount of data storage, thereby reduces the amount of data loading of the rendering CPU, and thus improves the rendering efficiency.

Abstract: According to one embodiment, a display apparatus includes a display device and an image display control circuit. The display device includes a display panel and a light controller. The light controller controls a direction of a light ray emitted from the display panel. The image display control circuit controls the display panel to display an image in which two or more parallax images having different parallax numbers are mixed, on a display surface of the display device. The image display control circuit displays a parallax image having a first parallax number in at least a first area and displays a parallax image having a second parallax number in a second area. The second parallax number is smaller than the first parallax number.

Abstract: An information processing apparatus includes a processor configured to make a character input into an input field invisible, the input field being displayed on a display, and to display a character image on the display so that the character image is superimposed on the character input into the input field, the character image being drawn with an external font installed in an external apparatus based on the character input into the input field.

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method includes obtaining a plurality of points of a point cloud based on the image of the scene. The method includes obtaining an object classification set based on the image of the scene. Each element of the object classification set includes a plurality of pixels respectively associated with a corresponding object in the scene. The method includes detecting a plane within the scene by identifying a subset of the plurality of points of the point cloud that correspond to a particular element of the object classification set.

Abstract: Disclosed is a method comprising receiving data associated with at least one drone, wherein the data at least indicates one or more areas covered by at least one camera of the at least one drone in a real-world environment; and superimposing, on a head-mounted display, a visualization indicating the one or more areas covered by the at least one camera of the at least one drone, wherein the visualization is superimposed on at least one of: the one or more areas in the real-world environment, or a virtual map of the real-world environment.

Abstract: A neural human performance capture framework (MVS-PERF) captures the skeleton, body shape and clothes displacement, and appearance of a person from a set of calibrated multiview images. It addresses the ambiguity of predicting the absolute position in monocular human mesh recovery, and bridges the volumetric representation from NeRF to animation-friendly performance capture. MVS-PERF includes three modules to extract feature maps from multiview images and fuse them to a feature volume, regress the feature volume to a naked human parameters vector, generating an SMPL-X skin-tight body mesh with a skeletal pose, body shape, and expression, and leverage a neural radiance field and a deformation field to infer the clothes as the displacement on the naked body using differentiable rendering. Clothed body mesh is obtained by adding the interpolated displacement vectors to the SMPL-X skin-tight body mesh vertices. The obtained radiance field is used for free-view volumetric rendering of the input subject.

Abstract: A context based augmented reality system can be used to display augmented reality elements over a live video feed on a client device. The augmented reality elements can be selected based on a number of context inputs generated by the client device. The context inputs can include location data of the client device and location data of nearby physical places that have preconfigured augmented elements. The preconfigured augmented elements can be preconfigured to exhibit a design scheme of the corresponding physical place.

Abstract: There is provided a computer-implemented method of defining bounding boxes for a primitive in a tile-based graphics processing pipeline comprising determining a part-way point on the primitive, wherein, for each pair of vertices, a part-way point is part-way between that pair of vertices, and defining a plurality of bounding boxes, wherein each bounding box intersects a part-way point. Also provided is a bounding box generation circuit comprising a part-way point calculation circuit to determine a part-way point on the primitive, wherein, for each pair of vertices, a part-way point is part-way between that pair of vertices, wherein the bounding box generation circuit to define a plurality of bounding boxes based upon the determined part-way point, wherein each bounding box intersects a part-way point. A method of defining bounding boxes for a point primitive is also provided.

Abstract: A computer-based extended reality health system is adapted for electronically rendering patient sessions within an XR environment incorporating immersive scenes. The XR health system utilizes an XR device adapted for being employed by a patient to visually display the immersive scenes during a patient session within the XR environment. An input device is adapted for electronically interacting with objects, content and features visually displayed in the immersive scenes. A billing module is configured to extract claims data applicable for medical billing claims. The claims data includes billing information associated with the immersive scenes visually rendered to the patient during the patient session within the XR environment.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

Abstract: A surgical hub for use with a surgical system in a surgical procedure performed in an operating room is disclosed. The surgical hub comprises a control circuit configured to generate a first image of a surgical site from a real-time imaging source, receive a second image of the surgical site from a non-real-time image source, align the first image and the second image, generate a representation of the surgical site based on the first image and the second image as aligned, display the representation on a display screen, receive a first user selection to overlay hidden structures on the representation, receive a second user selection to manipulate the representation, and adjust the representation based on the second user selection.

Abstract: A method for displaying a three dimensional (“3D”) image includes rendering a frame of 3D image data. The method also includes analyzing the frame of 3D image data to generate best known depth data. The method further includes using the best known depth data to segment the 3D image data into near and far frames of two dimensional (“2D”) image data corresponding to near and far depths respectively. Moreover, the method includes displaying near and far 2D image frames corresponding to the near and far frames of 2D image data at near and far depths to a user respectively.

Abstract: Herein is disclosed a method for ink or coating density compensation by means of applying calculated additional ink/coating to the pre-decorated substrate or film, and by additionally, if needed, apply colour-correction to pre-decoration print files to compensate for higher-than-desired ink density colour shifts. The method includes the steps of providing a flat polymer web having a uniform grid pattern of grid markers printed thereon, and then thermally transforming the web to form a three-dimensional target grid. The target grid is then digitized and the change in the spacing of the grid markers is measured and recorded. The recorded transformed spacing for each grid marker is then used to modify the initial graphics file to compensate for ink-density irregularities resulting from the shrinking and stretching deformations of the substrate during thermal transformation.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: An image rendering method for rendering a pixel of a virtual scene at a viewpoint includes: downloading a machine learning system corresponding to a current or anticipated state an application determining the virtual scene to be rendered from among a plurality of machine learning systems corresponding to a plurality of states of the application; providing a position and a direction based on the viewpoint to the machine learning system previously trained to predict a factor; combining the predicted factor from the machine learning system with a distribution function that characterises an interaction of light with a predetermined surface to generate the pixel value corresponding to an illuminated first element of the virtual scene at the position; and incorporating the pixel value into a rendered image for display.

Abstract: Provided is a method of framing a three dimensional (3D) target object for generation of a virtual camera layout. The method may include analyzing a reference video image to extract a framing rule for at least one reference object in the reference video image, generating a framing rule for at least one 3D target object using the framing rule for the at least one reference object in the reference video image, and using the framing rule for the at least one 3D target object for generation of a virtual camera layout.

Abstract: An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a first pixel location for a marker within a first frame and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a second pixel location for the marker in the second sensor using a second homography, to identify a third pixel location using a disparity mapping, and to determine a distance difference between the second pixel location and the third pixel location. The system is further configured to compare the distance difference to a difference threshold level and to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds the difference threshold level.

Abstract: An image sensor is positioned such that a field-of-view of the sensor encompasses portion of a beverage machine. The field-of-view includes a zone associated with operating the beverage machine. A subsystem receives a signal indicating beverage is dispensed and angled-view images from the image sensor. First image(s) associated with a start of beverage being dispensed are determined by detecting a hand of a person entering the zone associated with operating the beverage machine. A first identifier is determined for the person in the first image(s). Second image(s) associated with an end of the beverage being dispensed are determined by detecting a hand of the person exiting the zone. A second identifier is determined for the person in the second image(s). If the first identifier is the same as the second identifier, the beverage is assigned to the person with the first identifier.