Abstract: In one implementation, a method of generating a plane hypothesis is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method includes obtaining a plurality of points of a point cloud based on the image of the scene. The method includes obtaining an object classification set based on the image of the scene. Each element of the object classification set includes a plurality of pixels respectively associated with a corresponding object in the scene. The method includes detecting a plane within the scene by identifying a subset of the plurality of points of the point cloud that correspond to a particular element of the object classification set.

Abstract: Disclosed is a method comprising receiving data associated with at least one drone, wherein the data at least indicates one or more areas covered by at least one camera of the at least one drone in a real-world environment; and superimposing, on a head-mounted display, a visualization indicating the one or more areas covered by the at least one camera of the at least one drone, wherein the visualization is superimposed on at least one of: the one or more areas in the real-world environment, or a virtual map of the real-world environment.

Abstract: A context based augmented reality system can be used to display augmented reality elements over a live video feed on a client device. The augmented reality elements can be selected based on a number of context inputs generated by the client device. The context inputs can include location data of the client device and location data of nearby physical places that have preconfigured augmented elements. The preconfigured augmented elements can be preconfigured to exhibit a design scheme of the corresponding physical place.

Abstract: A computer-based extended reality health system is adapted for electronically rendering patient sessions within an XR environment incorporating immersive scenes. The XR health system utilizes an XR device adapted for being employed by a patient to visually display the immersive scenes during a patient session within the XR environment. An input device is adapted for electronically interacting with objects, content and features visually displayed in the immersive scenes. A billing module is configured to extract claims data applicable for medical billing claims. The claims data includes billing information associated with the immersive scenes visually rendered to the patient during the patient session within the XR environment.

Abstract: A neural human performance capture framework (MVS-PERF) captures the skeleton, body shape and clothes displacement, and appearance of a person from a set of calibrated multiview images. It addresses the ambiguity of predicting the absolute position in monocular human mesh recovery, and bridges the volumetric representation from NeRF to animation-friendly performance capture. MVS-PERF includes three modules to extract feature maps from multiview images and fuse them to a feature volume, regress the feature volume to a naked human parameters vector, generating an SMPL-X skin-tight body mesh with a skeletal pose, body shape, and expression, and leverage a neural radiance field and a deformation field to infer the clothes as the displacement on the naked body using differentiable rendering. Clothed body mesh is obtained by adding the interpolated displacement vectors to the SMPL-X skin-tight body mesh vertices. The obtained radiance field is used for free-view volumetric rendering of the input subject.

Abstract: There is provided a computer-implemented method of defining bounding boxes for a primitive in a tile-based graphics processing pipeline comprising determining a part-way point on the primitive, wherein, for each pair of vertices, a part-way point is part-way between that pair of vertices, and defining a plurality of bounding boxes, wherein each bounding box intersects a part-way point. Also provided is a bounding box generation circuit comprising a part-way point calculation circuit to determine a part-way point on the primitive, wherein, for each pair of vertices, a part-way point is part-way between that pair of vertices, wherein the bounding box generation circuit to define a plurality of bounding boxes based upon the determined part-way point, wherein each bounding box intersects a part-way point. A method of defining bounding boxes for a point primitive is also provided.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

Abstract: A surgical hub for use with a surgical system in a surgical procedure performed in an operating room is disclosed. The surgical hub comprises a control circuit configured to generate a first image of a surgical site from a real-time imaging source, receive a second image of the surgical site from a non-real-time image source, align the first image and the second image, generate a representation of the surgical site based on the first image and the second image as aligned, display the representation on a display screen, receive a first user selection to overlay hidden structures on the representation, receive a second user selection to manipulate the representation, and adjust the representation based on the second user selection.

Abstract: A method for displaying a three dimensional (“3D”) image includes rendering a frame of 3D image data. The method also includes analyzing the frame of 3D image data to generate best known depth data. The method further includes using the best known depth data to segment the 3D image data into near and far frames of two dimensional (“2D”) image data corresponding to near and far depths respectively. Moreover, the method includes displaying near and far 2D image frames corresponding to the near and far frames of 2D image data at near and far depths to a user respectively.

Abstract: Herein is disclosed a method for ink or coating density compensation by means of applying calculated additional ink/coating to the pre-decorated substrate or film, and by additionally, if needed, apply colour-correction to pre-decoration print files to compensate for higher-than-desired ink density colour shifts. The method includes the steps of providing a flat polymer web having a uniform grid pattern of grid markers printed thereon, and then thermally transforming the web to form a three-dimensional target grid. The target grid is then digitized and the change in the spacing of the grid markers is measured and recorded. The recorded transformed spacing for each grid marker is then used to modify the initial graphics file to compensate for ink-density irregularities resulting from the shrinking and stretching deformations of the substrate during thermal transformation.

Abstract: An image rendering method for rendering a pixel of a virtual scene at a viewpoint includes: downloading a machine learning system corresponding to a current or anticipated state an application determining the virtual scene to be rendered from among a plurality of machine learning systems corresponding to a plurality of states of the application; providing a position and a direction based on the viewpoint to the machine learning system previously trained to predict a factor; combining the predicted factor from the machine learning system with a distribution function that characterises an interaction of light with a predetermined surface to generate the pixel value corresponding to an illuminated first element of the virtual scene at the position; and incorporating the pixel value into a rendered image for display.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: Provided is a method of framing a three dimensional (3D) target object for generation of a virtual camera layout. The method may include analyzing a reference video image to extract a framing rule for at least one reference object in the reference video image, generating a framing rule for at least one 3D target object using the framing rule for the at least one reference object in the reference video image, and using the framing rule for the at least one 3D target object for generation of a virtual camera layout.

Abstract: Provided are augmented reality glasses, including: a lens unit; two temple brackets connected to a first side and a second side of the lens unit, respectively, the first side and the second side being two opposite sides of the lens unit; at least one optical module configured to form a screen, each of which is connected to at least one optical module holder that each is in rotatable connection with one of the temple brackets, an axis of rotation of the rotatable connection is set along a direction from the first side toward the second side so that the optical module is movable between a first position in front of the lens unit and a second position deviating from the front of the lens unit.

Abstract: An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a first pixel location for a marker within a first frame and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a second pixel location for the marker in the second sensor using a second homography, to identify a third pixel location using a disparity mapping, and to determine a distance difference between the second pixel location and the third pixel location. The system is further configured to compare the distance difference to a difference threshold level and to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds the difference threshold level.

Abstract: An image sensor is positioned such that a field-of-view of the sensor encompasses portion of a beverage machine. The field-of-view includes a zone associated with operating the beverage machine. A subsystem receives a signal indicating beverage is dispensed and angled-view images from the image sensor. First image(s) associated with a start of beverage being dispensed are determined by detecting a hand of a person entering the zone associated with operating the beverage machine. A first identifier is determined for the person in the first image(s). Second image(s) associated with an end of the beverage being dispensed are determined by detecting a hand of the person exiting the zone. A second identifier is determined for the person in the second image(s). If the first identifier is the same as the second identifier, the beverage is assigned to the person with the first identifier.

Abstract: A surgical system includes a camera tracking system that determines a first pose transform between a first object coordinate system and the first tracking camera coordinate system based on first object tracking information from the first tracking camera which indicates pose of the first object. The camera tracking system determines a second pose transform between the first object coordinate system and the second tracking camera coordinate system based on first object tracking information from the second tracking camera indicating pose of the first object, and determines a third pose transform between a second object coordinate system and the second tracking camera coordinate system based on second object tracking information from the second tracking camera indicating pose of the second object. The camera tracking system determines a fourth pose transform between the second object coordinate system and the first tracking camera coordinate system.

Abstract: A method includes determining a customized action routine for a person to carry out in a virtual world, where the person is wearing a virtual reality (VR) headset. Objects associated with events from the routine are presented to the person in a virtual world via the VR headset; and aspects of the person's virtual interaction with the object in the virtual world are rendered on the VR headset. The person's virtual interactions with virtual objects is analyzed and used to determine, in real time, one or more next events for said routine.

Abstract: An electronic device determines target information about a target and recommends a target based on the target information.