Abstract: An image rendering method for rendering a pixel of a virtual scene at a viewpoint includes: downloading a machine learning system corresponding to a current or anticipated state an application determining the virtual scene to be rendered from among a plurality of machine learning systems corresponding to a plurality of states of the application; providing a position and a direction based on the viewpoint to the machine learning system previously trained to predict a factor; combining the predicted factor from the machine learning system with a distribution function that characterises an interaction of light with a predetermined surface to generate the pixel value corresponding to an illuminated first element of the virtual scene at the position; and incorporating the pixel value into a rendered image for display.

Abstract: Provided is a method of framing a three dimensional (3D) target object for generation of a virtual camera layout. The method may include analyzing a reference video image to extract a framing rule for at least one reference object in the reference video image, generating a framing rule for at least one 3D target object using the framing rule for the at least one reference object in the reference video image, and using the framing rule for the at least one 3D target object for generation of a virtual camera layout.

Abstract: An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a first pixel location for a marker within a first frame and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a second pixel location for the marker in the second sensor using a second homography, to identify a third pixel location using a disparity mapping, and to determine a distance difference between the second pixel location and the third pixel location. The system is further configured to compare the distance difference to a difference threshold level and to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds the difference threshold level.

Abstract: An image sensor is positioned such that a field-of-view of the sensor encompasses portion of a beverage machine. The field-of-view includes a zone associated with operating the beverage machine. A subsystem receives a signal indicating beverage is dispensed and angled-view images from the image sensor. First image(s) associated with a start of beverage being dispensed are determined by detecting a hand of a person entering the zone associated with operating the beverage machine. A first identifier is determined for the person in the first image(s). Second image(s) associated with an end of the beverage being dispensed are determined by detecting a hand of the person exiting the zone. A second identifier is determined for the person in the second image(s). If the first identifier is the same as the second identifier, the beverage is assigned to the person with the first identifier.

Abstract: Provided are augmented reality glasses, including: a lens unit; two temple brackets connected to a first side and a second side of the lens unit, respectively, the first side and the second side being two opposite sides of the lens unit; at least one optical module configured to form a screen, each of which is connected to at least one optical module holder that each is in rotatable connection with one of the temple brackets, an axis of rotation of the rotatable connection is set along a direction from the first side toward the second side so that the optical module is movable between a first position in front of the lens unit and a second position deviating from the front of the lens unit.

Abstract: A surgical system includes a camera tracking system that determines a first pose transform between a first object coordinate system and the first tracking camera coordinate system based on first object tracking information from the first tracking camera which indicates pose of the first object. The camera tracking system determines a second pose transform between the first object coordinate system and the second tracking camera coordinate system based on first object tracking information from the second tracking camera indicating pose of the first object, and determines a third pose transform between a second object coordinate system and the second tracking camera coordinate system based on second object tracking information from the second tracking camera indicating pose of the second object. The camera tracking system determines a fourth pose transform between the second object coordinate system and the first tracking camera coordinate system.

Abstract: A method includes determining a customized action routine for a person to carry out in a virtual world, where the person is wearing a virtual reality (VR) headset. Objects associated with events from the routine are presented to the person in a virtual world via the VR headset; and aspects of the person's virtual interaction with the object in the virtual world are rendered on the VR headset. The person's virtual interactions with virtual objects is analyzed and used to determine, in real time, one or more next events for said routine.

Abstract: An electronic device determines target information about a target and recommends a target based on the target information.

Abstract: A method for mitigating interference across analog signal lines includes receiving a digital data stream including a plurality of discrete signal patterns configured to drive a plurality of different analog signal lines. An edge buffer for each analog signal line is populated with edge data representing pulse edges of upcoming signal patterns set to drive the analog signal line. A target buffer for a target signal line is populated with target data representing a target signal pattern. Based at least in part on determining that edge buffers corresponding to one or more potentially interfering analog signal lines include edge data corresponding to post-target pulse edges, one or more potentially interfering signal patterns are identified. A selected set of the potentially interfering signal patterns are used to modify the target signal pattern to perform preemptive interference mitigation.

Abstract: An image processing apparatus according to the present invention, includes at least one memory and at least one processor function as: a first determining unit configured to determine a first brightness range, which is included in a dynamic range of a first image and is higher than a predetermined brightness; and a first converting unit configured to convert the first image into a second image, of which a dynamic range is narrower than the dynamic range of the first image, based on the determination result by the first determining unit, wherein based on the first brightness range determined by the first determining unit, the first converting unit determines a second brightness range, which is included in the dynamic range of the second image and corresponds to the first brightness range.

Abstract: An information processing apparatus including circuitry that acquires information indicating a spatial relationship between a real object and a virtual object, and initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world, and wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

Abstract: Disclosed are systems and methods for mixed reality collaboration. A method may include receiving persistent coordinate data; presenting a first virtual session handle to a first user at a first position via a transmissive display of a wearable device, wherein the first position is based on the persistent coordinate data; presenting a virtual object to the first user at a second location via the transmissive display, wherein the second position is based on the first position; receiving location data from a second user, wherein the location data relates a position of the second user to a position of a second virtual session handle; presenting a virtual avatar to the first user at a third position via the transmissive display, wherein the virtual avatar corresponds to the second user, wherein the third position is based on the location data, and wherein the third position is further based on the first position.

Abstract: An augmented reality system is provided, including a physical apparatus operable to change detectably by a human between a first state and a second state, and an augmented reality application. The physical apparatus includes a signal receiver for receiving a signal, and at least one controllable element operable to effect the change between the first state and the second state upon receiving the signal. The AR application, when executed by at least one processor of a computing device, the computing device having at least one camera and a display, cause the computing device to capture at least one image of the physical apparatus, generate a virtual reality object that is presented in the at least one image on the display, and transmit the signal to the physical apparatus to cause the at least one controllable element of the physical apparatus to switch between the first state and the second state.

Abstract: A method, by an augmented reality device, of measuring a depth of an object includes determining, from a dot-pattern and a surface-pattern, a pattern of light to be emitted for measuring the depth of the object, identifying, from within an entire area of a pattern generator, a partial area of a light source unit corresponding to an area for the determined pattern, emitting light through the area for the determined pattern, by activating the identified partial area of the light source unit, receiving light reflected from the object; and measuring the depth of the object based on the emitted light and the received reflected light.

Abstract: The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.

Abstract: An augmented reality screen system includes an augmented reality device and a host. The augmented reality device is configured to take a physical mark through a camera. The host is configured to receive the physical mark, determine position information and rotation information of the physical mark, and fetch a virtual image from a storage device through a processor of the host. The processor transmits an adjusted virtual image to the augmented reality device according to the position information and the rotation information, and the augmentation device projects the adjusted virtual image to a display of the augmented reality device. The adjusted virtual image becomes a virtual extended screen, and the virtual extended screen and the physical mark are simultaneously displayed on the display of the augmented reality device.

Abstract: The invention includes a method including the steps of obtaining a plurality of images, each of the images in the plurality having at least one corresponding region, generating a merged image, the merged image also having the corresponding region. The step of generating includes selecting an image source from the plurality of images to source image data for the corresponding region in the merged image by comparing attributes of the corresponding regions of the plurality of images to identify the image source having preferred attributes.

Abstract: Aspects of the disclosure are directed to an artificial reality (XR) browser configured to trigger an immersive experience. Implementations display an element at a browser chrome of the XR browser when an immersive experience is loaded. For example, the XR browser can include an application programming interface (API) that supports configuration of a browser chrome element by components of a webpage. The API call can cause the display of the browser chrome element, change a display property for the browser chrome element, or configure the browser chrome element in any other suitable manner. Upon receiving input at the browser chrome element (configured by the API call), the XR browser can transition from displaying a two-dimensional panel view of a webpage supported by loaded web resources (e.g., hypertext transfer protocol (HTTP) pages, graphic images, etc.) to a three-dimensional environment supported by preloaded immersive resources (e.g., three-dimensional models, graphic images, etc.).

Abstract: Disclosed is an electronic device that includes: a camera; a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain, by the camera, an image including at least one object, identify an object type of the at least one object, obtain spatial information corresponding to the object type, based on the object type, and control the display, to display, a virtual object, based on the spatial information.

Abstract: In one implementation, a method for modeling a behavior with synthetic training data. The method includes: obtaining source content that includes an entity performing one or more actions within an environment; generating a first environment characterization vector characterizing the environment; generating a first set of behavioral trajectories associated with the one or more actions of the entity based on the source content and the first characterization vector for the environment; generating a second environment characterization vector for the environment by perturbing the first environment characterization vector; generating a second set of behavioral trajectories associated with one or more potential actions of the entity based on the source content and the second characterization vector for the environment; and training a behavior model for a virtual agent based on the first and second sets of behavioral trajectories in order to imitate the entity.