Abstract: A surgical hub for use with a surgical system in a surgical procedure performed in an operating room is disclosed. The surgical hub comprises a control circuit configured to generate a first image of a surgical site from a real-time imaging source, receive a second image of the surgical site from a non-real-time image source, align the first image and the second image, generate a representation of the surgical site based on the first image and the second image as aligned, display the representation on a display screen, receive a first user selection to overlay hidden structures on the representation, receive a second user selection to manipulate the representation, and adjust the representation based on the second user selection.

Abstract: A method for displaying a three dimensional (“3D”) image includes rendering a frame of 3D image data. The method also includes analyzing the frame of 3D image data to generate best known depth data. The method further includes using the best known depth data to segment the 3D image data into near and far frames of two dimensional (“2D”) image data corresponding to near and far depths respectively. Moreover, the method includes displaying near and far 2D image frames corresponding to the near and far frames of 2D image data at near and far depths to a user respectively.

Abstract: Herein is disclosed a method for ink or coating density compensation by means of applying calculated additional ink/coating to the pre-decorated substrate or film, and by additionally, if needed, apply colour-correction to pre-decoration print files to compensate for higher-than-desired ink density colour shifts. The method includes the steps of providing a flat polymer web having a uniform grid pattern of grid markers printed thereon, and then thermally transforming the web to form a three-dimensional target grid. The target grid is then digitized and the change in the spacing of the grid markers is measured and recorded. The recorded transformed spacing for each grid marker is then used to modify the initial graphics file to compensate for ink-density irregularities resulting from the shrinking and stretching deformations of the substrate during thermal transformation.

Abstract: An image rendering method for rendering a pixel of a virtual scene at a viewpoint includes: downloading a machine learning system corresponding to a current or anticipated state an application determining the virtual scene to be rendered from among a plurality of machine learning systems corresponding to a plurality of states of the application; providing a position and a direction based on the viewpoint to the machine learning system previously trained to predict a factor; combining the predicted factor from the machine learning system with a distribution function that characterises an interaction of light with a predetermined surface to generate the pixel value corresponding to an illuminated first element of the virtual scene at the position; and incorporating the pixel value into a rendered image for display.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: Provided is a method of framing a three dimensional (3D) target object for generation of a virtual camera layout. The method may include analyzing a reference video image to extract a framing rule for at least one reference object in the reference video image, generating a framing rule for at least one 3D target object using the framing rule for the at least one reference object in the reference video image, and using the framing rule for the at least one 3D target object for generation of a virtual camera layout.

Abstract: Provided are augmented reality glasses, including: a lens unit; two temple brackets connected to a first side and a second side of the lens unit, respectively, the first side and the second side being two opposite sides of the lens unit; at least one optical module configured to form a screen, each of which is connected to at least one optical module holder that each is in rotatable connection with one of the temple brackets, an axis of rotation of the rotatable connection is set along a direction from the first side toward the second side so that the optical module is movable between a first position in front of the lens unit and a second position deviating from the front of the lens unit.

Abstract: An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a first pixel location for a marker within a first frame and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a second pixel location for the marker in the second sensor using a second homography, to identify a third pixel location using a disparity mapping, and to determine a distance difference between the second pixel location and the third pixel location. The system is further configured to compare the distance difference to a difference threshold level and to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds the difference threshold level.

Abstract: An image sensor is positioned such that a field-of-view of the sensor encompasses portion of a beverage machine. The field-of-view includes a zone associated with operating the beverage machine. A subsystem receives a signal indicating beverage is dispensed and angled-view images from the image sensor. First image(s) associated with a start of beverage being dispensed are determined by detecting a hand of a person entering the zone associated with operating the beverage machine. A first identifier is determined for the person in the first image(s). Second image(s) associated with an end of the beverage being dispensed are determined by detecting a hand of the person exiting the zone. A second identifier is determined for the person in the second image(s). If the first identifier is the same as the second identifier, the beverage is assigned to the person with the first identifier.

Abstract: A surgical system includes a camera tracking system that determines a first pose transform between a first object coordinate system and the first tracking camera coordinate system based on first object tracking information from the first tracking camera which indicates pose of the first object. The camera tracking system determines a second pose transform between the first object coordinate system and the second tracking camera coordinate system based on first object tracking information from the second tracking camera indicating pose of the first object, and determines a third pose transform between a second object coordinate system and the second tracking camera coordinate system based on second object tracking information from the second tracking camera indicating pose of the second object. The camera tracking system determines a fourth pose transform between the second object coordinate system and the first tracking camera coordinate system.

Abstract: An electronic device determines target information about a target and recommends a target based on the target information.

Abstract: A method for mitigating interference across analog signal lines includes receiving a digital data stream including a plurality of discrete signal patterns configured to drive a plurality of different analog signal lines. An edge buffer for each analog signal line is populated with edge data representing pulse edges of upcoming signal patterns set to drive the analog signal line. A target buffer for a target signal line is populated with target data representing a target signal pattern. Based at least in part on determining that edge buffers corresponding to one or more potentially interfering analog signal lines include edge data corresponding to post-target pulse edges, one or more potentially interfering signal patterns are identified. A selected set of the potentially interfering signal patterns are used to modify the target signal pattern to perform preemptive interference mitigation.

Abstract: A method includes determining a customized action routine for a person to carry out in a virtual world, where the person is wearing a virtual reality (VR) headset. Objects associated with events from the routine are presented to the person in a virtual world via the VR headset; and aspects of the person's virtual interaction with the object in the virtual world are rendered on the VR headset. The person's virtual interactions with virtual objects is analyzed and used to determine, in real time, one or more next events for said routine.

Abstract: An information processing apparatus including circuitry that acquires information indicating a spatial relationship between a real object and a virtual object, and initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world, and wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

Abstract: An image processing apparatus according to the present invention, includes at least one memory and at least one processor function as: a first determining unit configured to determine a first brightness range, which is included in a dynamic range of a first image and is higher than a predetermined brightness; and a first converting unit configured to convert the first image into a second image, of which a dynamic range is narrower than the dynamic range of the first image, based on the determination result by the first determining unit, wherein based on the first brightness range determined by the first determining unit, the first converting unit determines a second brightness range, which is included in the dynamic range of the second image and corresponds to the first brightness range.

Abstract: An augmented reality system is provided, including a physical apparatus operable to change detectably by a human between a first state and a second state, and an augmented reality application. The physical apparatus includes a signal receiver for receiving a signal, and at least one controllable element operable to effect the change between the first state and the second state upon receiving the signal. The AR application, when executed by at least one processor of a computing device, the computing device having at least one camera and a display, cause the computing device to capture at least one image of the physical apparatus, generate a virtual reality object that is presented in the at least one image on the display, and transmit the signal to the physical apparatus to cause the at least one controllable element of the physical apparatus to switch between the first state and the second state.

Abstract: Disclosed are systems and methods for mixed reality collaboration. A method may include receiving persistent coordinate data; presenting a first virtual session handle to a first user at a first position via a transmissive display of a wearable device, wherein the first position is based on the persistent coordinate data; presenting a virtual object to the first user at a second location via the transmissive display, wherein the second position is based on the first position; receiving location data from a second user, wherein the location data relates a position of the second user to a position of a second virtual session handle; presenting a virtual avatar to the first user at a third position via the transmissive display, wherein the virtual avatar corresponds to the second user, wherein the third position is based on the location data, and wherein the third position is further based on the first position.

Abstract: The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.

Abstract: A method, by an augmented reality device, of measuring a depth of an object includes determining, from a dot-pattern and a surface-pattern, a pattern of light to be emitted for measuring the depth of the object, identifying, from within an entire area of a pattern generator, a partial area of a light source unit corresponding to an area for the determined pattern, emitting light through the area for the determined pattern, by activating the identified partial area of the light source unit, receiving light reflected from the object; and measuring the depth of the object based on the emitted light and the received reflected light.

Abstract: An augmented reality screen system includes an augmented reality device and a host. The augmented reality device is configured to take a physical mark through a camera. The host is configured to receive the physical mark, determine position information and rotation information of the physical mark, and fetch a virtual image from a storage device through a processor of the host. The processor transmits an adjusted virtual image to the augmented reality device according to the position information and the rotation information, and the augmentation device projects the adjusted virtual image to a display of the augmented reality device. The adjusted virtual image becomes a virtual extended screen, and the virtual extended screen and the physical mark are simultaneously displayed on the display of the augmented reality device.