Abstract: Provided is a method including: associating a virtual display of an object with first media content including a first type media content; associating the virtual display of the object with second media content that includes second type media content; generating a data model that includes the virtual display associated with the first media content and the second media content; and storing the data model in a storage device coupled to the computer system. As such, the method unifies all components (e.g., media content, tags, permissions, descriptions) of the data model—text description and attributes, image, video, and audio—to create a unique, multi-dimensional, transferable digital representation of an idea, memory, or history tied to the object.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

Abstract: A method and an apparatus for 4D road scene annotation based on time series data includes: obtaining point cloud data and multi-view 2D image data of a same road scene at a same time series, as well as a rotation matrix R and a translation vector t obtained through sensors; merging all the point cloud data into a combined point cloud using the rotation matrix R and the translation vector t; performing dynamic and static object annotation and lane line annotation on the combined point cloud in the 4D road scene; and mapping annotation information in the 4D road scene to all the 2D image data at the time series based on camera parameter information, to obtain annotation information on 2D images. The method fully utilizes time series information, avoiding the loss of important information.

Abstract: An image processing device includes at least one processor, in which in a case in which, for at least one image of a real image visually recognized by a user through a transmission unit of a glasses-type information display device or a projected image projected onto the transmission unit and visually recognized by the user, a visibility change unit that changes visibility by the user changes visibility of the real image, the processor generates an image simulating the real image after the change in visibility from a captured image obtained by imaging a real world with a camera based on at least one of information indicating the change in visibility, model information on the glasses-type information display device, or model information on the camera.

Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.

Abstract: In an example, a method includes receiving, at least one processor, voxelised object model data describing at least part of an intended fabrication chamber content to be generated using additive manufacturing. The voxelised object model data may be derived from a first data set modelling a first object as 2D slices and a second data set modelling a second object as a 3D mesh. Data within the voxelised object model data which is derived from the first data set may be identified, and a geometrical transformation may be applied thereto.

Abstract: A system comprises an encoder configured to compress and encode data for a three-dimensional mesh using a video encoding technique. To compress the three-dimensional mesh, the encoder determines sub-meshes and for each sub-mesh: texture patches and geometry patches. Also the encoder determines patch connectivity information and patch texture coordinates for the texture patches and geometry patches. The texture patches and geometry patches are packed into video image frames and encoded using a video codec. Additionally, the encoder determines boundary stitching information for the sub-meshes. A decoder receives a bit stream as generated by the encoder and reconstructs the three-dimensional mesh.

Abstract: A connection may be established between an extended reality (XR) device and a non-XR device. User input may be received via one of the XR device and the non-XR device. Position information associated with the user input may be captured by the one of the XR device and the non-XR device. The position information associated with the user input may be converted to be executable by the other device of the XR device and the non-XR device. The other device of the XR device and the non-XR device may perform an action in response to the user input received by the one of the XR device and the non-XR device.

Abstract: The present disclosure describes techniques for determining a bounding box. An image may be received. An X-frame, a Y-frame, and a normal frame may be estimated based on the image using a first neural network. At least one planar region may be detected from the image using a second neural network. A vanishing point detection may be performed on each of the at least one planar region. Output of the first neural network may be fused with results of the vanishing point detection. A depth value of each pixel in at least one plane corresponding to the at least one planar region may be determined based at least in part on a result of the fusing. A location of a bounding box may be determined based at least in part on the depth value of each pixel in the at least one plane.

Abstract: System, methods, and machine-readable media to facilitate adaptive content composite interaction with respect to digitally distributed content are disclosed. Discovery may be performed, and events specified for digital distribution may be detected. A particular endpoint identifier mapped to media devices may be detected. Events may be determined as a function of access specifications. Content objects corresponding to the events may be received from a remote system via a network. Content composites corresponding to mapping specifications may be created for each content object with selecting and configuring an adaptable content object with the content object to form a content composite to facilitate presentation of the content composite.

Abstract: Head-mounted display systems with power saving functionality are disclosed. The systems can include a frame configured to be supported on the head of the user. The systems can also include a head-mounted display disposed on the frame, one or more sensors, and processing electronics in communication with the display and the one or more sensors. In some implementations, the processing electronics can be configured to cause the system to reduce power of one or more components in response to at least in part on a determination that the frame is in a certain position (e.g., upside-down or on top of the head of the user). In some implementations, the processing electronics can be configured to cause the system to reduce power of one or more components in response to at least in part on a determination that the frame has been stationary for at least a threshold period of time.

Abstract: Systems and methods are provided for rendering hair. The systems and methods include receiving hair spline data comprising coordinates of a plurality of hair strands; selecting a first hair strand of the plurality of hair strands; retrieving coordinates of the first hair strand; identifying based on the respective coordinates of the plurality of hair strands a second hair strand that is adjacent to the first hair strand; storing a reference to the second hair strand in association with the coordinates of the first hair strand; and generating one or more additional hair strands between the first hair strand and the second hair strand based on the coordinates of the first hair strand and the reference to the second hair strand.

Abstract: A synthetic lifelike behavioral character animation system and methods for simulating lifelike behavior in an animated character. The system provides simulated lifelike character behavior for use in a variety of interactive contexts such as AI interactive agents, video games, and metaverse-type experiences. The system processes contextual information and generates behavioral outputs that drive character animation that is lifelike and believable in nature.

Abstract: Disclosed herein is a method of transmitting point cloud data, including encoding point cloud data, and transmitting a bitstream containing the point cloud data. Disclosed herein a method of receiving point cloud data, including receiving a bitstream containing point cloud data, and decoding the point cloud data.

Abstract: A photograph of a component after construction is acquired. A first program implemented on a communication terminal device displays a construction photograph solicitation icon soliciting a construction photograph on a display panel, when there is a component without the construction photograph in a first database stored in a memory of a second server. When accepting a selection of the construction photograph solicitation icon, the first program launches an imaging program and acquires photograph data. Alternatively, the first program acquires photograph data selected from a folder. The first program sends the acquired photograph data to the second server through a first server. A third program of the second server registers the sent photograph data in a second database. An operator selects one photograph from photographs shown by a plurality pieces of photograph data registered in the second database, and registers the photograph data of the selected photograph in the first database.

Abstract: A display driver integrated circuit and a display driving method are provided. The display driver integrated circuit is suitable for driving a display panel of an electronic device. The display driver integrated circuit includes an image processing circuit, a timing controller, and a data driving circuit. The image processing circuit is configured to generate an output image based on time information, a background image, and an original time indication image. The timing controller is coupled to the image processing circuit, and configured to receive the output image. The data driving circuit is coupled to the timing controller, and configured to receive the output image and generate data voltages according to the output image. The data driving circuit drives the display panel according to the data voltages regarding to the output image.

Abstract: A human model such as a 3D human mesh may be generated for a person in a medical environment based on one or more images of the person. The images may be captured using a sensing device that may be attached to an existing medical device such as a medical scanner in the medical environment. Such an arrangement may ensure that unblocked views of the person (e.g., body keypoints of the person) may be obtained and used to generate the human model. The position of the medical device in the medical environment may be determined and used to facilitate the human model construction such that the pose and body shape of the person in the medical environment may be accurately represented by the human model.

Abstract: A method is provided that includes generating a four-dimensional (4D) model of an environment based on three-dimensional (3D) coordinates of the environment captured at a first point in time. The method further includes updating the 4D model based at least in part to an update to at least a subset of the 3D coordinates of the environment captured at a second point in time. The method further includes enriching the 4D model by adding supplemental information to the model.

Abstract: Some embodiments provide a mapping application that displays a rotation of a 3D map and corresponding rotation of a set of map labels overlaying the 3D map in response to receiving input to rotate the 3D map. When a particular map label in the set of map labels rotates towards an upside down orientation, the mapping application also replaces the particular map label with a version of the particular map label arranged in a right side up orientation to prevent the particular map label from being displayed in the upside down orientation in the 3D map.

Abstract: Embodiments are disclosed for synchronously editing locally repeating vector geometry. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a selection of a first plurality of segments of a vector-based object to be edited, generating a stencil mask of the first plurality of segments, the stencil mask representing segment placement and primitive types for each of the first plurality of segments, identifying a second plurality of segments of the vector-based object using the stencil mask and a stencil predicate, determining a transform between the first plurality of segments and the second plurality of segments, receiving an edit to the first plurality of segments, and applying the edit to the second plurality of segments using the transform.

Abstract: One embodiment of a method for rendering one or more graphics images includes tracing one or more rays through a graphics scene; storing one or more hit points based on one or more coordinate frames associated with one or more voxels of a grid, where the one or more rays intersect one or more surfaces of geometry in the one or more voxels; and rendering one or more graphics images based on the one or more hit points that are stored.

Abstract: A display device includes a display unit, and a control unit that displays a flight route of a flying object flying while photographing surroundings of a crane on the display unit. The control unit is configured to display the crane and the flight route on the display unit, and to display the flight route in a display mode viewed from at least two different directions.

Abstract: This application discloses a three-dimensional human pose estimation method performed by a computer device. An initialization pose estimation result of a single video frame in a video frame sequence of n views is extracted based on a neural network model. Single-frame and single-view human pose estimation is performed on the initialization pose estimation result for each video frame, to obtain n single-view pose estimation sequences respectively corresponding to the n views. Single-frame and multi-view human pose estimation is performed according to single-view pose estimation results with the same timestamp in the n single-view pose estimation sequences, to obtain a multi-view pose estimation sequence. Multi-frame and multi-view human pose estimation is performed on a multi-view pose estimation result in the multi-view pose estimation sequence, to obtain a multi-view and multi-frame pose estimation result. Therefore, accuracy of human pose estimation is improved.

Abstract: Moving images are switched naturally when development of a story is switched according to a situation. An image display system includes a display unit that displays a moving image, an acquisition unit that acquires a motion of a user viewing the moving image displayed by the display unit, a determination unit that determines whether or not the motion of the user acquired by the acquisition unit satisfies a predetermined switching condition in which the user does not recognize switching between images even when the moving image displayed by the display unit is switched from first moving image data to second moving image data different from the first moving image, and a switching unit that switches the moving image displayed by the display unit from the first moving image data to the second moving image data when the determination unit determines that the switching condition is satisfied.

Abstract: Embodiments of the present disclosure disclose an image processing method for a head-mounted display device, an apparatus and an electronic device. The method includes: acquiring a to-be-processed image and eyeball-tracking information; determining a first region of interest of an eyeball on the to-be-processed image according to the eyeball-tracking information; assigning a weight value to each pixel point in the to-be-processed image, wherein weight values of pixel points located in the first region of interest are higher than weight values of pixel points located in other regions which are regions other than the first region of interest; adjusting a pixel value of each pixel point according to the weight value corresponding thereto so as to obtain a processed image.

Abstract: An information processing device including a control unit that controls a display terminal that arranges a virtual object in a real space by using a coordinate system associated with the real space, in which the control unit determines whether to update first spatial information of a first display terminal and second spatial information of a second display terminal, which are used for determining a position of the virtual object, based on a first operation state of the first display terminal and a second operation state of the second display terminal that shares the virtual object with the first display terminal.

Abstract: A method of detecting a collision of objects, an electronic device and a storage medium, which may be applied to a field of electronic technology, in particular to a field of intelligent transportation and map navigation technology. The method includes: acquiring an attribute information of each of two objects, including a rotation position of the object relative to a predetermined three-dimensional space and a size information of the object projected onto a predetermined two-dimensional plane; determining a size information of a bounding box for each object according to the size information of each object, a center of the bounding box is located at the rotation position of the object; and determining a collision result according to the rotation positions of the two objects and the size information of the bounding boxes for the two objects.

Abstract: The disclosed technology relates to detection of attention exhibited by a second user to a first user's avatar in a virtual reality environment. A first user's avatar activity and the second user's activity are evaluated to determine the second user's (e.g., via another avatar) attention level directed towards first user's avatar. Information can be sent to notify the first user that the second user's attention level has satisfied attention-level criterion data. For example, if the second user is viewing and/or is within audio range of the first user for too long of a time or too frequently, then the first user can be notified and/or other action taken. Remedial action is available for negative attention, such blocking the second user from experiencing the first user in the environment. Detection of attention for positive purposes is available, such as to reward an influencer whose avatar is receiving significant attention from followers.

Abstract: In some embodiments, a method receives a first request for a first time window and a first viewport position for a client that is viewing a 360 degree video. A first message is sent to indicate a computing device will push content in a stream for a first one or more time windows other than the first time window. The method pushes content based on the first viewport position for the first time window to the client. A second request is received to change from the first viewport position to a second viewport position. The method sends a second message to indicate the computing device will push content in the stream for a second one or more time windows without canceling the stream. The method pushes content based on the second viewport position in the 360 degree video for the second one or more time windows to the client.

Abstract: This specification relates to reconstructing three-dimensional (3D) scenes from two-dimensional (2D) images using a neural network. According to a first aspect of this specification, there is described a method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising: receiving a single two-dimensional image; identifying all objects in the image to be reconstructed and identifying the type of said objects; estimating a three-dimensional representation of each identified object; estimating a three-dimensional plane physically supporting all three-dimensional objects; and positioning all three-dimensional objects in space relative to the supporting plane.

Abstract: A method includes: receiving, by a computing device, user input to move content from an augmented reality (AR) interface to a virtual reality (VR) interface; obtaining, by the computing device, security levels of users in a VR environment associated with the VR interface; determining, by the computing device and based on the security levels, which of the users in the VR environment is permitted to see the content; and changing, by the computing device, at least one of the AR interface to the VR interface based on the determining.

Abstract: The present disclosure relates to a method for ultrasound imaging and a related equipment, the method for ultrasound imaging comprising: obtaining ultrasound body data of a test object; determining position information of a target anatomical structure from the ultrasound body data; determining an ultrasound image of the target anatomical structure from the ultrasound body data according to the position information of the target anatomical structure; and displaying the ultrasound image.

Abstract: Systems, devices, media, and methods are presented for object modeling using augmented reality. An object modeling mode for generating three-dimensional models of objects is initiated by one or more processors of a device. The processors of the device detect an object within a field of view. Based on a position of the object, the processors select a set of movements forming a path for the device relative to the object and cause presentation of at least one of the movements. The processors detect a set of object surfaces as portions of the object are positioned in the field of view. In response to detecting at least a portion of the object surface, the processors modify a graphical depiction of a portion of the object. The processors then construct a three-dimensional model of the object from the set of images, depth measurements, and IMU readings collected during the reconstruction process.

Abstract: A method is described comprising: applying a random pattern to specified regions of an object; tracking the movement of the random pattern during a motion capture session; and generating motion data representing the movement of the object using the tracked movement of the random pattern.

Abstract: Ray-tracing for terrain mapping is provided. A system of an aerial vehicle can identify points generated from data captured by a sensor of the aerial vehicle. The points can each indicate a respective altitude value of a portion of terrain. The system can determine, based on the altitude values of the points, a threshold altitude of the terrain, and can identify a boundary defined in part based on the threshold altitude of the terrain. The system can generate a terrain map for the terrain based on applying a ray-tracing process to the points. The ray-tracing process can be performed within the boundary, using the points as respective sources and the aerial vehicle as a destination. The system can present a graphical representation of the terrain map in a graphical user interface of the aerial vehicle.

Abstract: Systems and methods of delegating media capturing functionality from one device to another are presented. A first device configured with an object recognition engine captures a media representation of an environment and identifies an object within that environment. Then based on matched object traits from a database, the engine selects a delegation rules set, and delegates certain media capturing functionality to a second device according to the selected delegation rules set.

Abstract: A non-transitory machine-readable media stores instructions that, when run by one or more processors, cause the one or more processors to store a deformable model of a patient anatomy and deform the deformable model based on a measured deformation of a branched anatomical structure of the patient anatomy. The deformable model includes a skeleton tree of nodes and linkages representing the branched anatomical structure of the patient anatomy. Each of the nodes is located at a respective bifurcation of the branched anatomical structure, and at each respective bifurcation the corresponding linkages include an orientation. The deformable model is deformed by modifying the orientations of the linkages of the branched anatomical structure.

Abstract: An exemplary system accesses imagery of a surgical space captured by different imaging modalities and, based on the accessed imagery, generates composite imagery that includes integrated representations of the surgical space as captured by the different imaging modalities. An exemplary composite image includes a representation of the surgical space as captured by a first imaging modality augmented with an integrated representation of the surgical space as captured by a second imaging modality. The integrated representation of the surgical space as captured by the second imaging modality may be selectively movable and may be generated based on first imagery of the surgical space captured by the first imaging modality and second imagery of the surgical space captured by the second imaging modality in a manner that provides a visually realistic appearance of depth.

Abstract: A composite medical imaging system may direct a display device to display an image captured by an imaging device and showing a view of a surgical area and display an augmentation region within the image and that shows supplemental content. The view of the surgical area shows surface anatomy located at the surgical area and an object located at the surgical area. The augmentation region creates an occlusion over at least a portion of the view of the surgical area. The system may detect an overlap in the image between at least a portion of the object and at least a portion of the augmentation region. In response to the detection of the overlap, the system may adjust the image to decrease an extent of the occlusion within the overlap by the augmentation region.

Abstract: An information processing device includes an acquisition unit, a setting unit, a determination unit, and a control unit. The acquisition unit acquires an operation angle that is an angle formed by a first direction in a predetermined space pointed by a user and a second direction in the predetermined space pointed by the user. The setting unit sets, as a reference angle, the operation angle acquired at a time point when an instruction to start moving a virtual object on a line extending in the first direction is detected. The determination unit determines whether or not the operation angle acquired in response to a change in the second direction is equal to or more than the reference angle.

Abstract: First, an image processing apparatus obtains data of a captured image obtained by image capturing with an image capturing apparatus that captures an image of a surrounding of a reference point, and obtains distance information indicating a distance from the reference point to an object present in a vicinity of the reference point. Next, the image processing apparatus obtains first three-dimensional shape data corresponding to a shape of the object, based on the distance information. Then, the image processing apparatus obtains second three-dimensional shape data that corresponds to the surrounding of the reference point other than the object and that is formed of one or more flat planes or curved planes. Then, the image processing apparatus obtains third three-dimensional shape data in which the first three-dimensional shape data and the second three-dimensional shape data are integrated, and maps the captured image to the third three-dimensional shape data.

Abstract: A method, apparatus, system, and computer program product provide the ability to extract level information and reference grid information from point cloud data. Point cloud data is obtained and organized into a three-dimensional structure of voxels. Potential boundary points are filtered from the boundary cells. Level information is extracted from a Z-axis histogram of the voxels positioned along the Z-axis of the three-dimensional voxel structure and further refined. Reference grid information is extracted from an X-axis histogram of the voxels positioned along the X-axis of the three-dimensional voxel structure and a Y-axis histogram of the voxels positioned along the Y-axis of the three-dimensional voxel structure and further refined.

Abstract: Methods, devices and data stream are provided for signaling and decoding information representative of restrictions of navigation in a volumetric video. The data stream comprises metadata associated to video data representative of the volumetric video. The metadata comprise data representative of a viewing bounding box, data representative of a curvilinear path in the 3D space of said volumetric video; and data representative of at least one viewing direction range associated with a point on the curvilinear path.

Abstract: An image processing device acquires a plurality of images of an object to be measured by an imaging device, the plurality of images being taken while varying a focal position, and corrects blurring in the acquired plurality of images based on a point spread function for each focal position of an image forming optical system in the imaging device. The image processing device also calculates, for the plurality of images after correction of the blurring, an evaluation value for a focusing degree for each pixel, and generates three-dimensional shape data of the object to be measured based on the calculated evaluation value for the focusing degree for each pixel.

Abstract: A robot-assisted endoscope system and control methods thereof allow a user to perform intraluminal interventional procedures using a steerable sheath. A processor generates a ghost image based on a non-real-time insertion trajectory of the sheath, and a real-time image based on a real-time insertion trajectory for inserting an interventional tool through the sheath towards the target site. A display screen outputs navigation guidance data for informing a user how to manipulate the distal section of the sheath towards the target site such that the real-time image overlaps or coincides with at least part the ghost image and the real-time insertion trajectory becomes aligned with the non-real-time insertion trajectory.

Abstract: A method for determining a focal point depth of a user of a three-dimensional (“3D”) display device includes tracking a first gaze path of the user. The method also includes analyzing 3D data to identify one or more virtual objects along the first gaze path of the user. The method further includes when only one virtual object intersects the first gaze path of the user identifying a depth of the only one virtual object as the focal point depth of the user.

Abstract: The present disclosure generally relates to user interfaces for adjusting simulated image effects. In some embodiments, user interfaces for adjusting a simulated depth effect is described. In some embodiments, user interfaces for displaying adjustments to a simulated depth effect is described. In some embodiments, user interfaces for indicating an interference to adjusting simulated image effects is described.

Abstract: An electronic device according to the present invention includes at least one memory and at least one processor which function as: a gaze acquisition unit configured to acquire right gaze information related to a gaze of a right eye of a user that wears a display apparatus of an optical see-through type on a head and left gaze information related to a gaze of a left eye of the user; and a correlation acquisition unit configured to acquire, as information on personal differences, correlation information related to correlation between the gaze of the right eye and the gaze of the left eye on a basis of the right gaze information and the left gaze information.

Abstract: A method for generating a three-dimensional (3D) model of an object includes receiving a two-dimensional (2D) view of at least one object as an input, measuring geometrical shape coordinates of the at least one object from the input, identifying texture parameters of the at least one object from the input, predicting geometrical shape coordinates and texture parameters of occluded portions of the at least one object in the 2D view by processing the measured geometrical shape coordinates of the at least one object, the identified texture parameters of the at least one object, and the occluded portions of the at least one object, and generating a 3D model of the at least one object by mapping the measured geometrical shape coordinates and the identified texture parameters to the predicted geometrical shape coordinates and the predicted texture parameters of the occluded portions of the at least one object.

Abstract: An object forming structure includes: at least two first original images, respectively provided with different vectors, and extended for forming at least two first original image three-dimensional shapes; at least one intersect fixed point, extended with at least one intersect direction through the intersect fixed point, wherein at least two second image three-dimensional shapes are formed through the first original image three-dimensional shapes being extended; and at least two third image three-dimensional shapes, stacked with the at least two second image three-dimensional shapes, wherein a Boolean function is utilized for confirming a selected desired zone for forming a new object formation; wherein, the new object formation formed via the third image three-dimensional shapes is obtained through a combination of intersect or union or equalize the different vectors, the first original image three-dimensional shapes are presented via the corresponding vectors for presenting different shapes in other angles.