Abstract: Improved video compression and video streaming systems and methods are disclosed for environments where camera motion is common, such as cameras incorporated into head-mounted displays. This is accomplished by combining a 3D representation of the shape of the user's environment (walls, floor, ceiling, furniture, etc.), image data, and data representative of changes in the location and orientation (pose) of the camera between successive image frames, thereby reducing data bandwidth needed to send streaming video in the presence of camera motion.

Abstract: A virtual reality (VR) system that includes a three-dimensional (3D) point cloud having a plurality of points, a VR viewer having a current position, a graphics processing unit (GPU), and a central processing unit (CPU). The CPU determines a field-of-view (FOV) based at least in part on the current position of the VR viewer, selects, using occlusion culling, a subset of the points based at least in part on the FOV, and provides them to the GPU. The GPU receives the subset of the plurality of points from the CPU and renders an image for display on the VR viewer based at least in part on the received subset of the plurality of points. The selecting a subset of the plurality of points is at a first frame per second (FPS) rate and the rendering is at a second FPS rate that is faster than the first FPS rate.

Abstract: A mechanism is described for facilitating using of a shared local memory for register spilling/filling relating to graphics processors at computing devices. A method of embodiments, as described herein, includes reserving one or more spaces of a shared local memory (SLM) to perform one or more of spilling and filling relating to registers associated with a graphics processor of a computing device.

Abstract: An immersive device and method for streaming of immersive media are provided. The method includes receiving body motion coordinates of a user in each of three dimensions and determining deviation in the body motion by a first immersive device. A dominant direction of motion based on the deviation calculated is then determined. A request by the first immersive device for at least one media frame to be displayed is received by a second immersive device. The received request is processed by the second immersive device and the media frame to be displayed is segmented into a plurality of tiles by the second immersive device. One tile with the first pixel density and a remainder of the tiles of the plurality of tiles with a second pixel density are stitched by the second immersive device to render one immersive media frame which is displayed to the user.

Abstract: System includes: an operation terminal that (i) receives an operation instruction that is given by the operator to a movable section of a machine and (ii) senses an operation standby state that allows the operation instruction to be received; an image sensor estimating section configured to estimate a positional relationship between the operator and the machine; a model generating section configured to, in response to sensing of the operation standby state by the operation terminal, generate an operating direction indicating image; and in accordance with the positional relationship, in a direction that is in accordance with a direction in which the operator views the machine, the operating direction indicating image indicating an operating direction of the movable section; an combining section configured to generate a combined image obtained by combining the operating direction indicating image with a captured image of the movable section that has been photographed.

Abstract: Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for virtual-life-based human-machine interaction. One of the method includes obtaining cognitive data for a target user in response to a human-machine interaction from the target user by performing perception analysis on the target user. Target response content is identified based on the cognitive data and sent to the target user. A virtual interactive animation that comprises a virtual life image is dynamically generated based on the cognitive data and the target response content, where the virtual life image has an animation effect that matches the human-machine interaction performed by the target user.

Abstract: Disclosed herein is a web-based videoconference system that allows for video avatars to navigate within the virtual environment. The system has a presented mode that allows for a presentation stream to be texture mapped to a presenter screen situated within the virtual environment. The relative left-right sound is adjusted to provide sense of an avatar's position in a virtual space. The sound is further adjusted based on the area where the avatar is located and where the virtual camera is located. Video stream quality is adjusted based on relative position in a virtual space. Three-dimensional modeling is available inside the virtual video conferencing environment.

Abstract: Systems and methods are described for displaying 360-degree video using a viewing direction and/or a viewpoint position that is determined based on a user-selected level of automation. If full automation is selected by the user, the viewing direction and/or viewport position is determined by a defined viewing path. If partial automation is selected, the viewing direction and/or viewport position is determined by user input in combination with the defined viewing path. For example, a high-pass-filtered version of the user input may be added to the defined viewing path to obtain a partially-automated viewing direction and/or viewpoint position. Example systems and methods may be implemented using a head-mounted display (HMD).

Abstract: A system and method for matching audio content to virtual reality visual content. The method includes analyzing received visual content and received metadata to determine an optimal audio source associated with the received visual content; configuring the optimal audio source to capture audio content; synthesizing the captured audio content with the received visual content; and providing the synthesized captured audio content and received visual content to a virtual reality (VR) device.

Abstract: Aspects of the subject disclosure may include, for example, a method for training a deep learning model that includes encoding a content item; generating a blended image by combining a background image and the encoded content; decoding the blended image to generate decoded content corresponding to the content item; and defining or specifying a loss function related to the deep learning model. The method also includes determining values of training parameters for the deep learning model to minimize the loss function, thereby obtaining a trained deep learning model. The method also includes an information concealing procedure using the trained deep learning model to conceal user content by encoding the user content and blending the encoded user content with a user-selected image; the information concealing procedure is substantially independent of the user-selected image. Other embodiments are disclosed.

Abstract: Provided are an optimization system and method of a deep learning model. According to example embodiments, by optimizing the structure of the deep learning model appropriately for a target dataset without fixing the structure of the deep learning model, it is possible to generate a model structure capable of having high performance on the target dataset and also saving resources.

Abstract: The present specification relates generally to geopositioning and more specifically relates to a method, system and computer program product for calibrating absolute user location and bearing on a low geolocation accuracy computing device or a computing device without location capabilities by using a high geometric resolution map.

Abstract: An augmented reality system generates computer-mediated reality on a client device. The client device has sensors including a camera configured to capture image data of an environment and a location sensor to capture location data describing a geolocation of the client device. The client device creates a three-dimensional (3-D) map with the image data and the location data for use in generating virtual objects to augment reality. The client device transmits the created 3-D map to an external server that may utilize the 3-D map to update a world map stored on the external server. The external server sends a local portion of the world map to the client device. The client device determines a distance between the client device and a mapping point to generate a computer-mediated reality image at the mapping point to be displayed on the client device.

Abstract: A method includes obtaining video data including a representation of a first plurality of motions of a real-world entity. The method includes determining, from the video data, a plurality of estimated torque values using a motion controller. The plurality of estimated torque values is associated with a plurality of real-world joints corresponding to the first plurality of motions. The method includes generating a second plurality of motions of a virtual agent by providing the plurality of estimated torque values to a corresponding plurality of virtual joints of the virtual agent. Movement of the virtual agent is controllable by the plurality of virtual joints, corresponding to the plurality of real-world joints of the real-world entity. The method includes, responsive to a determination that a comparison between the first plurality of motions and the second plurality of motions does not satisfy a performance metric, changing an operational value of the motion controller.

Abstract: There is disclosed a method of creating a three-dimensional image comprising: establishing a mapping between a two-dimensional template and the three-dimensional image; applying a graphic to the two-dimensional template; receiving the two-dimensional template with the graphic applied; and creating the three-dimensional image based on the mapping and the applied graphic.

Abstract: A system and method for generating a set of samples stratified across two-dimensional elementary intervals of a two-dimensional space is disclosed within the application. A computer-implemented technique for generating the set of samples includes selecting an elementary interval associated with a stratification of the two-dimensional space, initializing at least one data structure that indicates valid regions within the elementary interface based on other samples previously placed within the two-dimensional space, and generating a sample in a valid region of the elementary interval utilizing the at least one data structure to identify the valid region prior to generating the sample. In some embodiments, the data structures comprise a pair of binary trees. The process can be repeated for each elementary interval of a selected stratification to generate the set of stratified two-dimensional samples.

Abstract: In one example, a method performed by a processing system includes controlling a first device in a first location to project a virtual presences in a current field of view of a subscriber of a virtual presence projection service, computing a moving trajectory of the subscriber from the first location to a second location, identifying a second device that is located along the moving trajectory, wherein the second device includes a second set of resources for projecting virtual presences, and wherein the second device is registered with the virtual presence projection service to lend the second set of resources to subscribers of the virtual presence projection service, and controlling the second device to project the virtual presence in the current field of view of the subscriber when the subscriber is no longer within range of the first device but is within range of the second device.

Abstract: A driver-assistance device includes: a memory; and a processor including hardware. The processor is configured to perform switching control on an operation of a function that is included in a wearable device to be worn by a user and that acquires information on privacy, based on an occupancy state indicating a relative positional relation between a vehicle and the user.

Abstract: The invention provides an augmented reality device and method for assisting a user in choosing appropriate luminaire fixtures to install within their home. A user may point the camera of a mobile device toward the region or location in a room where a new luminaire is desired, and based upon data generated by an orientation determination means included within the device, an appropriate luminaire or luminaire category is selected automatically for the user from a stored catalogue or database. Once an appropriate luminaire has been chosen, it is inserted within an image captured by the camera to generate an augmented reality image depicting the luminaire fixture in place within the user's room.

Abstract: A method of interaction using augmented reality includes capturing a first video image using a camera, generating first augmented reality (AR) coordinates corresponding to the first video image, transmitting AR coordinates and first video image to remote user, receiving first video image and annotations from remote user, capturing a second video image using a camera, generating second AR coordinates corresponding to the second video image, and viewing annotations registered to second video image.