Abstract: Embodiments of the present disclosure provide an end-cloud collaborative media data processing method and apparatus, a device and a storage medium, where the method includes: calling a target processing flow in response to a first operation triggering a target functionality, where the target processing flow includes a local algorithm node and a remote algorithm node, and the target functionality is used for adding a special effect to media data to be processed; executing, based on the target processing flow, a corresponding local algorithm node and remote algorithm node to obtain first processing data output by the local algorithm node and second processing data output by the remote algorithm node; and generating third processing data through the first processing data and/or the second processing data.

Abstract: A display controller controls a display device provided for a mobile object to display a display image including at least one item of contents of data. The display controller includes an image data acquisition unit configured to obtain image data from an imaging device configured to capture an image around the mobile object, and a display-image generation unit configured to generate the display image. The display-image generation unit changes a display mode of the at least one item of contents of data included in the display image to increase visual recognizability of a desired item of contents of data included in the at least one item of contents of data to a level at least higher than visual recognizability of an area in the image data not including the desired item of contents of data.

Abstract: A system and method for generating a virtual content to a physical object is described. A processor includes an augmented reality application. The augmented reality application creates virtual content at the head mounted device, and associates the virtual content with predefined conditions based on data from sensors embedded in the head mounted device at a time of creation of the virtual content. The virtual content is displayed in a display of the head mounted device in response to sensor data satisfying the predefined conditions.

Abstract: A method and system for objectively tracking and analyzing the social and emotional activity of a patient using an augmented reality computing device is provided. A patient is permitted to manually manipulate a target object in the physical world while viewing an augmented version showing a unique animated character representing either an abstract language, emotions, or social skills, depending on the module. The present system tracks and records the active face and the time spent on the active face, where the active face is the face upon which the patient's focus is automatically estimated, through calculation, to be trained upon. An observer views the session, the data recorded, and an automatically generated graphical representation of the data, which permits the observer to speak to patient regarding the character or scene rendered on the face which is determined to be the active face, helping the student engage in the session.

Abstract: Systems and methods for generating a virtual view of a virtual camera based on an input scene are described. A capturing device typically includes a physical camera and a depth sensor and captures an input scene. A controller determines an actual pose of the capturing device and a desired pose of the virtual camera for showing the virtual view. The controller defines an epipolar geometry between the actual pose of the capturing device and the desired pose of the virtual camera. The controller generates an output image for the virtual camera based on an epipolar relation between the actual pose of the capturing device, the input scene, and the desired pose of the virtual camera.

Abstract: Systems, processes, and techniques to automatically detect and enlarge a speaking one of plurality of participants on one side of a video conference. In at least one embodiment, the speaking participant is identified using one or more heuristics and/or one or more neural networks.

Abstract: A photo filter (e.g., artistic) light field effect system comprises an eyewear device that includes a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the photo filter light field effect system to apply a photo filter selection to: (i) a left raw image or a left processed image to create a left photo filter image, and (ii) a right raw image or a right processed image to create a right photo filter image. The photo filter light field effect system generates, a photo filter light field effect image with an appearance of a spatial rotation or movement, by blending together the left photo filter image and the right photo filter image based on a left image disparity map and a right image disparity map.

Abstract: A method for determining three-dimensional (3D) coordinates of an object surface with a 3D measuring device includes forming from the determined 3D coordinates a mesh having a first face, constructing a voxel array aligned to the first face, obtaining a plurality of images from a first camera having a corresponding plurality of poses, obtaining for each voxel in the voxel array a plurality of voxel values obtained from the corresponding plurality of images, determining for each voxel row a quality value determined based at least in part on an average value of a first quantity and a dispersion of the first quantity, the first quantity based at least in part on first voxel values determined as a function of pose, and determining a distance from a point on the first face to the object surface based at least in part on the determined quality values for the voxel rows.

Abstract: An apparatus and method for efficient image reprojection in a virtual reality system. For example, one embodiment of an apparatus comprises: a sensor interface to collect motion data from one or more sensors during a virtual reality session; graphics circuitry to execute graphics program code to render an image frame during the virtual reality session; a processor to generate motion transform data using the motion data, the motion transform data specifying how the image frame is to be adjusted prior to display; a reprojection engine to perform an in-line reprojection of the frame using the motion transform data to generate a reprojected image frame; and display circuitry to display the reprojected frame.

Abstract: A graphics rendering system is disclosed for generating and streaming graphics data of a 3D environment from a server for rendering on a client in 2.5D. 2D textures can be transmitted in advance of frames showing the textures. Data transmitted for each frame can include 2D vertex positions of 2D meshes and depth data. The 2D vertex positions can be positions on a 2D projection as seen from a viewpoint within the 3D environment. Data for each frame can include changes to vertex positions and/or depth data. A prediction system can be used to predict when new objects will be displayed, and textures of those new objects can be transmitted in advance.

Abstract: Systems and methods are disclosed for directed image capture of a subject of interest, such as a home. Directed image capture can produce higher quality images such as more centrally located within a display and/or viewfinder of an image capture device, higher quality images have greater value for subsequent uses of captured images such as for information extraction or model reconstruction. Graphical guide(s) facilitate content placement for certain positions and quality assessments for the content of interest can be calculated such as for pixel distance of the content of interest to a centroid of the display or viewfinder, or the effect of obscuring objects. Quality assessments can further include instructions for improving the quality of the image capture for the content of interest.

Abstract: Implementations of the subject matter described herein relate to mixed reality object rendering based on ambient light conditions. According to the embodiments of the subject matter described herein, while rendering an object a wearable computing device acquires light conditions of the real world, thereby increasing the reality of the rendered object. In particular, the wearable computing deice is configured to acquire an image of an environment where the wearable computing deice is located. The image is adjusted based on a cement parameter used when the image is captured. Subsequently, ambient light information is determined based on the adjusted image. In this way, the wearable computing deice can obtain more real and accurate emblem light information, so as to render to the user an object with enhanced reality. Accordingly, the user can have a better interaction experience.

Abstract: Disclosed are systems and methods for single image-based body animation. An example method includes receiving an input image, the input image including a body image of a person, extracting the body image of the person from the input image, fitting a generic model to the body image, where the generic model is configured to receive a set of pose parameters corresponding to a pose of the person and generate a generic body shape adopting the pose, generating a three-dimensional (3D) model, where the 3D model is configured to receive a set of further pose parameters corresponding to the pose of the person and generate an output image of the person adopting the pose, the output image including a feature of the body image being omitted from the generic body shape, and providing a further set of further pose parameters to generate a frame of an output video.

Abstract: Systems and method for image generation in a gaze tracking display. A gaze tracking display system includes a graphics processor and display circuitry. The graphics processor is configured to perform foveated rendering of image data, and to output foveated image data. The display circuitry is coupled to the graphics processor. The display circuitry includes a display device and a display controller. The display device is configured to produce a viewable image. The display controller is configured to drive the display device. The display controller includes foveated data reconstruction circuitry configured to produce an image at a resolution of the display device based on the foveated image data received from the graphics processor.

Abstract: Techniques are disclosed for learning a machine learning model that maps control data, such as renderings of skeletons, and associated three-dimensional (3D) information to two-dimensional (2D) renderings of a character. The machine learning model may be an adaptation of the U-Net architecture that accounts for 3D information and is trained using a perceptual loss between images generated by the machine learning model and ground truth images. Once trained, the machine learning model may be used to animate a character, such as in the context of previsualization or a video game, based on control of associated control points.

Abstract: Techniques are disclosed for real-time mapping in a movable object environment. A system for real-time mapping in a movable object environment, may include at least one movable object including a computing device, a scanning sensor electronically coupled to the computing device, and a positioning sensor electronically coupled to the computing device. The system may further include a client device in communication with the at least one movable object, the client device including a visualization application which is configured to receive point cloud data from the scanning sensor and position data from the positioning sensor, record the point cloud data and the position data to a storage location, generate a real-time visualization of the point cloud data and the position data as it is received, and display the real-time visualization using a user interface provided by the visualization application.

Abstract: A photo filter (e.g., artistic/stylized painting) light field effect system includes an eyewear device having a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the stylized image painting effect system to apply a photo filter selection to: (i) a left raw image or a left processed image to create a left photo filter image, and (ii) a right raw image or a right processed image to create a right photo filter image. The stylized image painting effect system generates a photo filter stylized painting effect image with an appearance of a spatial rotation or movement, by blending together the left photo filter image and the right photo filter image based on a left image disparity map and a right image disparity map.

Abstract: A method for determining experience quality of virtual reality (VR) multimedia includes, in a process of playing VR multimedia, obtaining a first sensory parameter, a second sensory parameter, and a third sensory parameter of the VR multimedia, where the first sensory parameter, the second sensory parameter, and the third sensory parameter are obtained by performing sampling separately according to at least two same perceptual dimensions, and are respectively parameters that affect fidelity experience, enjoyment experience, and interaction experience, and determining a mean opinion score (MOS) of the VR multimedia based on the first sensory parameter, the second sensory parameter, and the third sensory parameter of the VR multimedia. Because the third sensory parameter is a parameter that affects the interaction experience, an interaction feature of the VR multimedia is considered.

Abstract: A method of rendering at least one of paths forming an object includes setting an initial value to each of tiles included in a frame based on a position and a proceeding direction of the at least one of paths, calculating a winding number of each of the tiles through which the at least one of paths passes, among the tiles included in the frame, based on the set initial value, and determining whether to perform shading based on the set initial value and the calculated winding number.

Abstract: A method for road surface detection includes receiving ranging data including a plurality of ranging data points, extracting one or more ranging data points lying within a height range from the plurality of ranging data points, dividing the one or more ranging data points into one or more grid cells, setting a first horizontal position of a first cell point of a first grid cell of the one or more grid cells as being centered on the first grid cell, setting a first vertical position of the first cell point, and detecting the road surface based on the first vertical position and first horizontal position of the first cell point.