Abstract: A method for providing real-time three-dimensional facial animation from video is provided. The method includes collecting images of a subject, and forming a three-dimensional mesh for the subject based on a facial expression factor and a head pose of the subject extracted from the images of the subject. The method also includes forming a texture transformation based on an illumination parameter associated with an illumination configuration for the images from the subject, forming a three-dimensional model for the subject based on the three-dimensional mesh and the texture transformation, determining a loss factor based on selected points in a test image from the subject and a rendition of the test image by the three-dimensional model, and updating the three-dimensional model according to the loss factor. A system and a non-transitory, computer-readable medium storing instructions to perform the above method are also provided.

Abstract: A multiple graphics processing unit (GPU) based parallel graphics system comprising multiple graphics processing pipelines with multiple GPUs supporting a parallel graphics rendering process having an object division mode of operation. Each GPU comprises video memory, a geometry processing subsystem and a pixel processing subsystem. According to the principles of the present invention, pixel (color and z depth) data buffered in the video memory of each GPU is communicated to the video memory of a primary GPU, and the video memory and the pixel processing subsystem in the primary GPU are used to carry out the image recomposition process, without the need for dedicated or specialized apparatus.

Abstract: To enable shared user experiences using augmented reality systems, shared reference points must be provided to have consistent placement (position and orientation) of virtual objects. Furthermore, the position and orientation (pose) of the users must be determinable with respect to the same shared reference points. However, without highly sensitive and expensive global positioning system (GPS) devices, pose information can be difficult to determine to a reasonable level of accuracy. Therefore, what is provided is an alternative approach to determining pose information for augmented reality systems, which can be used to perform location based content acquisition and sharing. Further, what is provided is an alternative approach to determining pose information for augmented reality systems that uses information from already existing GPS devices.

Abstract: Described herein is a graphics processing unit (GPU) comprising a first processing cluster to perform parallel processing operations, the parallel processing operations including a ray tracing operation and a matrix multiply operation; and a second processing cluster coupled to the first processing cluster, wherein the first processing cluster includes a floating-point unit to perform floating point operations, the floating-point unit is configured to process an instruction using a bfloat16 (BF16) format with a multiplier to multiply second and third source operands while an accumulator adds a first source operand with output from the multiplier.

Abstract: Embodiments of this application provide a rendering method and apparatus, and the like. The method includes: A processor (which is usually a CPU) modifies a rendering instruction based on a relationship between a first frame buffer and a second frame buffer, so that a GPU renders a rendering job corresponding to the first frame buffer to the second frame buffer based on a new rendering instruction. In this application, render passes of one or more frame buffers are redirected to another frame buffer. In this way, memory occupation in a rendering process of an application program is effectively reduced, bandwidth of the GPU is reduced, and power consumption can be reduced.

Abstract: A method for augmenting procedures at production equipment includes: linking a first description of a first step in a procedure and a capture type to a first capture field; generating a digital draft procedure comprising descriptions of steps extracted from the procedure and the first capture field specifying data capture of the first capture type from an equipment unit; generating augmented guidance for the first step in the procedure based on visual content recorded by a mobile device—assigned to an exemplary operator—during completion of the first step in the digital draft procedure; linking the augmented guidance to a location proximal the equipment unit based on locations of the first mobile device during completion of the first step in the digital draft procedure; and generating an augmented digital procedure based on steps in the procedure, the augmented guidance, and definitions for data capture according to the first capture field.

Abstract: Systems and methods for generating a three-dimensional (3D) model of a user's dental arch based on two-dimensional (2D) images of dental impressions include a model training system that provides a machine learning model using training image(s) of a dental impression of a respective dental arch and a 3D training model of the respective dental arch. A model generation system receives first image(s) of a first dental impression of a user's dental arch and second image(s), which may be of the first dental impression or a second dental impression of the dental arch. The model generation system generates a first and second 3D model of the dental arch by applying the first image(s) and second image(s) to the machine learning model.

Abstract: Embodiments described herein include an expressive icon system to present an animated graphical icon, wherein the animated graphical icon is generated by capture facial tracking data at a client device. In some embodiments, the system may track and capture facial tracking data of a user via a camera associated with a client device (e.g., a front facing camera, or a paired camera), and process the facial tracking data to animate a graphical icon.

Abstract: A realistic feather growth may be represented between two surface manifolds in a modeling system. To perform the feather growth, a feather groom for a plurality of feathers between an inner shell of a creature and an outer shell of the creature is received. An inner manifold for the inner shell and an outer manifold for the outer shell is determined with a plurality of follicle points and a plurality of tip points. A first surface contour definition for the inner manifold and a second surface contour definition for the outer manifold is determined and used to determine a volumetric vector field between the inner manifold and the outer manifold. Thereafter, the plurality of feathers is generated between the inner manifold and the outer manifold using the follicle points, the tip points, and the volumetric vector fields.

Abstract: There is provided an information processing apparatus and an information processing method that can enhance the image quality while, at the same time, reducing increase in client's throughput. Image data of a plurality of projection directions is generated by projecting 3D data in a plurality of projection directions and converting the 3D data into two-dimensional data, and projection direction information indicating the projection direction of each piece of the image data is generated as projection metadata. Further, the projection direction information includes additional projection direction identification information that indicates that the image data has been generated by projection in additional projection directions. The present technology is applicable, for example, to a data generation apparatus that generates data for delivering a point cloud.

Abstract: A map having surfaces that are depicted at different levels that are not related to topography, with boundaries between the surfaces, where the boundaries are disposed at travel ways. The travel ways form cliff faces in the map between the surfaces, with information items disposed on the cliff faces at positions corresponding to items of interest at locations along the travel ways where the information items are disposed.

Abstract: Methods and systems are disclosed for creating a computerized 3D model to include material property information for one or more regions of image textures of the computerized 3D model, including a method comprising: creating a computerized 3D model having image textures; examining a portion of a first image texture of the computerized 3D model having unknown material properties; assigning a material having a material property to the portion of the first image texture to indicate a physical material of a physical object represented by the portion of the first image texture, the material property having material property information about the physical materials; associating the material property information with the portion of the first image texture; and replacing the portion of the first image texture in the 3D model with a simulated texture of the assigned material.

Abstract: To provide an I/O signal information system in which a monitoring target portion and I/O signal information are automatically associated with each other and displayed on a display screen when monitoring a facility including a robot. An object identification unit identifies an object based on a correlation between a change of an actual photographed image displayed on a display device by actual photographed data supplied from an imaging device and a change of I/O signal information, and a display control unit causes the display device to display an augmented reality image in a display form in which an image of the I/O signal information has a specific relationship with an image of the object identified.

Abstract: A method of rendering an image of an environment is disclosed. Environment data for the environment is accessed. The environment data corresponds to a frame of a video. A plurality of subframes associated with the frame is determined. An angle for each of the plurality of subframes is determined. One or more lights corresponding to the environment are selected. For each light of the one or more lights, a shadow map is generated. The shadow map corresponds to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe. The image of the environment is rendered. The rendering includes using the generated shadow map for each light of the one or more lights.

Abstract: An information processing apparatus includes a processor configured to acquire three-dimensional shape data of a product or a part constituting the product and attribute information given to each of a face and an end portion constituting the three-dimensional shape data, recognize a type of a three-dimensional note regarding the three-dimensional shape data from the attribute information, recognize an orientation and a position where the three-dimensional note is arranged from the three-dimensional shape data, and arrange the three-dimensional note in the three-dimensional shape data in accordance with the type of the three-dimensional note and the orientation and the position where the three-dimensional note is arranged.

Abstract: According to a first aspect of this specification, there is described a computer-implemented method of tagging video frames. The method comprises generating, using a frame tagging model, a tag for each of a plurality of frames of an animation sequence. The frame tagging model comprises: a first neural network portion configured to process, for each frame of the plurality of frames, a plurality of features associated with the frame and generate an encoded representation for the frame. The frame tagging model further comprises a second neural network portion configured to receive input comprising the encoded representations of each frame and generate output indicative of a tag for each of the plurality of frames.

Abstract: A method for providing sign language is disclosed. The method includes receiving, by an electronic device, a natural language information input from at least one source for conversion into sign language. The natural language information input includes at least one sentence. The method further includes predicting, by the electronic device, an emphasis score for each word of the at least one sentence based on acoustic components. The method further includes rephrasing, by the electronic device, the at least one sentence based on the emphasis score of each of the words. The method further includes converting, by the electronic device, the at least one rephrased sentence into the sign language. The method further includes delivering, by the electronic device, the sign language.

Abstract: Modulated display AR tracking systems and methods include a display configured to transmit first image content, the image content comprising a first image content frame and a second image content frame and a device comprising an image sensor, a processor and a memory. The device identifies the first image content frame and the second image content frame by distinguishing at least one feature of the first image content frame. The device recognizes at least one tracking pattern embedded of the second image content frame and retrieves a graphical element based on the at least one tracking pattern of the second image content frame. The device displays augmented image content including the first image content frame and the graphical element.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network. The policy neural network is used to select actions to be performed by an agent that interacts with an environment by receiving an observation characterizing a state of the environment and performing an action from a set of actions in response to the received observation. A trajectory is obtained from a replay memory, and a final update to current values of the policy network parameters is determined for each training observation in the trajectory. The final updates to the current values of the policy network parameters are determined from selected action updates and leave-one-out updates.

Abstract: A computer program product may cause one or more processors to generate stereoscopic images of one or more 3D models within a 3D model space. As part of the generation of the stereoscopic images, special case surfaces that are non-flat and specularly reflective or refractive are rendered in a special manner. The special manner involves rendering a texture for the special case surface based on a third projection corresponding to a third viewpoint that is spaced from both a first viewpoint (i.e., a left eye viewpoint) and a second viewpoint (i.e., a right eye viewpoint). Accordingly, when rendering first and second images (i.e., images corresponding respectively to the first and second viewpoints), the texture corresponding to the third viewpoint may be applied to the special case surface in both the first and second images.