Abstract: A background scenery portion may be identified in each of a plurality of image sets of an object, where each image set includes images captured simultaneously from different cameras. A correspondence between the image sets may determined, where the correspondence tracks control points associated with the object and present in multiple images. A multi-view interactive digital media representation of the object that is navigable in one or more dimensions and that includes the image sets may be generated and stored.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and modifying digital animations based on user interactions with a unique user interface portraying a one-dimensional layer motion element and/or elements for generating and utilizing animation path for digital design objects and animation layers. The disclosed system can provide a dynamic one-dimensional layer motion element that adapts to a selected animation layer and portrays selectable animation frames from the animation layer. The disclosed systems can provide options for generating and modifying various frames of the digital animation based on user interactions with the one-dimensional layer motion element, an animation timeline, and/or a corresponding animation canvas.

Abstract: A system and method for performing facial recognition is described. In some implementations, the system and method identify points of a three-dimensional scan that are associated with occlusions, such as eyeglasses, to a face of a target subject and remove the identified points from the three-dimensional scan.

Abstract: The present disclosure describes techniques for generating content based on rhythm. The techniques comprises acquiring a plurality of images comprising an object with movements in the plurality of images; determining whether at least one portion of the at least one part of the object in a first image aligns with a target image overlaid on the first image and whether the at least one portion aligns with the target image at a time proximate to a first rhythmic point of a playback of a selected piece of music; segmenting the at least one part of the object from the first image in response to determining that the at least one portion aligns with the target image at the time proximate to the first rhythmic point of the playback of the selected piece of music; and generating a first overlay based on the at least one part of the object.

Abstract: When a multimedia conference between multiple users at multiple locations is established, a first cursor controlled by a first user in a first location is provided. A first event associated with the multimedia conference is detected. In response to detecting the first event associated with the multimedia conference, control of cursors in the multimedia conference can be handled in various ways to enhance the multimedia conference. For example, control of the first cursor can be switched from the first user to a second user at a second location, a second cursor can be provided to the multimedia conference that is controlled by the second user, or control of the first cursor can be merged so that the first cursor can be controlled by both the first user and the second user. This allows for enhanced control of the cursor in the multimedia conference that more closely resembles an in-person conference.

Abstract: A computer graphics production control system is configured to generate scenes (including three-dimensional, deformable characters (“3DD characters”)) that can be manipulated to produce still images and/or animated videos. Such control systems may utilize 3DD characters that are controlled by a series of control points that are positioned and/or moved under the control of the artist. Body characteristics of 3DD characters are modeled as a series of inter-related points (e.g., skin triangles) that can be manipulated under the control of the model and the reference points (e.g., bones) of the body.

Abstract: Described herein is a graphics processing unit (GPU) configured to receive an instruction having multiple operands, where the instruction is a single instruction multiple data (SIMD) instruction configured to use a bfloat16 (BF16) number format and the BF16 number format is a sixteen-bit floating point format having an eight-bit exponent. The GPU can process the instruction using the multiple operands, where to process the instruction includes to perform a multiply operation, perform an addition to a result of the multiply operation, and apply a rectified linear unit function to a result of the addition.

Abstract: Disclosed herein is a web-based videoconference system that allows for video avatars to navigate within a virtual environment. Various methods for efficient modeling, rendering, and shading are disclosed herein.

Abstract: Systems and methods can enable the control the motion of an animated character based on imagery (e.g., captured by an image capture device such as a web camera or “webcam”) which shows a person in motion. Specifically, the animated character can be automatically rendered to have the same motion as the entity shown in the imagery (e.g., in real time). According to one aspect of the present disclosure, the animated character can be rendered by iteratively transforming (e.g., including deforming the actual geometry of) a vector-based surface illustration. Specifically, the systems and methods present disclosure can leverage the scalable and transformable nature of a vector-based surface illustration to provide more realistic motion-controlled animation, in which the underlying geometry of the animated character is able to be adjusted to imitate human motion more realistically (e.g., as opposed to basic rotations of fixed character geometry).

Abstract: The present disclosure generally relates to techniques for presenting operations in a presentation. A technique is provided for allowing a user to insert media items on consecutive slides. During a presentation mode, the media items are determined to be two instances of a same media item, and matched such that the media item plays continuously across multiple slides. Another technique is provided to allow a user to utilize a user input to override a linearly-defined set of actions within a presentation. Another technique is utilized to temporarily select a visually obstructed object in a graphic editing interface such that the visually obstructed object may be manipulated during the temporary selection mode.

Abstract: Systems and methods are disclosed herein for generating content based on format-specific screenplay parsing techniques. The techniques generate and present content by generating new dynamic content structures to generate content segments for output on electronic devices. In one disclosed technique, a first instance of a first character name is identified from the screenplay document. A first set of character data following the first instance of the first character name from the screenplay document and preceding an instance of a second character name from the screenplay document is then identified. Upon identification of the first set of character data, a content structure including an object is generated. The object includes attribute table entries based on the first set of character data. A content segment is generated for output based on the content structure (e.g., a 3D animation of the first character interacting within a scene).

Abstract: Embodiments of this disclosure include an information processing method, an information processing apparatus, and non-transitory computer-readable storage medium. A first key frame that includes initial posture data corresponding to an initial posture of a target virtual character is obtained. Target posture data is determined by inputting the initial posture data and a target task to a policy network trained by reinforcement learning. An output of the policy network indicates the target posture data corresponding to a target posture. At least one force to be acted on at least one first joint of the target virtual character is determined according to the initial and the target posture data. A posture of the target virtual character is adjusted from the initial posture to the target posture by applying the at least one force on the at least one first joint to obtain a second key frame.

Abstract: A method for human body model reconstruction and a reconstruction system are disclosed. The method includes acquiring a target image, and acquiring a segmented image by segmenting the target image based on an object to be reconstructed in the target image, the target image being one front image of the object; acquiring an initial estimate shape and a part of texture information of the object respectively, according to the segmented image; determining an initial 3D model of the object through the initial estimate shape, the initial 3D model being a 3D model without texture; acquiring complete texture information of the object according to the part of texture information and a texture generation model; and generating a 3D reconstruction model of the object based on the initial 3D model and the complete texture information, the 3D reconstruction model being a 3D model with texture.

Abstract: A spectator system may provide for spectating in online gaming. The spectator system may receive, at a spectator server, game state data from a game simulation server hosting an online game for one or more players, generate one or more spectator game state data corresponding to one or more spectator devices and output the one or more spectator game state data to the spectator devices. The spectator server may further output the game state data to another spectator server.

Abstract: Methods and systems of simulating a fluid. An outflow of the fluid is determined. An inflow of the fluid is determined. Determining a simulated fluid using the inflow and the outflow.

Abstract: A computer-implemented method is provided. The computer-implemented method includes rendering a dynamic effect to one or more objects in an image. The method includes setting boundary points surrounding a foreground object to define a boundary area in which a dynamic movement is to be realized; setting a movement line to define a dynamic movement direction along which the dynamic movement is to be realized, wherein setting the movement line includes detecting a continuous touch over different positions on the touch control display panel; setting a stationary area to define an area in which the dynamic movement is prohibited, wherein setting the stationary area includes detecting a first touch area corresponding to the stationary area on the touch control display panel; and generating an animation of the foreground object in the boundary area moving along the dynamic movement direction, thereby realizing the dynamic effect in the image.

Abstract: There is provided a data processing device including: a data acquisition unit configured to acquire animation data in which clothing moves according to a motion of a wearer's body wearing the clothing; and a data update unit configured to update the animation data based on three types of elements having ratios in accordance with a specified type of an emotion. There is provided a data processing method that is executed by a computer, the data processing method including: acquiring animation data in which clothing moves according to a motion of a wearer's body wearing the clothing; and updating the animation data based on three types of elements having ratios in accordance with a specified characteristic of a motion.

Abstract: Apparatuses, systems, and techniques to render images. In at least one embodiment, at least one visibility parameter determined for a first image region is reused for a different second image region that neighbors the first image region (e.g., spatially and/or temporally).

Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions, and determining second costs associated with the warping mesh that include costs of edge-related transformations for preserving straightness of edges of the image. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.

Abstract: The present teaching relates to a method and system for path planning. Information of a current pose of a robotic arm having a plurality of operable segments is obtained. The information includes a plurality of values, each of which corresponds to an angle formed between consecutive operable segments of the robotic arm. A desired pose where the robotic arm needs to reach is also obtained. An angle step-value is computed for the current pose of the robotic arm based on a function of a distance between the current pose and the desired pose, wherein the angle step value is to be used to determine a plurality of candidate next poses of the plurality of operable segments. One or more of candidate next poses is selected based on at least one criterion, and a trajectory is determined from the current pose to the desired pose based on the selected next poses.

Abstract: A computer is configured to receive an input operation from a distribution user and connect to game circuitry that manages progress of a game, and to video distribution circuitry. The computer sends, to the video distribution circuitry, display control data for displaying, on a viewing user device, a video of the game. The computer receives, from the game circuitry, gift data that is based on a gift output request sent from the viewing user device. The computer displays a gift that is based on the gift data. An image corresponding to consecutive behaviors of the game is displayed on the screen of the game, when a specified command included in game operation data sent from a user device used by any of the users participating in the game and the gift output request sent from the viewing user device cause the consecutive behaviors to be established.

Abstract: Systems and methods for image retargeting are provided. Image data may be acquired that includes motion capture data indicative of motion of a plurality of markers disposed on a surface of a first subject. Each of the markers may be associated with a respective location on the first subject. A plurality of blendshapes may be calculated for the motion capture data based on a configuration of the markers. An error function may be identified for the plurality of blendshapes, and it may be determined that the plurality of blendshapes can be used to retarget a second subject based on the error function. The plurality of blendshapes may then be applied to a second subject to generate a new animation.

Abstract: In a method for operating a computer having a user interface, e.g., a graphical and/or interactive user interface, a robot, which has members, e.g., arms, rotatable relative to each other and a tool and/or a load, are displayed graphically. One of the members is selectable from an indicated set of members, and a value of an inertial characteristic, e.g., a value of the mass, of this member is able to be inputted. The value of the mass of the member, the position of the center of mass of the member, and both the magnitude and the direction of each of the principal axes of inertia of the selected member are displayed graphically.

Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for providing editable keyframe-based animation data for applying to a character to animate motion of the character in three-dimensional space. Three-dimensional motion data is constructed from two-dimensional videos. The three-dimensional motion data represents movement of people in the two-dimensional videos and includes, for each person, a root of a three-dimensional skeleton of the person. The three-dimensional skeleton comprises multiple three-dimensional poses of the person during at least a portion of frames of a video from the two-dimensional videos. The three-dimensional motion data is converted into editable keyframe-based animation data in three-dimensional space and provided to animate motion.

Abstract: A user interface may present multiple presentation slides for user selection. Each slide may include one or more shapes, and a shape thread may be associated with each shape on the slide. For a selected slide, the user interface may present one or more shape threads associated with the selected slide. Each shape displayed on the selected slide may be associated with a particular shape thread, and a user may independently adjust one or more slide transition attributes of a shape via the associated shape thread.

Abstract: A method for analyzing an object. The method includes capturing, using a camera device, a sequence of images of a scene comprising a light source attached to a first element of a plurality of elements comprised in an object, detecting, by a hardware processor based on a pattern of local light change across the sequence of images, the light source in the scene, determining, by the hardware processor, a location of the light source in at least one image of the sequence of images, generating, by the hardware processor based on the location of the light source and a dynamic model of the object, a region-of-interest for analyzing the object, and generating an analysis result of the object based on the region-of-interest, wherein a pre-determined task is performed based on the analysis result.

Abstract: A method of producing an avatar video, the method comprising the steps of: providing a reference image of a person's face; providing a plurality of characteristic features representative of a facial model X0 of the person's face, the characteristic features defining a facial pose dependent on the person speaking; providing a target phrase to be rendered over a predetermined time period during the avatar video and providing a plurality of time intervals t within the predetermined time period; generating, for each of said times intervals t, speech features from the target phrase, to provide a sequence of speech features; and generating, using the plurality of characteristic features and sequence of speech features, a sequence of facial models Xt for each of said time intervals t.

Abstract: A computer graphics animation system is provided to assist prevent the generation of undesirable shapes, by providing realistic examples of a subject which are incorporated into an interpolation function which can be used to animate a new shape deformation of the subject.

Abstract: Technique includes acquiring first contact data of first time, associated with first particle in first region; calculating first position data on particles in the first region at second time, and receiving second position data on particles in second region at the second time; detecting second particle being in contact with the first particle and in the first region at the first time and being in the first region at the second time; copying, when the first and second particles are in contact at the second time, displacement of the second particle from the first contact data to second contact data of the second time; detecting third particle being in the first or second region at the second time and in contact with the first particle; and copying, when the third particle is listed in the first contact data, displacement of the third particle to the second contact data therefrom.

Abstract: At an electronic device with a touch-sensitive display, display a message transcript, where the message transcript includes at least one message from at least a first user. Determine, based at least in-part on the at least one message, a plurality of suggested one or more characters. Display, on the touch-sensitive display, the plurality of suggested one or more characters. Detect an input on the touch-sensitive display. Determine whether the input represents user selection of one of the plurality of suggested one or more characters. Display the selected one of the plurality of suggested one or more characters in the message transcript. Send the selected one or more characters to the first user.

Abstract: The present technology provides solutions for crowd-sourcing stream productions for a virtual esports environment. A method can include generating a virtual environment associated with an interactive session that includes a plurality of spectator devices, wherein each of the spectator devices is presented with a different display based on a corresponding vantage point located within the virtual environment; receiving a plurality of media captures from the spectator devices, wherein each of the media captures is captured from the corresponding vantage point of the spectator device within the virtual environment; selecting one of the media captures based on a comparison of visibility of an asset in the virtual environment; and streaming the selected media capture to a primary display on a requesting device.

Abstract: Systems and methods for synchronous content viewing are provided. In embodiments, a method includes providing an interface to a user of a remote user device; receiving a user selection of one or more contacts of the user as invitees to a synchronized session; providing content synchronously to the remote user device of the user and other remote user devices of the respective one or more contacts of the user based on receiving an acceptance from the one or more contacts of the user, wherein the user and the one or more contacts of the user view the content synchronously during the synchronized session; and providing functions of an application to the participants synchronously, wherein the functions enable the participants to interact with the content in real-time during the synchronized session, and wherein all participants view interactions with the content simultaneously.

Abstract: A system configured for full-body tracking with magnetic fields in virtual reality (“VR”) and augmented reality (“AR”) applications includes at least one tracker, at least one wearable article, and a computational device. Each of the at least one trackers hosts a joint sensor suite. The joint sensor suite is configured to track positions, orientations, and joint angles of a joint along a body. Each of the at least one trackers is configured to be attached to the body. Each of the at least one wearable articles is configured to enable one of the at least one trackers to be fastened to the joint along the body. The computational device is configured to capture real-time user generated movements via each of the at least one trackers and digitize user poses and body positions.

Abstract: Technologies for determining the spatial orientation of input imagery to produce a three-dimensional model includes a device having circuitry to obtain two-dimensional images of an anatomical object (e.g., a bone of a human joint), to determine candidate values indicative of translation and rotation of the anatomical object in the two-dimensional images, and to produce, as a function of the obtained two-dimensional images and the candidate values, a candidate three-dimensional model of the anatomical object. The circuitry is also to determine a score indicative of an accuracy of the candidate three-dimensional model, to determine whether the score satisfies a threshold, and to produce, in response to a determination that the score satisfies the threshold, data indicating that the candidate three-dimensional model is an accurate representation of the anatomical object.

Abstract: Avatar animation systems disclosed herein provide high quality, real-time avatar animation that is based on the varying countenance of a human face. In some example embodiments, the real-time provision of high quality avatar animation is enabled, at least in part, by a multi-frame regressor that is configured to map information descriptive of facial expressions depicted in two or more images to information descriptive of a single avatar blend shape. The two or more images may be temporally sequential images. This multi-frame regressor implements a machine learning component that generates the high quality avatar animation from information descriptive of a subject's face and/or information descriptive of avatar animation frames previously generated by the multi-frame regressor.

Abstract: Embodiments described herein include an expressive icon system to present an animated graphical icon, wherein the animated graphical icon is generated by capture facial tracking data at a client device. In some embodiments, the system may track and capture facial tracking data of a user via a camera associated with a client device (e.g., a front facing camera, or a paired camera), and process the facial tracking data to animate a graphical icon.

Abstract: This application discloses a method for perspective rotation in a virtual environment performed at a terminal. The method includes: displaying a first perspective image of an application program associated with the virtual environment, a first function control and a second function control being overlaid on the first perspective image; receiving a first perspective rotation operation triggered based on the first function control; enabling a first function and a perspective rotation function of the first function control, and switching the first perspective image to a second perspective image; receiving a second perspective rotation operation triggered based on the second function control while the first function control is in an enabled state; and disabling the perspective rotation function of the first function control, enabling a second function and a perspective rotation function of the second function control, and switching the second perspective image to a third perspective image.

Abstract: Provided is a method of granting access, including compiling a training profile of each of two or more persons, wherein the training profile includes one or more oculomotor features exhibited during one or more tasks, associating a training profile of one of the two or more persons with a permission, detecting a testing profile of one of the two or more persons, wherein the testing profile includes one or more of the one or more oculomotor features exhibited by during one or more of the one or more tasks, identifying a target profile, granting access if and only if the target profile is associated with the permission, wherein the one or more tasks are selected from fixation, saccade speed, saccade accuracy, visually-guided saccades, anti-saccades, memory-guided saccades, free viewing, pursuit.

Abstract: A simulation environment (e.g., multi-player game) hosted by a provider network may implement componentized entities to reduce the amount of resource usage for a simulation (e.g., by reducing the amount of input/state data transmitted through the use of dynamically changing input structures). A user may add or remove any number of components to an entity that is simulated at the local client device. When inputs are received for one or more components, values for predictive states are locally determined for each component. An input packet is generated and sent to the provider network, which includes the inputs as well as data that is based on the values for the locally predicted states (e.g., a fingerprint or other unique ID). If necessary, a correction packet may be generated at the provider network and sent back to the client.

Abstract: A method and system to create a versatile graft surface and at least one corresponding graft groom having the versatile graft surface as a base, to fit and transform the versatile graft surface onto a target surface and process the created graft groom into a modifiable hair model after the versatile graft surface is transformed onto the target surface.

Abstract: Disclosed are a wavelet-based progressive fast recompression method and system for large-deformed mesh data. Given mesh data is compressed. The compressed mesh data is recompressed by propagating part of a deformed mesh to all meshes based on information extracted from the compressed data when the mesh data is deformed.

Abstract: Techniques are disclosed for providing an avatar personalized for a specific person based on known data from a relatively large population of individuals and a relatively small data sample of the specific person. Auto-encoder neural networks are used in a novel manner to capture latent-variable representations of facial models. Once such models are developed, a very limited data sample of a specific person may be used in combination with convolutional-neural-networks or statistical filters, and driven by audio/visual input during real-time operations, to generate a realistic avatar of the specific individual's face. In some embodiments, conditional variables may be encoded (e.g. gender, age, body-mass-index, ethnicity, emotional state). In other embodiments, different portions of a face may be modeled separately and combined at run-time (e.g., face, tongue and lips). Models in accordance with this disclosure may be used to generate resolution independent output.

Abstract: A method and system for detecting performance regression in a software application is disclosed. The software application comprises a feature flag having at least two states, wherein in a first state a first version of a feature is provided and in a second state a second version of the feature is provided when the software application is executed. The feature flag is further wrapped in a timer for recording the time taken for the corresponding first and second versions of the feature to be provided.

Abstract: In an aspect, an apparatus for generating a virtual avatar is presented. An apparatus includes at least a processor and a memory communicatively connected to the at least a processor. At least a processor is configured to generate a virtual avatar model. A virtual avatar model includes a virtual entity and an operational model of the virtual entity. At least a processor is configured to receive user input. User input includes an avatar modifier. At least a processor is configured to modify at least a portion of a virtual avatar model as a function of an avatar modifier. At least a processor is configured to display a virtual avatar model to a user through a display device.

Abstract: Gaze is corrected by adjusting multi-view images of a head. Image patches containing the left and right eyes of the head are identified and a feature vector is derived from plural local image descriptors of the image patch in at least one image of the multi-view images. A displacement vector field representing a transformation of an image patch is derived, using the derived feature vector to look up reference data comprising reference displacement vector fields associated with possible values of the feature vector produced by machine learning. The multi-view images are adjusted by transforming the image patches containing the left and right eyes of the head in accordance with the derived displacement vector field.

Abstract: In the multi-target detection and tracking method, lidar (2D laser scanner) scans point cloud data of surroundings and transfers the collected data to the edge server. Then, the edge server uploads the data to the cloud. After obtaining the lidar data, point clouds of footsteps are extracted through dynamic point extraction, point clustering, and random forest model, respectively. Footsteps are matched to form human tracking trajectory by using trajectory matching. After the tracking process, the walking information is published to the users, in a visual form. Meanwhile, the gait parameters are saved into files, including walking speed and step length, when human is detected. Comparing to the visual sensor based human tracking methods, the present invention employs lidar to avoid the interference of ambient light, which leads to easier implementation and larger universality, especially for multi-target scenarios.

Abstract: A method for managing a skin care system. The skin care system includes circuitry to receive skin information from a user. The circuitry determines treatable skin conditions based on the skin information. The skin care system generates severity information and cosmetic formulation compatibility information based on the treatable skin conditions. The skin care system is further able to virtually display user-specific compatible cosmetic formulations based on at least one parameter associated with the severity information and cosmetic formulation compatibility.

Abstract: A UI state crawler system may allow for the crawling of a video game UI that may identify and map UI states of the video game. The UI state crawler system may determine a user interface (UI) state identifier (ID) for a UI state of a UI of a video game based at least in part on a plurality of node IDs corresponding to a plurality of nodes of a hierarchical structure of the UI and determine a UI state map does not include a UI state map node corresponding to the UI state ID. In response to the determining the UI state map does not include the UI state map node corresponding to the UI state ID, the UI state crawler system may generate an updated UI state map including the UI state map node corresponding to the UI state ID.

Abstract: The present disclosure generally relates to systems and methods for providing individual customizations of a stream for an eSports event of eSports events. The customizations including incorporating customized graphic and audio displays into a viewed stream. Additionally, viewers may be able to access a limited social network for spectators having similar interests or viewing rights. In other aspects, the eSports presentation may be linked to third-party services for enhancing spectator engagement.

Abstract: Devices, systems and process for facilitating a Deepfake content watch party (DFWP) are disclosed. A system includes server, Deepfake presenter user device (DFPUD), Deepfake user device (DFUD), and content provider. The DFPUD instructs a content capture device (CCD) to capture a Deepfake Reaction (DFR) by a DFPUD user; receives the DFR from the CCD; and communicates the DFR to the server. The content provider provides Deepfake Content (DFC) segments, which include a Deepfake Target (DFT), and Deepfake Augmented Content (DFAC), which identifies a Deepfake model for the DFT. The server, based on the DFR, generates a Deepfake Subject (DFS), identifying a DFPUD user characteristic, generates Deepfake secondary content (DFSC), and communicates DFSC to the DFUD. The DFUD integrates the DFSC with the DFC to facilitate presentation of DFRs in a DFWP.