Abstract: This application provides a audio-driven three-dimensional facial animation model generation method and apparatus, and an electronic device.

Abstract: Methods, systems, and non-transitory computer readable media are disclosed for intelligently enhancing details in edited images. The disclosed system iteratively updates residual detail latent code for segments in edited images where detail has been lost through the editing process. More particularly, the disclosed system enhances an edited segment in an edited image based on details in a detailed segment of an image. Additionally, the disclosed system may utilize a detail neural network encoder to project the detailed segment and a corresponding segment of the edited image into a residual detail latent code. In some embodiments, the disclosed system generates a refined edited image based on the residual detail latent code and a latent vector of the edited image.

Abstract: An augmented reality viewer includes components, assemblies, and executable logic to provide a user with the perception of rich augmented reality experiences, including aspects of the aquatic world.

Abstract: A method and apparatus for tracking hand joints are disclosed, where the method of tracking hand joints includes estimating angle information of finger joints based on initial positions of hand joints obtained from an image, generating a kinematic model of the hand joints based on the initial positions of the hand joints and the angle information of the finger joints, and tracking the hand joints by updating at least one of positions or angles of the hand joints based on the initial positions of the hand joints and the kinematic model.

Abstract: A method of the present disclosure includes (a) a step of receiving track information for specifying a track of a target operation of a robot, (b) a step of acquiring, according to an instruction to adjust a parameter set for controlling the target operation, about one or more initial parameter sets, values of evaluation indicators of control results obtained when causing the robot to execute the target operation using the respective initial parameter sets, (c) a step of displaying, on a display section, one or more reference displays based on the acquired values of the evaluation indicators, and (d) a step of receiving an input of condition information for deciding a condition of optimization processing for the parameter set, the condition information being condition information about the evaluation indicators, performing the optimization processing for the parameter set according to the condition information, and determining a value of a new parameter set.

Abstract: Systems and methods for avatar eye animation is provided. The method include obtaining a first gaze vector of a user associated with a first eye of the user and obtaining a second gaze vector of the user associated with a second eye of the user. The method further includes determining a first point in a three-dimensional (3D) virtual space that the user is looking toward. The method also includes creating a second point in the 3D virtual space corresponding to the first point in the 3D virtual space. The method then includes rendering a first eye and a second eye of an avatar of the user in the 3D virtual space based on the second point in the 3D virtual space such that the first eye and the second eye of the avatar are looking toward the second point in the 3D virtual space.

Abstract: The present invention concern systems and methods for maintaining diversity in AI and ML environments through the cooperation of various AI and ML systems such that they optimize for social and cultural diversity. The examination of behavior, infrastructure, and governance, mimicking of genetic biodiversity, and application of the foregoing to machine reasoning mitigates the tendency of systems to find optimized or single best solutions. AI and ML environments may thus derive multiple diverse solutions that contribute to richer ecosystems in which human beings may function and thrive.

Abstract: The present disclosure generally relates to creating and editing user avatars. In some examples, guidance is provided to a user while capturing image data for use in generating a user-specific avatar. In some examples, a user interface allows a user to intuitively customize a user avatar. In some examples, avatars are generated for a messaging session based on an avatar model for a user of the messaging application. In some examples, an avatar editing interface updates a user avatar in response to gestures and based on the type of gesture and the avatar feature that is selected for editing.

Abstract: A program production apparatus or the like that is suitable for reflecting information with respect to persons who distribute or broadcast programs as well as viewers. A program production apparatus 1 generates program data using a character. A performance determination unit 5 determines a performance using the character. A program production unit 7 generates the program data using the performance determined by the performance determination unit 5. The performance determination unit 5 adjusts movement of the character and/or adjusts at least one from among lines spoken by the character, BGM, and sound effects, according to the file format of the program data generated by the program production unit 7, so as to determine the performance.

Abstract: Systems and methods are described for rendering garments. The system includes a first machine learning model trained to generate coarse garment templates of a garment and a second machine learning model trained to render garment images. The first machine learning model generates a coarse garment template based on position data. The system produces a neural texture for the garment, the neural texture comprising a multi-dimensional feature map characterizing detail of the garment. The system provides the coarse garment template and the neural texture to the second machine learning model trained to render garment images. The second machine learning model generates a rendered garment image of the garment based on the coarse garment template of the garment and the neural texture.

Abstract: Various methods, apparatuses/systems, and media for generating a data model are disclosed. A processor receives data from a plurality of data sources; displays, onto a graphical user interface (GUI), a plurality of selectable icons for receiving user input in selecting a set of attributes data related to generating a desired data model; receives user input of the selected set of attributes data; automatically creates an executable custom code based on the received data from the plurality of data sources and the selected set of attributes data; executes the custom code; calls, in response to executing, a backend platform for processing the received data from the plurality of data sources and the selected set of attributes data; and automatically generates, in response to calling, the desired data model based on the processed received data and the selected set of attributes data.

Abstract: A system can determine, from a video of an online session, respective bounding boxes of text names of people, wherein the text names are presented in the video, and wherein images of the people are present in the video. The system can determine, from the video, respective faces of the people. The system can associate a first bounding box of the bounding boxes with a first face of the faces based on the first bounding box satisfying a function of distance with respect to the first face among the faces. The system can extract a name from the first bounding box via optical character recognition. The system can extract a subportion of the video that comprises the first face. The system can store an association between the name and the subportion of the video that comprises the first face.

Abstract: Methods and systems for cooperative or coached gameplay in virtual environments are disclosed. Memory may store a content control profile regarding a set of control input associated with an action in a virtual environment of a digital content title. A request may be received from a set of one or more users associated with different source devices regarding cooperative gameplay of the digital content title. At least one virtual avatar may be generated for an interactive session of the digital content title in response to the request. A plurality of control inputs may be received from the plurality of different source devices and combined into a combination set of control inputs. Generating the combination set of control input may be based on the content control profile. Virtual actions associated with the virtual avatar may be controlled within the virtual environment in accordance with the combination set of control inputs.

Abstract: A providing apparatus configured to provide three-dimensional geometric data to be used to generate a virtual viewpoint image receives a data request from a communication apparatus, decides which of a plurality of pieces of three-dimensional geometric data including first three-dimensional geometric data and second three-dimensional geometric data with a different quality than the first three-dimensional geometric data is to be provided to the communication apparatus from which the received data request was transmitted, and provides the three-dimensional geometric data decided on from among the plurality of pieces of three-dimensional geometric data, to the communication apparatus as a response to the received data request.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

Abstract: A boundary surface setting section sets the boundary surface for delimiting a permissible region that is the region which is located inside the real space and in which the user wearing a head-mounted display is permitted to exist and an impermissible region that is the region which is located inside the real space and in which the user is not permitted to exist. A display control section causes a portion or a whole of a virtual object representing the boundary surface to be displayed on the head-mounted display. The boundary surface includes a first boundary line having a first height and a second boundary line having a second height, and a portion or all of the first boundary line is different from the second boundary line.

Abstract: The specification relates to the generation of in-game animation data and the evaluation of in-game animations.

Abstract: A user may create an avatar and/or animated sequence illustrating a particular object or living being performing a certain activity, using images of portions of the object or living being extracted from a still image or set of still images of the object or living being.

Abstract: Systems, methods, and non-transitory computer-readable media can identify a virtual character being presented to a user within a real-time immersive environment. A first animation to be applied to the virtual character is determined. A nonverbal communication animation to be applied to the virtual character simultaneously with the first animation is determined. The virtual character is animated in real-time based on the first animation and the nonverbal communication animation.

Abstract: Implementations relate to a computer-implemented method to display a rigid body object within a virtual environment. Some implementations include receiving a model representation of the rigid body object, wherein the model representation includes a plurality of constraints for the rigid body object; splitting the rigid body object into two or more splinters based on the model representation, obtaining a modified representation of the rigid body object based on a modified rigid body object formed by connecting each of the two or more splinters to one or more other splinters by one or more rigid joints, determining an input state of the rigid body object in the virtual environment, solving a set of equations based on the input state and the modified representation of the rigid body object to determine an updated state of the rigid body object.

Abstract: Systems, methods, and non-transitory computer-readable media can identify a virtual character being presented to a user within a real-time immersive environment. A first animation to be applied to the virtual character is determined. A nonverbal communication animation to be applied to the virtual character simultaneously with the first animation is determined. The virtual character is animated in real-time based on the first animation and the nonverbal communication animation.

Abstract: A patient digital twin platform includes a clinical subsystem, a patient subsystem, and a performance capture system. The clinical subsystem generates physical therapy exercises for a patient. The patient subsystem directs the patient to perform the exercises generated by the clinical patient subsystem. The performance capture system captures measurement data, such as video, of the patient performing at least one physical therapy exercise prescribed by the clinical subsystem. The output of the performance capture system is input to a patient digital twin engine that updates a patient digital twin of the patient.

Abstract: The present technology pertains to shoulder positioning with body tracking in a multiuser virtual reality (VR) application. In one aspect, a client device that is participating in the multiuser VR application and may be configured to: determine a shoulder position based on a height of a user, wherein the shoulder position is mapped to an avatar in a virtual world; receive sensor data from an arm motion sensor attached to the user, wherein the sensor data identifies motion of an arm of the user; determine a movement vector of the shoulder position of the user using the sensor data; determine a shoulder avatar movement vector for a shoulder of the avatar from the movement vector; and render the avatar with the shoulder positioned based on the shoulder avatar movement vector.

Abstract: A digital content package includes first content comprising a video feature such as a motion picture or the like, and a user-selectable application configured to operate as follows. When activated using an icon off of a menu screen, the application records an identifier for scenes (discrete portions) of the first content that are selected by a user to generate a playlist. The user may select the scenes by indicating a start and end of each scene. The application saves the playlist locally, then uploads to a server. Via a user account at the server, a user may publish the playlist to a user-created distribution list, webpage, or other electronic publication, and modify the playlist by deleting or reordering scenes.

Abstract: The invention proposes a method of training a convolutional neural network in which, at each convolutional layer, weights for one seed convolutional filter per layer are updated during each training iteration. All other convolutional filters are polynomial transformations of the seed filter, or, alternatively, all response maps are polynomial transformations of the response map generated by the seed filter.

Abstract: A method for avatar virtual arm movement, the method may include (i) capturing, by a camera of a participant of a video conference, a hand gesture made by the participant without capturing an arm movement of the participant, the arm movement is associated with the hand gesture; (ii) determining the arm movement of the participant, based on the hand gesture; (iii) determining a virtual movement of an arm of an avatar of the participant within a virtual video conference environment, the virtual movement of the arm of the avatar of the participant mimics the arm movement of the participant; and (iv) assisting to perform the virtual arm movement of the avatar within the virtual video conference environment.

Abstract: Systems and methods for training machine learning models are disclosed. An example method includes receiving a semi-labeled set of training samples including a first set of training samples, where each training sample in the first set is assigned a known label, and a second set of training samples, where each training sample in the second set has an unknown label, determining a first loss component, the first loss component providing a loss associated with the first set, determining a second loss component, the second loss component having a value which increases based on a difference between a distribution of individually predicted values of at least the second set and an expected overall distribution of at least the second set, and training the machine learning model, based on the first loss component and the second loss component, to predict labels for unlabeled input data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating images. In one aspect, a method includes: receiving an input text prompt including a sequence of text tokens in a natural language; processing the input text prompt using a text encoder neural network to generate a set of contextual embeddings of the input text prompt; and processing the contextual embeddings through a sequence of generative neural networks to generate a final output image that depicts a scene that is described by the input text prompt.

Abstract: A face-processing system is described for producing a target image based on a source image and driving information. The source image includes data depicting at least a face of a source subject having a source identity, a source pose, and a source expression. The driving information specifies one or more driving characteristics. The target image combines characteristics of the source image and the driving information. According to illustrative implementations, the face-processing system produces the target image by using plural warping subcomponents that operate at plural respective levels of a neural network and at increasing respective resolutions. Each warping subcomponent operates, in part, based on geometric displacement field (GDF) information that describes differences between a source mesh derived from the source image and a driving mesh derived from the driving information.

Abstract: Systems and methods are provided for performing operations including: retrieving a plurality of content items; determining a first category of a first content item in the plurality of content items; selecting a first presentation arrangement from a plurality of presentation arrangements based on the first category determined for the first content item; and presenting the first content item to a user according to the selected first presentation arrangement.

Abstract: A system method of managing a software repository. The method including receiving a dataset comprising a set of computer instructions, retrieving feature-related data from the dataset, determining, by a machine learning model executing on the computing device, a function of the computer instructions based on the feature-related data, and generating a synopsis of the function of the computer instructions. The method can include transmitting the synopsis to a user interface and receiving feedback from a user indicative of whether the synopsis accurately describes the function of the computer instructions. If the feedback indicates that the synopsis accurately describes the function of the computer instructions, the method can include storing the synopsis in a memory of the computing device. If the feedback indicates that the synopsis inaccurately describes the function of the computer instructions, the method can include generating a revised synopsis of the function of the computer instructions.

Abstract: A method for determining a plurality of layers of bounding boxes of an object includes determining a polyhedron capable of accommodating the object therein as a first-layer bounding box. The method also includes selecting one vertex from a plurality of vertices of the first-layer bounding box as a target vertex, and determining, by processing circuitry of a computing device, a support plane of the object. The support plane has a normal vector that has a specific direction corresponding to the target vertex and that is closest to the target vertex, the support plane being a plane passing through a point on a surface of the object such that the object is completely located on one side of the support plane. The method further includes cutting the first-layer bounding box based on at least the support plane to form a smaller bounding box, as a second-layer bounding box.

Abstract: A computing device having a touch-sensitive surface and a display, detects a stylus input on the touch-sensitive surface while displaying a user interface. A first operation is performed in the user interface in accordance with a determination that the stylus input includes movement of the stylus across the touch-sensitive surface while the stylus is detected on the touch-sensitive surface. A second operation different from the first operation is performed in the user interface in accordance with a determination that the stylus input includes rotation of the stylus around an axis of the stylus while the stylus is detected on the touch-sensitive surface. A third operation is performed in the user interface in accordance with a determination that the stylus input includes movement of the stylus across the touch-sensitive surface and rotation of the stylus around an axis of the stylus while the stylus is detected on the touch-sensitive surface.

Abstract: A method and apparatus for detecting an object in a three-dimensional (3D) point cloud are disclosed. The apparatus includes a three-dimensional (3D) model reconstruction unit configured to reconstruct a three-dimensional (3D) model using three-dimensional (3D) point cloud data, a two-dimensional (2D) object detection unit configured to detect an object in at least one two-dimensional (2D) image captured to generate the three-dimensional (3D) point cloud data, a three-dimensional (3D) position estimation unit configured to form a cylinder surrounding the object in the three-dimensional (3D) model based on a two-dimensional (2D) position of the detected object so as to estimate a three-dimensional (3D) position of the detected object using the cylinder, and an object region extraction unit configured to acquire three-dimensional (3D) point cloud data contained in the cylinder so as to extract an object region from the acquired three-dimensional (3D) point cloud data.

Abstract: A server device is configured to provide a combined environment and includes processor circuitry configured to determine first parameters indicative of a first location, generate first environment data indicative of the first location, determine second parameters indicative of a second location, and associate the second parameters with the first environment data for providing combined environment data, and output the combined environment data.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure encode features of a source image to obtain a source appearance encoding that represents inherent attributes of a face in the source image; encode features of a target image to obtain a target non-appearance encoding that represents contextual attributes of the target image; combine the source appearance encoding and the target non-appearance encoding to obtain combined image features; and generate a modified target image based on the combined image features, wherein the modified target image includes the inherent attributes of the face in the source image together with the contextual attributes of the target image.

Abstract: The present invention relates to the field of video image processing, and provides a filtering method and an apparatus, to resolve a problem that subjective quality and objective quality of an image deteriorate because filtering processing cannot be performed on internal blocks of a non-translational motion prediction unit.

Abstract: A meme generation method, an electronic device, and a storage medium are provided. The method includes: determining a plurality of second expression images corresponding to a target face image based on a plurality of first expression images contained in a first meme; generating a second meme corresponding to the target face image based on the plurality of second expression images corresponding to the target face image; wherein, determining an affine transformation parameter between the target face image and an i-th first expression image in the plurality of first expression images according to a corresponding relation between a face key point in the target face image and a face key point in the i-th first expression image; and transforming the target face image based on the affine transformation parameter to obtain an i-th second expression image corresponding to the target face image.

Abstract: An information processing apparatus includes a specification unit, a determination unit, and a generation unit. The specification unit specifies a relationship between a plurality of objects detected in a video. The determination unit determines arrangement of motion trajectories of the plurality of objects, the relationship between which has been specified by the specification unit, while maintaining a relative appearance order in the video and avoiding an overlap of the motion trajectories in a temporal direction. The generation unit generates a summary video of the video based on the arrangement determined by the determination unit.

Abstract: The present disclosure provides a method for generating an animation figure, a device and a storage medium. The method includes: acquiring an image including at least one target object; acquiring position information about key points of the target object in the image; determining target angle information about lines connecting the key points in accordance with the position information about the key points; and adjusting a predetermined animation figure in accordance with the target angle information. A pose of a target animation figure acquired through adjustment is identical to a pose of the target object in the image, and the target animation figure corresponds to the target object.

Abstract: The disclosed embodiments relate to a mechanism for allowing concurrent access, e.g. write operations, to a data structure, e.g. a file, which may have limited capacity, by multiple processes or threads and, in particular, for allowing concurrent, or otherwise substantially simultaneously, appending of data to the file thereby. Each process/thread requests the amount of space it needs to append its data. Upon request, the disclosed mechanism allocates and reserves the requested amount of space and allows the requesting process/thread to commence its append operation. Subsequent to the commencement of the append operation, another thread is then free to request space for its needs and commence its append operation concurrently, or otherwise substantially simultaneously, with the previous append operation, i.e. before the prior append operation is completed.

Abstract: A user may create an avatar and/or animated sequence illustrating a particular object or living being performing a certain activity, using images of portions of the object or living being extracted from a still image or set of still images of the object or living being.

Abstract: Apparatus and methods are provided for mapping and displaying three dimensional surgical pathways within imaging of cranial structures derived from voxels of a cranial scan. Entry voxel and target voxel are selected as pathway endpoints. A series of respective neighbor voxels are mapped between the entry voxel and the target voxel to define a pathway. For each voxel Vxi,yi,zi in the series, an immediately succeeding voxel of voxel Vxi,yi,zi is selected from among the group of neighbor voxel of voxel Vxi,yi,zi which excludes neighbor voxels of the immediately preceding voxel of voxel Vxi,yi,zi. The selection is made by comparing selection weights determined based on relative distances with respect to the endpoint voxels and relative distance from voxels within a predetermined distance that represent at least a threshold density. The voxels of the pathway are then selectively highlighted in a displayed view to provide a visualization of the 3D surgical pathway.

Abstract: The present disclosure provides a method and apparatus of displaying an image, an electronic device and a storage medium. The method includes: obtaining a preset number of candidate sub-images by segmenting a to-be-displayed first candidate image; assigning each of the candidate sub-images to a corresponding first image sub-template in a preset first image template sequentially, where a number of the first image sub-templates is equal to a number of the candidate sub-images; adjusting each of the first image sub-templates in the first image template such that the first candidate image displays a preset animation effect, where the preset animation effect is obtained from configuration information of an electronic device.

Abstract: A client device can display various augmented reality and virtual reality images to a user. For example, the client device captures images of an external environment and identifies objects in the environment. If an object satisfies a virtual environment criterion, the client device displays a window to a virtual environment at the location of the object in the environment. In another example, the client device includes a light assembly that can generate light. If the user turns the light assembly on, virtual objects displayed in the environment will gather to a region highlighted by the light from the light assembly. In another example, the client device estimates depth information of an object in the environment. The client device then displays a virtual object in the environment based on the depth information and may also display the virtual object interacting with an object in the external environment.

Abstract: Embodiments of the present disclosure disclose image processing methods and apparatuses, image devices, and storage media. The image processing method includes: obtaining an image; obtaining the feature of the limb of the body based on the image, where the limb includes the upper limb and/or the lower limb; determining first-type movement information of the limb based on the feature; and controlling the movement of the limb of a controlled model according to the first-type movement information.

Abstract: Provided is a system that receives a three-dimensional (3D) skinned model of a first object and a set of images of a second object. The second object is same as or structurally similar to the first object in appearance. The system executes a differentiable rendering operation to generate a first image which includes a contour of the 3D skinned model. The system further generates a reference image which includes a contour of the second object, based on the received set of images. The system updates first values of bone parameters associated with the 3D skinned model based on a difference between the generated first image and the generated reference image. The system updates the 3D skinned model based on the updated first values of the bone parameters.

Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.

Abstract: A pointing device includes an acceleration sensor, a motion sensor and a processor. The processor includes a first attitude angle calculator, a second attitude angle calculator and an attitude angle selector. When a predetermined condition is satisfied, the processor control the acceleration sensor to generate acceleration data, the first attitude angle calculator calculates a first attitude angle output according to the acceleration data, and the attitude angle selector receives the first attitude angle output as the reference data. When the predetermined condition is not satisfied, the second attitude angle calculator calculates a second attitude angle output according to angular velocity data generated by the motion sensor and the reference data, and the attitude angle selector receives the second attitude angle output as the reference data. The present disclosure also provides a control method of the pointing device.

Abstract: A method, apparatus and system for facilitating navigation toward a region of interest in an extended scene of video content include determining a timeline including information regarding at least one region of interest in the video content and displaying, in a portion of the video content currently being displayed, a visual indicator indicating a direction in which to move in the video content to cause the display of the at least one region of interest. In one embodiment of the present principles a timeline is attached to the content and carries information evolving over time about the region(s) of interest. A renderer processes the timeline and provides navigation information to a user using available means such as a graphical representation or haptic information, or a combination of several means.