Abstract: A first region within image data corresponding to a first portion of a user is determined, and a classifier associated with the first portion of the user is determined. A second region within the image data that includes the first region is determined. Within the second region, first pixels corresponding to the first portion of the user, second pixels corresponding to a second portion of the user adjoined to the first portion of the user, and third pixels corresponding to an environment of the user are identified. Based on identifying the second pixels, the classifier associated with the first portion of the user is confirmed. Upon the confirmation an item is rendered on the first portion of the user.

Abstract: Systems and methods are provided for generating a flat garment pattern and/or 3D mesh representation of a garment from one or more images depicting the garment laid flat or hung up. A system may obtain both a front image depicting a front view of a garment and a back image depicting a back view of the garment. A front and back silhouette of the garment may then be generated, which may include segmenting the garment depiction from background image content. A parametric representation of the garment may then be generated based on the front and back silhouettes, which may be implemented by iteratively optimizing, using differentiable rendering techniques, a garment representation within a parametric garment space previously learned for the particular garment type. A 3D mesh garment representation may then be generated based on the parametric representation, from which a flat sewing pattern may subsequently be generated if desired.

Abstract: Described are systems and method directed to generation of a dimensionally accurate three-dimensional (“3D”) model of a body, such as a human body, based on two-dimensional (“2D”) images of that body. A user may use a 2D camera, such as a digital camera typically included in many of today's portable devices (e.g., cell phones, tablets, laptops, etc.) and obtain a series of 2D body images of their body from different views with respect to the camera. The 2D body images may then be used to generate a plurality of predicted body parameters corresponding to the body represented in the 2D body images. Those predicted body parameters may then be further processed to generate a dimensionally accurate 3D model or avatar of the body of the user.

Abstract: The present disclosure describes a system and method for implementing a virtual shoe try-on feature. The method includes predicting, using a machine learning model, a first plurality of pixels including pixels of an image corresponding to a foot in the image and adding a first virtual cylindrical object to a three-dimensional model of a shoe such that a portion of the first virtual cylindrical object is positioned within the three-dimensional model of the shoe. The method also include determining a set of pixels including pixels of the first plurality of pixels that intersect with the first virtual cylindrical object and removing, from the three-dimensional model of the shoe, a first portion of the three-dimensional model of the shoe that intersects with the set of pixels.

Abstract: The present disclosure describes a system and method for implementing a virtual shoe try-on feature. The method includes receiving, from a database, a three-dimensional model of a shoe with a first annotated point on the three-dimensional model of the shoe, a second annotated point on the three-dimensional model of the shoe, and a third annotated point on the three-dimensional model of the shoe. The method also includes predicting, using a machine learning model and based on an image of a foot, one or more features of the foot, generating, based on the first annotated point, the second annotated point, the third annotated point, and the one or more features, a two-dimensional model of the shoe using the three-dimensional model of the shoe, and superimposing the two-dimensional model of the shoe onto the image of the foot.

Abstract: Systems and methods are provided for generating custom garment patterns for producing clothing sized to fit any newly input human body representation. A set of reference sample bodies may be selected for which custom designed garment patterns are then obtained. Once reference patterns are obtained, a custom garment pattern for a particular new body may be automatically created by blending two or more of the reference patterns. For example, the neighboring reference bodies to the new body may be identified within a low-dimensional embedding space, and interpolation of garment parameters for the previously designed garment patterns for these reference bodies may be performed to produce a custom garment pattern.

Abstract: A method includes extracting, using a backbone of a machine learning model, a plurality of features from an image of a foot and predicting, using a first portion of the machine learning model and based on one or more features of the plurality of features, a first aspect of the foot. The method also includes predicting, using a second portion of the machine learning model and based on one or more features of the plurality of features, a second aspect of the foot different from the first aspect, generating, using at least the first aspect and the second aspect, a two-dimensional model of a shoe, and superimposing the two-dimensional model of the shoe onto the image of the foot.

Abstract: According to an embodiment, a method includes estimating, based at least in part on a first location of a bounding box for a foot in a first frame of a sequence of frames and a second location of the bounding box in a second frame of the sequence of frames, a smoothing transformation that characterizes a movement of the foot in the sequence of frames and smoothing, based at least in part on the smoothing transformation, a movement of a structure of the foot from a first location of the structure in the first frame to a second location of the structure in the second frame. The method also includes superimposing a model of a shoe onto the foot in the sequence of frames based at least in part on the smoothed movement.

Abstract: Described are systems and methods directed to generation of a personalized three-dimensional (“3D”) body model of a body, such as a human body, based on two-dimensional (“2D”) images of that body and the generation and presentation of predicted personalized 3D body models of the body when one or more body measurements (e.g., body fat, body weight, muscle mass) are changed. For example, a user may provide a target body measurement value and the implementations will generate one or more predicted personalized 3D body models representative of a predicted appearance of the body with the target body measurement value.

Abstract: Systems and methods are provided for improved 3D garment draping simulation. A garment pattern may be obtained that includes a number of flat, 2D garment panels designated to be connected at seam lines. Triangulated versions of each of the 2D garment panels may then be positioned in 3D virtual space relative to a 3D model of a human body, such that one or more annotated points on each triangulated garment panel are aligned with a corresponding labelled point or region on the 3D body. A warped 3D garment mesh may then be generated by repeatedly applying geometric manipulations to the triangulated garment panels to connect their corresponding seam lines without causing collisions between the triangulated garment panels and the 3D body. This warped 3D garment may then be provided as input to a physics-based draping simulator.

Abstract: Techniques are disclosed for performing video synthesis of audiovisual content. In an example, a computing system may determine first facial parameters of a face of a particular person from a first frame in a video shot, whereby the video shot shows the particular person speaking a message. The system may determine second facial parameters based on an audio file that corresponds to the message being spoken in a different way from the video shot. The system may generate third facial parameters by merging the first and the second facial parameters. The system may identify a region of the face that is associated with a difference between the first and second facial parameters, render the region of the face based on a neural texture of the video shot, and then output a new frame showing the face of the particular person speaking the message in the different way.

Abstract: Techniques are disclosed for performing video synthesis of audiovisual content. In an example, a computing system may determine first parameters of a face and body of a source person from a first frame in a video shot. The system also determines second parameters of a face and body of a target person. The system determines that the target person is a replacement for the source person in the first frame. The system generates third parameters of the target person based on merging the first parameters with the second parameters. The system then performs deferred neural rendering of the target person based on a neural texture that corresponds to a texture space of the video shot. The system then outputs a second frame that shows the target person as the replacement for the source person.

Abstract: Described are systems and method directed to generation of a dimensionally accurate three-dimensional (“3D”) model of a body, such as a human body, based on two-dimensional (“2D”) images of that body. A user may use a 2D camera, such as a digital camera typically included in many of today's portable devices (e.g., cell phones, tablets, laptops, etc.) and obtain a series of 2D body images of their body from different views with respect to the camera. The 2D body images may then be used to generate a plurality of predicted body parameters corresponding to the body represented in the 2D body images. Those predicted body parameters may then be further processed to generate a dimensionally accurate 3D model or avatar of the body of the user.

Abstract: Described are systems and methods directed to generation of a personalized three-dimensional (“3D”) body model of a body, such as a human body, based on two-dimensional (“2D”) images of that body and the generation and presentation of predicted personalized 3D body models of the body when one or more body measurements (e.g., body fat, body weight, muscle mass) are changed. For example, a user may provide a target body measurement value and the implementations will generate one or more predicted personalized 3D body models representative of a predicted appearance of the body with the target body measurement value.

Abstract: Described are systems and methods directed to generation of a personalized three-dimensional (“3D”) body model of a body, such as a human body, based on two-dimensional (“2D”) images of that body and the generation and presentation of predicted personalized 3D body models of the body when one or more body measurements (e.g., body fat, body weight, muscle mass) are changed. For example, a user may provide a target body measurement value and the implementations will generate one or more predicted personalized 3D body models representative of a predicted appearance of the body with the target body measurement value.

Abstract: Described are systems and methods directed to generation of a personalized three-dimensional (“3D”) body model of a body, such as a human body, based on two-dimensional (“2D”) images of that body and the generation and presentation of predicted personalized 3D body models of the body when one or more body measurements (e.g., body fat, body weight, muscle mass) are changed. For example, a user may provide a target body measurement value and the implementations will generate one or more predicted personalized 3D body models representative of a predicted appearance of the body with the target body measurement value.

Abstract: A method of simulating a cloth includes: determining a first shape for the cloth at a first time; determining a first goal shape for the cloth at the first time; determining a first goal-directed force for the cloth at the first time from the first shape and the first goal shape, wherein the first goal directed force includes a component that measures a difference between the first shape and the first goal shape; and determining a second shape for the cloth at a second time from the first shape and the first goal-directed force by dynamically advancing the cloth from the first time to the second time.