Abstract: In one or more implementations, body features of a subject can be tracked and a pose estimation produced using an unconstrained video sequence. The video sequence constitutes a physical record of the body features of the subject. A set of 2D coordinates corresponding to the body features of the subject can be received from a first neural network and 2D Keypoints are detected and transmitted to a second neural network. The second neural network can return corresponding depth estimation values, which can be processed to generate a set of 3D coordinates that correspond to the body features of the subject in camera space. Furthermore, one or more non-linear optimizations can be applied to fit a predetermined skeleton to the generate 3D Keypoints, and produce the pose estimation corresponding to the body features of the subject.

Abstract: A method for locating and tracking facial features in an unconstrained video sequence includes: in a face-detecting process, delineating, with region-bounding coordinates, the face of the subject within an image selected from the sequence; detecting, in the selected image, a small set of landmarks, corresponding to the face of the subject, using a convolutional neural network, trained to take as input an image region corresponding to the face of the—subject and to return a set of coordinates at computational speeds approximating real time; projectively fitting a three-dimensional character model to the detected landmarks, and using the fitted model to estimate physical locations of additional landmarks, so as to provide a complete hypothesized set of facial landmarks; and in a feature tracker process, updating the hypothesized set of facial landmarks to improve convergence between predicted feature locations and their actual physical locations based on data sampled from the selected image.

Abstract: A method for locating and tracking facial features in an unconstrained video sequence includes: in a face-detecting process, delineating, with region-bounding coordinates, the face of the subject within an image selected from the sequence; detecting, in the selected image, a small set of landmarks, corresponding to the face of the subject, using a convolutional neural network, trained to take as input an image region corresponding to the face of the—subject and to return a set of coordinates at computational speeds approximating real time; projectively fitting a three-dimensional character model to the detected landmarks, and using the fitted model to estimate physical locations of additional landmarks, so as to provide a complete hypothesized set of facial landmarks; and in a feature tracker process, updating the hypothesized set of facial landmarks to improve convergence between predicted feature locations and their actual physical locations based on data sampled from the selected image.