Abstract: Systems and techniques are described for localizing a camera. An example method includes generating first image data comprising a first image plane representing the planar scene from a first point-of-view. The example includes generating second image data comprising the second image plane representing the planar scene from a second point-of-view. The example includes identifying a plurality of keypoints of the planar scene that is visible in the first image plane and the second image plane. The example includes matching the plurality of keypoints of a planar scene to a first pixel of a first image plane and a second pixel of a second image plane. The example includes generating a homography relating the planar scene, the first image plane, and the second image plane. The example includes generating, based on the homography, at least one of a calibration parameter or a localization parameter for an unlocalized camera.

Abstract: Summaries of broadcasts of live events are generated based on intensities of audio signals captured during the live events. Streams of multimedia including video signals and audio signals are captured by one or more cameras. The video signals are processed to identify specific activities of interest (e.g., plays of a sporting event) during the live event. Audio signals captured concurrently with the video signals are processed to determine their respective intensities or other acoustic characteristics. The activities of interest are ranked based on intensities of the audio signals. A multimedia stream representing a summary of a media program and includes the highest-ranking video signals and corresponding audio signals is generated and transmitted to one or more devices of viewers.

Abstract: Characteristics of a user's movement are evaluated based on performance of activities by a user within a field of view of a camera. Video data representing performance of a series of movements by the user is acquired by the camera. Pose data is determined based on the video data, the pose data representing positions of the user's body while performing the movements. The pose data is compared to a set of existing videos that correspond to known errors to identify errors performed by the user. The errors may be used to generate scores for various characteristics of the user's movement. Based on the errors, exercises or other activities to improve the movement of the user may be determined and included in an output presented to the user.

Abstract: Video output is generated based on first video data that depicts the user performing an activity. Poses of the user during performance of the activity are compared with second video data that depicts an instructor performing the activity. Corresponding poses of the user's body and the instructor's body may be determined through comparison of the first and second video data. The video data is used to determine the rate of motion of the user and to generate video output in which a visual representation of the instructor moves at a rate similar to the that of the user. For example, video output generated based on an instructional fitness video may be synchronized so that movement of the presented instructor matches the rate of movement of the user performing an exercise, improving user comprehension and performance.

Abstract: Described are systems and methods directed to the processing of two-dimensional (“2D”) images of a body to determine a physical activity performed by the body, repetitions of the physical activity, whether the body is performing the physical activity with proper form, and providing physical activity feedback. In addition, the disclosed implementations are able to determine the physical activity, repetitions, and/or form through the processing of 2D partial body images that include less than all of the body of the user.

Abstract: Systems for assisting a user in performance of a meditation activity or another type of activity are described. The systems receive user input and sensor data indicating physiological values associated with the user. These values are used to determine a recommended type of activity and a length of time for the activity. While the user performs the activity, sensors are used to measure physiological values, and an output that is provided to the user is selected based on the measured physiological values. The output may be selected to assist the user in reaching target physiological values, such as a slower respiration rate. After completion of the activity, additional physiological values are used to determine the effectiveness of the activity and the output that was provided. The effectiveness of the activity and the output may be used to determine future recommendations and future output.

Abstract: Characteristics of a user's movement are evaluated based on performance of activities by a user within a field of view of a camera. Video data representing performance of a series of movements by the user is acquired by the camera. Pose data is determined based on the video data, the pose data representing positions of the user's body while performing the movements. The pose data is compared to a set of existing videos that correspond to known errors to identify errors performed by the user. The errors may be used to generate scores for various characteristics of the user's movement. Based on the errors, exercises or other activities to improve the movement of the user may be determined and included in an output presented to the user.