Abstract: Methods and systems for remote multiplayer gameplay are disclosed. An example method can include joining a multiplayer interactive physical game including first and second players competing in an activity; capturing a first game video of a first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area; determining a first object flow associated with the first player from the first game video, by performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player; generating a first analytic based on the first object flow; receiving a second analytic associated with the second player in a second gaming area; and generating a feedback to the first player based on the first analytic or the second analytic.

Abstract: Methods and systems are disclosed for pose comparison, interactive physical gaming, and remote fitness training on a user computing device. The methods and systems are configured to first receive a reference feature generated from a frame of a reference video, the reference feature computed from a reference posture of a reference person in the frame of the reference video. Next, receive a frame of a user video, the frame of the user video comprising a user. Next, extract a user posture from the frame of the user video, by performing a machine learning-based computer vision algorithm that detects one or more body key points of the user in an image plane of the user video. Finally, generate a user feature from the user posture; and determine an output score based on a distance between the reference feature and the user feature.

Abstract: Methods and systems for image clustering are described, and include first receiving a plurality of images from a video of a game. Next, determining a first feature vector of a first player from the plurality of images by inputting the plurality of images to an artificial intelligence (AI)-based process. Next, dividing the plurality of images into a first subset of images and a second subset of images based on the first feature vector. Next, receiving a first identifier tagging the first player in a first image of the first subset. Finally, identifying a second player in a second image in the second subset by propagating the first identifier of the first subset, based on a distance measure associated with the first feature vector from the artificial intelligence (AI)-based process. Running on a mobile computing device, the method allows multiplayer tagging to be easily performed in almost any environment.

Abstract: Methods and systems for image clustering are described, and include determining a plurality of images from a video of a game, the video captured by a camera on a mobile device, where at least one image of the plurality of images is segmented from a video frame of the video; determining a feature vector from the at least one image; dividing the images into a first subset and a second subset based on the feature vector; tagging a first player in a first image of the first subset with an identifier, where the identifier differentiates the images in the first subset to a plurality of players; and identifying a second player in a second image in the second subset by propagating the identifier of the first subset, based on a distance measure associated with the feature vector. Running on a mobile computing device, this invention allows multiplayer tagging to be easily performed in almost any environment.

Abstract: Methods and systems for performing location and movement determination in a three-dimensional environment using a device with a camera are disclosed. The methods and systems perform the steps of capturing an image of a user using the camera of the computing device, identifying the user from the image using a first algorithm, determining a first physical information associated with the user using a first algorithm, determining a second physical information of the user based on the first physical information and an input parameter of the user using a search process, and determining the location of the user in the environment based on the first physical information and the second physical information.

Abstract: Methods and systems for performing location and movement determination in a three-dimensional environment using a mobile device with a camera are disclosed. The methods and systems perform the steps of capturing one or more images of a user using the camera on the mobile computing device, determining the user from the images using a machine learning algorithm, determining a pose associated with the user using a machine vision algorithm, determining a depth of the user with respect to the camera based on the pose and an input height of the user using a binary search process, and determining the location of the user in the environment based on the pose and the depth.

Abstract: Methods and systems for multi-player training of body-eye coordination using mobile computing devices each having a camera are provided. The methods and systems execute instructions to capture a first training video of a first of player, using a first camera on a first mobile computing device; superimpose a visual cue onto the first training video at a visual cue location and for a cue period starting from a first time instant; determine whether the first player has responded to the visual cue at a second time instant within the cue period; receive a notification from a second mobile computing device, where the notification was generated in response to determining that a second player has responded to the visual cue at a third time instant within the cue period; and in response to a determination that the first player has responded to the visual cue and to the notification, generate a feedback to the first player.

Abstract: Methods and systems for performing location and movement determination in a three-dimensional environment using a mobile device with a camera are disclosed. The methods and systems perform the steps of capturing one or more images of a user using the camera on the mobile computing device, determining the user from the images using a machine learning algorithm, determining a pose associated with the user using a machine vision algorithm, determining a depth of the user with respect to the camera based on the pose and an input height of the user using a binary search process, and determining the location of the user in the environment based on the pose and the depth.

Abstract: Various features and processes related to document collaboration are disclosed. In some implementations, animations are presented when updating a local document display to reflect changes made to the document at a remote device. In some implementations, a user can selectively highlight changes made by collaborators in a document. In some implementations, a user can select an identifier associated with another user to display a portion of a document that includes the other user's cursor location. In some implementations, text in document chat sessions can be automatically converted into hyperlinks which, when selected, cause a document editor to perform an operation.

Abstract: Methods and systems for image clustering are described, and include determining a plurality of images from a video of a game, the video captured by a camera on a mobile device, where at least one image of the plurality of images is segmented from a video frame of the video; determining a feature vector from the at least one image; dividing the images into a first subset and a second subset based on the feature vector; tagging a first player in a first image of the first subset with an identifier, where the identifier differentiates the images in the first subset to a plurality of players; and identifying a second player in a second image in the second subset by propagating the identifier of the first subset, based on a distance measure associated with the feature vector. Running on a mobile computing device, this invention allows multiplayer tagging to be easily performed in almost any environment.

Abstract: Methods and systems for virtual coaching and performance training using a mobile device are disclosed. The methods and systems perform the steps of capturing a training video of one or more players, using the camera on the mobile computing device; superimposing a visual cue onto the training video at a first location and for a cue period starting from a first time instant; determining whether at least one of the one or more players has responded to the visual cue at a second time instant within the cue period, by analyzing a body posture flow of each player between the first time instant and the second time instant, wherein each body posture flow is extracted from the training video by performing a computer vision algorithm on one or more frames of the training video; and in response to determining that at least one player has responded to the visual cue, generating a feedback to the one or more players.

Abstract: Various features and processes related to document collaboration are disclosed. In some implementations, animations are presented when updating a local document display to reflect changes made to the document at a remote device. In some implementations, a user can selectively highlight changes made by collaborators in a document. In some implementations, a user can select an identifier associated with another user to display a portion of a document that includes the other user's cursor location. In some implementations, text in document chat sessions can be automatically converted into hyperlinks which, when selected, cause a document editor to perform an operation.

Abstract: Methods and systems for touchless control of a device include receiving an image from a camera and applying a machine learning technique the image to determine a first map, the first map including a function that generates a correspondence between data associated with the image to a feature space. A second map including a representation of a user in the image can be determined based on the first map. Further, a pose of the user can be determined based on the second map. The pose of the user and a portion of a body of the user can be tracked to determine a mapping between the portion of the user's body and an input feature of a device. The input feature can be modified based on the mapping to enable a touchless interaction between the user and the device.

Abstract: Methods and systems for detecting a player in a gaming environment using a mobile device are disclosed. The methods and systems perform the steps of receiving calibration data from a camera of a user device, the calibration data comprising a first reference point associated with the gaming environment, a second reference point associated with the gaming environment, and sensor data associated with the user device; computing a camera projection based on the first reference point, the second reference point, and the sensor data obtained from the user device; receiving an input video of the gaming environment, wherein the input video is captured using the camera, and wherein frames of the input video comprise at least one player in the gaming environment; identifying an image location of the player by performing a computer vision algorithm on the input video; and determining a real-world location of the player based on the image location and the camera projection.

Abstract: Methods and systems are disclosed for real-time tracking of a multiplayer ball game using a mobile computing device. The methods and systems are configured to receive an input video of the multiplayer ball game captured using a camera on the mobile computing device in a gaming area associated with n players, where n is an integer and n?2; detect a plurality of player postures, by performing a computer vision algorithm on each of a plurality of frames of the input video; extract an associated player feature from each of the plurality of player postures; assign each of the plurality of player postures, based on the associated player feature, to one of at least n posture groups, where each player is represented by one of the at least n posture groups; and determine a player location for each player, based on the player's posture group.

Abstract: Methods and systems for remote multiplayer gameplay are disclosed. An example method can include joining a multiplayer interactive physical game including first and second players competing in an activity; capturing a first game video of a first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area; determining a first object flow associated with the first player from the first game video, by performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player; generating a first analytic based on the first object flow; receiving a second analytic associated with the second player in a second gaming area; and generating a feedback to the first player based on the first analytic or the second analytic.

Abstract: Methods and systems for virtual coaching and performance training using a mobile device are disclosed. The methods and systems perform the steps of capturing a training video of one or more players, using the camera on the mobile computing device; superimposing a visual cue onto the training video at a first location and for a cue period starting from a first time instant; determining whether at least one of the one or more players has responded to the visual cue at a second time instant within the cue period, by analyzing a body posture flow of each player between the first time instant and the second time instant, wherein each body posture flow is extracted from the training video by performing a computer vision algorithm on one or more frames of the training video; and in response to determining that at least one player has responded to the visual cue, generating a feedback to the one or more players.

Abstract: Methods and systems for detecting a player in a gaming environment using a mobile device are disclosed. The methods and systems perform the steps of receiving calibration data from a camera of a user device, the calibration data comprising a first reference point associated with the gaming environment, a second reference point associated with the gaming environment, and sensor data associated with the user device; computing a camera projection based on the first reference point, the second reference point, and the sensor data obtained from the user device; receiving an input video of the gaming environment, wherein the input video is captured using the camera, and wherein frames of the input video comprise at least one player in the gaming environment; identifying an image location of the player by performing a computer vision algorithm on the input video; and determining a real-world location of the player based on the image location and the camera projection.

Abstract: Methods and systems for remote real-time or asynchronous multiplayer gameplay using mobile devices are disclosed. The methods and systems include steps of mounting mobile computing devices in gaming areas associated with users; initiating a game between the users; analyzing movements of the users, ball(s), and/or other game equipment of the game by performing a computer vision algorithm on one or more frames of an input video of the game; and providing feedback to the users, where feedback may include instructions to be performed by the users and player analytics. Embodiments of the present invention use computer vision techniques to enable a resource-limited mobile device such as a smartphone to conduct the aforementioned steps.

Abstract: Methods and systems for mobile device-based touchless control can include receiving an image of a user and applying a convolutional neural network to the image to determine a feature map. Further, a key-points map and an affinity-field map can be determined based on the feature map, and the pose of the user can be determined based on the maps. The methods and system further include identifying an activation gesture of the user and determining a validity of a bounding box associated with the pose of the user, and tracking the pose of the user and a portion of a body of the user for performing the touchless control. Further, the methods and systems include determining a mapping between the portion of the user's body and a cursor position associated with an application, and moving the cursor position to enable touchless user interaction. Embodiments use computer vision techniques to enable a resource-limited mobile device to conduct the aforementioned steps.