Abstract: A method and system for augmenting live video feeds with augmented reality (AR) effects. A live video feed comprising a plurality of video frames is received and the format of the video frames is determined. The video frames are converted to a format compatible with an AR software development kit (SDK). One or more AR effects from the AR SDK are applied to the converted frames. This can include detecting depictions of objects in the frames and applying effects to the detected objects. The effects can be selected based on detected object types. The frames are then re-converted back to the original format. If the frame rate differs between the video feed and AR SDK, frame rate conversion is performed before and after applying the AR effects. The augmented video frames including the AR effects are provided as output, such as for broadcast or display.

Abstract: Methods and systems are disclosed for generating an extended reality (XR) experience using a statically positioned device. The system receives, from a camera of a stationary device, an image depicting a real-world object, the camera being directed in a stationary manner towards a specified field of view of a real-world environment. The system analyzes the image using a machine learning model to predict tracking information for the real-world object, the machine learning model trained based on a plurality of training images depicting real-world objects in the specified field of view of the real-world environment and corresponding ground-truth tracking information for the real-world objects. The system selects an extended reality (XR) experience from a plurality of XR experiences and overlays one or more XR elements associated with the XR experience on the image based on the predicted tracking information to generate a modified image.

Abstract: A method and system for augmenting live video feeds with augmented reality (AR) effects. A live video feed comprising a plurality of video frames is received and the format of the video frames is determined. The video frames are converted to a format compatible with an AR software development kit (SDK). One or more AR effects from the AR SDK are applied to the converted frames. This can include detecting depictions of objects in the frames and applying effects to the detected objects. The effects can be selected based on detected object types. The frames are then re-converted back to the original format. If the frame rate differs between the video feed and AR SDK, frame rate conversion is performed before and after applying the AR effects. The augmented video frames including the AR effects are provided as output, such as for broadcast or display.

Abstract: Methods and systems are disclosed for generating an extended reality (XR) experience using a statically positioned device. The system receives, from a camera of a stationary device, an image depicting a real-world object, the camera being directed in a stationary manner towards a specified field of view of a real-world environment. The system analyzes the image using a machine learning model to predict tracking information for the real-world object, the machine learning model trained based on a plurality of training images depicting real-world objects in the specified field of view of the real-world environment and corresponding ground-truth tracking information for the real-world objects. The system selects an extended reality (XR) experience from a plurality of XR experiences and overlays one or more XR elements associated with the XR experience on the image based on the predicted tracking information to generate a modified image.