Abstract: Methods and systems for determining a displacement of a peripheral device are provided. In one example, a peripheral device comprises: an image sensor, and a hardware processor configured to: control the image sensor to capture a first image of a surface when the peripheral device is at a first location on the surface, the first image comprising a feature of the first location of the surface; execute a trained machine learning model using data derived from the first image to estimate a displacement of the feature between the first image and a reference image captured at a second location of the surface; and determine a displacement of the peripheral device based on the estimated displacement of the feature.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, the method comprises: capturing a first image of a user's appendage and a physical peripheral input device, wherein the user's appendage covers at least a part of the physical peripheral input device; performing a segmentation of the first image into a first region of pixels corresponding to the user's appendage, wherein a second region of pixels corresponds to the physical peripheral input device; generating a composite image from the first image based on rendering, in place of the second region of pixels, pixels representing a virtual peripheral input device, such that the virtual peripheral input device appears in the composite image as at a same spatial location as the physical peripheral input device and covered at least partially by the user's appendage; and displaying the composite image to the user.

Abstract: Methods and systems for determining a displacement of a peripheral device are provided. In one example, a peripheral device comprises: an image sensor, and a hardware processor configured to: control the image sensor to capture a first image of a surface when the peripheral device is at a first location on the surface, the first image comprising a feature of the first location of the surface; execute a trained machine learning model using data derived from the first image to estimate a displacement of the feature between the first image and a reference image captured at a second location of the surface; and determine a displacement of the peripheral device based on the estimated displacement of the feature.

Abstract: Methods and systems for determining a displacement of a peripheral device are provided. In one example, a peripheral device comprises: an image sensor, and a hardware processor configured to: control the image sensor to capture a first image of a surface when the peripheral device is at a first location on the surface, the first image comprising a feature of the first location of the surface; execute a trained machine learning model using data derived from the first image to estimate a displacement of the feature between the first image and a reference image captured at a second location of the surface; and determine a displacement of the peripheral device based on the estimated displacement of the feature.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, a method comprises: capturing, at a first time and using a camera of a head-mounted display (HMD) of a user, a first image of a physical interaction of the user with a physical object; measuring a movement of the HMD with respect to the physical object between the first time and a second time; processing the first image based on the measurement of the movement of the HMD to generate a second image; generating, based on the second image, a composite image of a virtual interaction involving the user; and displaying, via the HMD and based on the composite image, the virtual interaction in place of the physical interaction to the user at the second time.

Abstract: Methods and systems for determining a displacement of a peripheral device are provided. In one example, a peripheral device comprises: an image sensor, and a hardware processor configured to: control the image sensor to capture a first image of a surface when the peripheral device is at a first location on the surface, the first image comprising a feature of the first location of the surface; execute a trained machine learning model using data derived from the first image to estimate a displacement of the feature between the first image and a reference image captured at a second location of the surface; and determine a displacement of the peripheral device based on the estimated displacement of the feature.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, a method comprises: capturing, at a first time and using a camera of a head-mounted display (HMD) of a user, a first image of a physical interaction of the user with a physical object; measuring a movement of the HMD with respect to the physical object between the first time and a second time; processing the first image based on the measurement of the movement of the HMD to generate a second image; generating, based on the second image, a composite image of a virtual interaction involving the user; and displaying, via the HMD and based on the composite image, the virtual interaction in place of the physical interaction to the user at the second time.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, the method comprises: capturing a first image of a user's appendage and a physical peripheral input device, wherein the user's appendage covers at least a part of the physical peripheral input device; performing a segmentation of the first image into a first region of pixels corresponding to the user's appendage, wherein a second region of pixels corresponds to the physical peripheral input device; generating a composite image from the first image based on rendering, in place of the second region of pixels, pixels representing a virtual peripheral input device, such that the virtual peripheral input device appears in the composite image as at a same spatial location as the physical peripheral input device and covered at least partially by the user's appendage; and displaying the composite image to the user.

Abstract: In one embodiment, captured video summaries are tagged with metadata so the videos can be easily searched. The videos are classified into different scenes, depending on the type of action in the video, so searching can be based on the type of scene. In one embodiment, tags are provided for moving objects or people. The type of object that is moving is tagged (car, ball, person, pet, etc.). Video search results are ranked based on the weighting of the video events or video summaries. The video summary weighting focuses on important events, with multiple videos/images over a period of time condensed into a short summary video. This creates a weighted video summary with different time-lapse speeds that focuses on important events.

Abstract: In one embodiment of the present invention, a video camera selectively streams to a remote server. Still images or short video events are intermittently transmitted when there is no significant motion detected. When significant motion is detected, video is streamed to the remote server. The images and video can be higher resolution than the bandwidth used, by locally buffering the images and video, and transmitting it at a lower frame rate that extends to when there is no live streaming. This provides a time-delayed stream, but with more resolution at lower bandwidth.

Abstract: In one embodiment, the User Interface (UI) provides a live stream from the webcam, with markers on the side indicating the stored, detected important events (such as by using a series of bubbles indicating how long ago an event occurred). The indicators are marked to indicate the relative importance, such as with color coding. Upon selection of an indicator by the user, the time-lapse summary is displayed, along with a time of day indication. Alternately, the user can select to have a time-lapse display of all the events in sequence, using a more condensed time lapse, with less important events having less time or being left out.

Abstract: In one embodiment of the present invention, a video camera selectively streams to a remote server. Still images or short video events are intermittently transmitted when there is no significant motion detected. When significant motion is detected, video is streamed to the remote server. The images and video can be higher resolution than the bandwidth used, by locally buffering the images and video, and transmitting it at a lower frame rate that extends to when there is no live streaming. This provides a time-delayed stream, but with more resolution at lower bandwidth.

Abstract: In one embodiment, captured video summaries are tagged with metadata so the videos can be easily searched. The videos are classified into different scenes, depending on the type of action in the video, so searching can be based on the type of scene. In one embodiment, tags are provided for moving objects or people. The type of object that is moving is tagged (car, ball, person, pet, etc.). Video search results are ranked based on the weighting of the video events or video summaries. The video summary weighting focuses on important events, with multiple videos/images over a period of time condensed into a short summary video. This creates a weighted video summary with different time-lapse speeds that focuses on important events.

Abstract: In one embodiment, image recognition is used to determine the type of location where the camera is mounted, such as indoors or outdoors, in a conference room or in a dining room. A filter for selecting the types of events for a summary has parameters varied depending on the type of location. For example, an indoor location may tag events where humans are detected, and ignore animals (pets). An outdoor location can have the parameters set to detect both human and animal movement.

Abstract: In one embodiment, the User Interface (UI) provides a live stream from the webcam, with markers on the side indicating the stored, detected important events (such as by using a series of bubbles indicating how long ago an event occurred). The indicators are marked to indicate the relative importance, such as with color coding. Upon selection of an indicator by the user, the time-lapse summary is displayed, along with a time of day indication. Alternately, the user can select to have a time-lapse display of all the events in sequence, using a more condensed time lapse, with less important events having less time or being left out.

Abstract: In one embodiment, the User Interface (UI) provides a live stream from the webcam, with markers on the side indicating the stored, detected important events (such as by using a series of bubbles indicating how long ago an event occurred). The indicators are marked to indicate the relative importance, such as with color coding. Upon selection of an indicator by the user, the time-lapse summary is displayed, along with a time of day indication. Alternately, the user can select to have a time-lapse display of all the events in sequence, using a more condensed time lapse, with less important events having less time or being left out.