Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In some embodiments, a system comprising one or more processors configured to track a location of an input device within a physical environment via a three-dimensional (3D) tracking system, and modify a tracking parameter of the 3D tracking system while tracking the location of the input device based on the determined location of the input device within the physical environment. The input device may be coupled to a virtual reality display system and tracking the location of the location of the input device can be used for interacting with the virtual reality display system.

Abstract: A method can include intercepting a command generated by a computer peripheral device; determining a function corresponding to the command; generating a lighting output schema configured to be output on one or more visual output elements on the computer peripheral device based on the determined function; identifying, based on the determined function, whether a second software component can accept the command; and in response to determining that the second software component can accept the command, sending the intercepted command to the second software component.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, the method comprises: capturing a first image of a user's appendage and a physical peripheral input device, wherein the user's appendage covers at least a part of the physical peripheral input device; performing a segmentation of the first image into a first region of pixels corresponding to the user's appendage, wherein a second region of pixels corresponds to the physical peripheral input device; generating a composite image from the first image based on rendering, in place of the second region of pixels, pixels representing a virtual peripheral input device, such that the virtual peripheral input device appears in the composite image as at a same spatial location as the physical peripheral input device and covered at least partially by the user's appendage; and displaying the composite image to the user.

Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In an embodiment there is provided a user interface device for interfacing a user with a computer, the computer comprising at least one processor to generate instructions for display of a graphical environment of a virtual reality simulation. The user interface device includes a sensor for detecting activation and/or a sensor for detecting proximity of a body part of the user (e.g., hand). At least partially based on detection by one of these sensors, a transition between virtual reality states is triggered.

Abstract: A method can include intercepting a command generated by a computer peripheral device; determining a function corresponding to the command; generating a lighting output schema configured to be output on one or more visual output elements on the computer peripheral device based on the determined function; identifying, based on the determined function, whether a second software component can accept the command; and in response to determining that the second software component can accept the command, sending the intercepted command to the second software component.

Abstract: In some embodiments, a system comprising one or more processors configured to track a location of an input device within a physical environment via a three-dimensional (3D) tracking system, and modify a tracking parameter of the 3D tracking system while tracking the location of the input device based on the determined location of the input device within the physical environment. The input device may be coupled to a virtual reality display system and tracking the location of the location of the input device can be used for interacting with the virtual reality display system.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, a method comprises: capturing, at a first time and using a camera of a head-mounted display (HMD) of a user, a first image of a physical interaction of the user with a physical object; measuring a movement of the HMD with respect to the physical object between the first time and a second time; processing the first image based on the measurement of the movement of the HMD to generate a second image; generating, based on the second image, a composite image of a virtual interaction involving the user; and displaying, via the HMD and based on the composite image, the virtual interaction in place of the physical interaction to the user at the second time.

Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In some embodiments, a system comprising one or more processors configured to track a location of an input device within a physical environment via a three-dimensional (3D) tracking system, and modify a tracking parameter of the 3D tracking system while tracking the location of the input device based on the determined location of the input device within the physical environment. The input device may be coupled to a virtual reality display system and tracking the location of the location of the input device can be used for interacting with the virtual reality display system.

Abstract: In an embodiment there is provided a user interface device for interfacing a user with a computer, the computer comprising at least one processor to generate instructions for display of a graphical environment of a virtual reality simulation. The user interface device includes a sensor for detecting activation and/or a sensor for detecting proximity of a body part of the user (e.g., hand). At least partially based on detection by one of these sensors, a transition between virtual reality states is triggered.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, a method comprises: capturing, at a first time and using a camera of a head-mounted display (HMD) of a user, a first image of a physical interaction of the user with a physical object; measuring a movement of the HMD with respect to the physical object between the first time and a second time; processing the first image based on the measurement of the movement of the HMD to generate a second image; generating, based on the second image, a composite image of a virtual interaction involving the user; and displaying, via the HMD and based on the composite image, the virtual interaction in place of the physical interaction to the user at the second time.

Abstract: Methods and systems for providing a mixed reality (MR) interaction are provided. In one example, the method comprises: capturing a first image of a user's appendage and a physical peripheral input device, wherein the user's appendage covers at least a part of the physical peripheral input device; performing a segmentation of the first image into a first region of pixels corresponding to the user's appendage, wherein a second region of pixels corresponds to the physical peripheral input device; generating a composite image from the first image based on rendering, in place of the second region of pixels, pixels representing a virtual peripheral input device, such that the virtual peripheral input device appears in the composite image as at a same spatial location as the physical peripheral input device and covered at least partially by the user's appendage; and displaying the composite image to the user.

Abstract: In an embodiment there is provided a user interface device for interfacing a user with a computer, the computer comprising at least one processor to generate instructions for display of a graphical environment of a virtual reality simulation. The user interface device includes a sensor for detecting activation and/or a sensor for detecting proximity of a body part of the user (e.g., hand). At least partially based on detection by one of these sensors, a transition between virtual reality states is triggered.

Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In some embodiments, a system comprising one or more processors configured to track a location of an input device within a physical environment via a three-dimensional (3D) tracking system, and modify a tracking parameter of the 3D tracking system while tracking the location of the input device based on the determined location of the input device within the physical environment. The input device may be coupled to a virtual reality display system and tracking the location of the location of the input device can be used for interacting with the virtual reality display system.

Abstract: In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

Abstract: In an embodiment there is provided a user interface device for interfacing a user with a computer, the computer comprising at least one processor to generate instructions for display of a graphical environment of a virtual reality simulation. The user interface device includes a sensor for detecting activation and/or a sensor for detecting proximity of a body part of the user (e.g., hand). At least partially based on detection by one of these sensors, a transition between virtual reality states is triggered.