Abstract: A system defining a visual sensing pipeline architecture flexibly allows application developers to dynamically adapt the spatial resolution and update rate of different “rhythmic pixel encoding regions” in a scene. The system ingests pixel streams with raster-scan pixel read-out patterns, but only encodes relevant pixels prior to storing them in memory. The system includes streaming hardware to decode the stored rhythmic pixel encoding region stream into traditional frame-based representations to feed into standard computer vision algorithms.

Abstract: A haptic feedback system to stimulate physical sensation of handling a fluid in virtual spaces. The system includes a physical vessel containing a solid object and mechanical means to move the solid object therein, a virtual reality module to track a position of the physical vessel and a corresponding virtual vessel, logic to simulate a fluid contained in the virtual vessel based on the tracked position of the virtual vessel, logic to calculate coordinates of a center of gravity for the simulated fluid based on the tracked position of the virtual vessel, logic to translate the calculated coordinates of the center of gravity for the simulated fluid into cylindrical coordinates to which to move the solid object in the physical vessel, and logic to send instructions to the mechanical means to move the solid object in the physical vessel based on the cylindrical coordinates to shift the weight of the physical vessel in accordance with the center of gravity of the simulated fluid in the virtual vessel.

Abstract: In accordance with embodiments disclosed herein, there are provided herein systems, methods, and apparatuses for implementing an image sensor reconfiguration framework for seamless resolution-based tradeoffs.

Abstract: A system to output a cross-talk cancelled spatial audio signal to two speakers, including a user tracking module to provide a location and orientation of a head of a user, and a time-domain dynamic crosstalk cancellation module to receive a spatial audio signal, the location and orientation of the head of the user from the user tracking module, a location for each of the two speakers, and generate the time-domain crosstalk cancelled spatial audio signal based thereon.

Abstract: Fidelity-driven runtime thermal management for near-sensor architectures is provided. In this regard, a runtime controller is provided for controlling an operational mode of a vision or imaging system driven by fidelity demands. The runtime controller is responsible for guaranteeing the fidelity demands of a vision application and coordinating state transfer between operating modes to ensure a smooth transition. Under this approach, the vision application only needs to provide the runtime controller with high-level vision/imaging fidelity demands and when to trigger them. The runtime controller translates these demands into effective thermal management. To do this, the runtime controller applies application-specific requirements into appropriate policy parameters and activates temperature reduction mechanisms, such as clock gating and task offload.

Abstract: Systems and methods for collaborative illumination estimation for mobile mixed-reality devices are provided. Embodiments described herein compose an illumination estimate using video data capturing an environment which includes a reflective object, such as a light probe. Radiance samples are computed from light reflections from the reflective object, which are then interpolated to compose a realistic estimation of physical lighting of the environment. Robust illumination estimation is provided in a computationally efficient manner, supplying real-time updates to facilitate integration with augmented reality (AR) systems and other image processing applications. The computational efficiency of this approach allows for implementation in lower-resource environments, such as mobile devices. In some examples, multiple devices can collaborate to capture the environment from different viewpoints and enhance realism and fidelity in their illumination estimates.

Abstract: In accordance with embodiments disclosed herein, there are provided herein systems, methods, and apparatuses for implementing an image sensor reconfiguration framework for seamless resolution-based tradeoffs.

Abstract: A haptic feedback system to stimulate physical sensation of handling a fluid in virtual spaces. The system includes a physical vessel containing a solid object and mechanical means to move the solid object therein, a virtual reality module to track a position of the physical vessel and a corresponding virtual vessel, logic to simulate a fluid contained in the virtual vessel based on the tracked position of the virtual vessel, logic to calculate coordinates of a center of gravity for the simulated fluid based on the tracked position of the virtual vessel, logic to translate the calculated coordinates of the center of gravity for the simulated fluid into cylindrical coordinates to which to move the solid object in the physical vessel, and logic to send instructions to the mechanical means to move the solid object in the physical vessel based on the cylindrical coordinates to shift the weight of the physical vessel in accordance with the center of gravity of the simulated fluid in the virtual vessel.

Abstract: Fidelity-driven runtime thermal management for near-sensor architectures is provided. In this regard, a runtime controller is provided for controlling an operational mode of a vision or imaging system driven by fidelity demands. The runtime controller is responsible for guaranteeing the fidelity demands of a vision application and coordinating state transfer between operating modes to ensure a smooth transition. Under this approach, the vision application only needs to provide the runtime controller with high-level vision/imaging fidelity demands and when to trigger them. The runtime controller translates these demands into effective thermal management. To do this, the runtime controller applies application-specific requirements into appropriate policy parameters and activates temperature reduction mechanisms, such as clock gating and task offload.

Abstract: Systems and methods for collaborative illumination estimation for mobile mixed-reality devices are provided. Embodiments described herein compose an illumination estimate using video data capturing an environment which includes a reflective object, such as a light probe. Radiance samples are computed from light reflections from the reflective object, which are then interpolated to compose a realistic estimation of physical lighting of the environment. Robust illumination estimation is provided in a computationally efficient manner, supplying real-time updates to facilitate integration with augmented reality (AR) systems and other image processing applications. The computational efficiency of this approach allows for implementation in lower-resource environments, such as mobile devices. In some examples, multiple devices can collaborate to capture the environment from different viewpoints and enhance realism and fidelity in their illumination estimates.

Abstract: The subject disclosure is directed towards energy saving mechanisms of image sensor circuitry (e.g., in a camera). Image quality data, such as provided by an application, is processed to make energy consumption of image sensor circuitry more proportional to output image quality by controlling the operation of one or more controllable power saving mechanisms of the image sensor circuitry. Power saving mechanisms may include a frequency controlled clock, the ability to turn off unneeded components, an inter-frame standby mode that puts the image sensor circuitry into a standby mode between capturing sequential frames, selectable parallel analog chains having different energy usage properties and column circuitry that allows turning off circuitry corresponding to unneeded columns of the sensor array.

Abstract: The subject disclosure is directed towards energy saving mechanisms of image sensor circuitry (e.g., in a camera). Image quality data, such as provided by an application, is processed to make energy consumption of image sensor circuitry more proportional to output image quality by controlling the operation of one or more controllable power saving mechanisms of the image sensor circuitry. Power saving mechanisms may include a frequency controlled clock, the ability to turn off unneeded components, an inter-frame standby mode that puts the image sensor circuitry into a standby mode between capturing sequential frames, selectable parallel analog chains having different energy usage properties and column circuitry that allows turning off circuitry corresponding to unneeded columns of the sensor array.

Abstract: An electronic pegboard setup is provided for assessing patient dexterity. In certain embodiments, the pegboard setup may be wireless and may employ pegs equipped with sensors that that allow tracking of the motion of the peg in three-dimensions and over time. In one such embodiment, a visual output of the motion path may be provided to the clinician. In such an embodiment, the device may provide quantitative data regarding motion path to assess patient dexterity.

Abstract: The present disclosure relates to an integrated system for measuring hand strength and dexterity. Specifically, the integrated system allows for measuring of hand muscle strength through the pinch-grip test, of intrinsic hand muscle strength, and testing of performance in various dexterity tests.