Abstract: A device receives a sensor payload at a sensor service. The sensor payload includes a data field that comprises a first value that corresponds to a reading by a sensor device. The device sends the sensor payload to a broker process that is separate from the sensor service and that hosts plug-in logic. The device receives a sensor payload update from the broker process. The sensor payload update includes a second value for the data field that is different from the first value. The second value was determined by the plug-in logic. The device creates a modified sensor payload from the sensor payload, including updating the data field to comprise the second value. The device initiates an operating system action based on the modified sensor payload.

Abstract: A computer system receives a sensor payload from a human presence sensor that is associated with the computer system. The computer system determines, based on an arbitration policy, that the computer system is enabled to perform a particular action based on the priority of the human presence sensor. The arbitration policy determines the priority of the human presence sensor among a plurality of human presence sensors that are associated with the computer system. The computer system triggers the particular action based on the computer system being enabled to perform the particular action based on the content of the sensor payload.

Abstract: A device receives a sensor payload at a sensor service. The sensor payload includes a data field that comprises a first value that corresponds to a reading by a sensor device. The device sends the sensor payload to a broker process that is separate from the sensor service and that hosts plug-in logic. The device receives a sensor payload update from the broker process. The sensor payload update includes a second value for the data field that is different from the first value. The second value was determined by the plug-in logic. The device creates a modified sensor payload from the sensor payload, including updating the data field to comprise the second value. The device initiates an operating system action based on the modified sensor payload.

Abstract: User engagement is detected and used to control operation of a computing device. User engagement is detected by a sensor such as a camera that identifies if a user's face is oriented towards a display device. If the user is not facing the display device, the sensor determines that the user is unengaged. The computing device is thus able to perform a power-saving operation, such as dimming the display device, when the user is unengaged. The computing device includes an API that abstracts sensor data into a user engagement signal indicating that the user is either engaged or unengaged. The OS and applications running on the computing device act on the user engagement signal provided by the API without communicating directly with the sensor. The user engagement signal may be provided as an input to a state machine.

Abstract: A method of providing environmental information to an electronic device includes, at the electronic device, receiving ambient light from a surrounding environment of the electronic device with one or more camera sensors in data communication with a processor and capturing a raw pattern with the camera sensor, wherein the raw pattern includes light intensity information and light spectrum information. The method further includes deriving at least one ambient light value from the light intensity information and light spectrum information and providing the ambient light value to the processor of the electronic device.

Abstract: A method of providing environmental information to an electronic device includes, at the electronic device, receiving ambient light from a surrounding environment of the electronic device with one or more camera sensors in data communication with a processor and capturing a raw pattern with the camera sensor, wherein the raw pattern includes light intensity information and light spectrum information. The method further includes deriving at least one ambient light value from the light intensity information and light spectrum information and providing the ambient light value to the processor of the electronic device.

Abstract: A system for camera device discovery and enumeration detects a camera device connected to a computing device using a Universal Plug and Play (UPnP) protocol and generates a device object corresponding to the detected camera device with a user-mode camera driver. The user-mode camera driver associated with an operating system executing on the computing device. The system registers the generated device object with the operating system to provide a direct interface from the operating system to the camera device and to make the camera device available to a plurality of applications executing on the computing device and commands the camera device from at least one of the plurality of applications via the registered device object through the operating system, thereby improving the functioning of the computing device and the user experience.

Abstract: The methods described herein are configured to correlate frames of a video stream to a system clock. A correlator application receives a frame of a video stream from a capture device via a bus interface, the frame including start frame and end frame timestamps based on a device clock of the capture device. A second start frame timestamp and second end frame timestamp are predicted based on previously received frames and a system clock. A skew value of the frame is calculated based on differences between the start frame timestamps and the end frame timestamps. Upon the calculated skew value exceeding a skew threshold, the frame is corrected to correlate to the predicted start frame and end frame timestamps. The corrected frame is then provided for display. Correcting timestamps of video stream frames in the described manner reduces jitter and enables accurate synchronization of multiple video streams.

Abstract: Methods and devices for enabling controls of an imaging device may include receiving a data stream with a request for at least one control of an imaging device, wherein the at least one control is related to Universal Serial Bus (USB) video. The methods and devices may include identifying a defined functionality of the at least one control and transmitting the data stream to the imaging device to specify the operation of the imaging device based on the defined functionality of the at least one control. The methods and devices may also include generating metadata information from received video frame packets from an imaging device. The methods and devices may include identifying metadata information in the header of a video packet when the header size exceeds the standard header size. The methods and devices may include generating a metadata buffer with the identified metadata information.

Abstract: Described are examples for securing stream data received from a stream source. A secure mode can be enabled, based on a request from an application, for storing the stream data captured from the stream source in a secured buffer. The secured buffer can be allocated in a secure memory based at least in part on enabling the secure mode. A secured buffer identifier of the secured buffer can be provided to a driver of a device providing the stream source for storing the stream data captured from the stream source in the secured buffer. The secured buffer identifier of the secured buffer can also be provided to the application for accessing the stream data stored in the secured buffer.

Abstract: Techniques for arbitrating control access to a shared resource among multiple client applications concurrently consuming a shared data stream associated with the shared resource are described. A server component instantiates a shared resource component representing the shared resource and associated shared data stream. The server component publishes a shared resource interface that provides central interface for the multiple client applications to access the shared resource and associated shared data stream. Each client application may access the shared resource in either control mode or shared mode to consume the shared data stream. In control mode, the client application has read-write access to control parameters representing a configuration of the shared resource that are usable to modify aspects of the shared data stream. In shared mode, the client application has read-only access to the control parameters.

Abstract: The methods described herein are configured to correlate frames of a video stream to a system clock. A correlator application receives a frame of a video stream from a capture device via a bus interface, the frame including start frame and end frame timestamps based on a device clock of the capture device. A second start frame timestamp and second end frame timestamp are predicted based on previously received frames and a system clock. A skew value of the frame is calculated based on differences between the start frame timestamps and the end frame timestamps. Upon the calculated skew value exceeding a skew threshold, the frame is corrected to correlate to the predicted start frame and end frame timestamps. The corrected frame is then provided for display. Correcting timestamps of video stream frames in the described manner reduces jitter and enables accurate synchronization of multiple video streams.

Abstract: Memory descriptor list caching and pipeline processing techniques are described. In one or more examples, a method is configured to increase efficiency of buffer usage within a pipeline of a computing device. The method includes creation of a buffer in memory of the computing device and caching of a memory descriptor list by the computing device that describes the buffer in a buffer information cache and has associated therewith a handle that acts as a lookup to the memory descriptor list. The method also includes passing the handle through the pipeline of the computing device for processing of data within the buffer by one or more stages of the pipeline such that access to the data is obtained by the one or more stages by using the handle as the lookup as part of a call to obtain the memory descriptor list for the buffer from the buffer information cache.

Abstract: Described are examples for securely storing stream data received from a stream source. Stream data can be received from the stream source for storing in one or more secured buffers of a secure memory, wherein the stream data includes a header portion generated by a driver for the stream source and a payload portion generated from the stream source. The payload portion of the stream data can be stored in the one or more secured buffers and contiguous to a previous payload portion of previous stream data received from the stream source. The header portion of the stream data can be stored in another portion of the secure memory or a non-secure memory. A data frame, from the one or more secured buffers, comprising at least the payload portion of the stream data and the previous payload portion of the previous stream data can be provided to an application.

Abstract: Described are examples for securing stream data received from a stream source. A secure mode can be enabled, based on a request from an application, for storing the stream data captured from the stream source in a secured buffer. The secured buffer can be allocated in a secure memory based at least in part on enabling the secure mode. A secured buffer identifier of the secured buffer can be provided to a driver of a device providing the stream source for storing the stream data captured from the stream source in the secured buffer. The secured buffer identifier of the secured buffer can also be provided to the application for accessing the stream data stored in the secured buffer.

Abstract: A system for camera device discovery and enumeration detects a camera device connected to a computing device using a Universal Plug and Play (UPnP) protocol and generates a device object corresponding to the detected camera device with a user-mode camera driver. The user-mode camera driver associated with an operating system executing on the computing device. The system registers the generated device object with the operating system to provide a direct interface from the operating system to the camera device and to make the camera device available to a plurality of applications executing on the computing device and commands the camera device from at least one of the plurality of applications via the registered device object through the operating system, thereby improving the functioning of the computing device and the user experience.

Abstract: A system for camera device command and control detects a camera device connected to the system and accesses a set of interfaces defined by a standardized driver. The set of interfaces are a common set of interfaces associated with a device object and the device object is compatible with camera devices from different camera device manufacturers. The system queries the detected camera device for descriptors defining capabilities of the camera device using a packetized command generated based at least on the accessed set of interfaces and provides, based on the descriptors, one or more interfaces of the accessed set of interfaces to one or more applications to make the camera device available to the one or more applications, thereby simplifying connection of camera devices from different manufacturers and improving the user experience. The system then allows the camera device to be controlled.

Abstract: Described are examples for securing stream data received from a stream source. A secure mode can be enabled, based on a request from an application, for storing the stream data captured from the stream source in a secured buffer. The secured buffer can be allocated in a secure memory based at least in part on enabling the secure mode. A secured buffer identifier of the secured buffer can be provided to a driver of a device providing the stream source for storing the stream data captured from the stream source in the secured buffer. The secured buffer identifier of the secured buffer can also be provided to the application for accessing the stream data stored in the secured buffer.

Abstract: Technology related to virtualizing sensors is disclosed. In one example of the disclosed technology, a method can be used to allocate resources of a computing system. The method can include enumerating hardware sensors and software modules that are capable of interacting with one another. A topology can be negotiated that is compatible with capabilities of the enumerated hardware sensors and software modules. A first request can be received from a first requestor to configure at least one of the enumerated hardware sensors and software modules. The at least one of the enumerated hardware sensors and software modules can be configured in response to the received first request.

Abstract: The present disclosure provides devices and techniques for processing a media capture stream captured by a camera device using a chain device media foundation transform (DMFT). The techniques include configuring multiple DMFTs such that an original manufacturer (OEM) may have flexibility in independently selecting various functionalities from different sources (e.g., OS, OEM, IHV, ISV, or VARs) in order to maximize hardware capabilities while minimizing the drawbacks of creating a single DMFT. To that end, the implementation of the present disclosure includes a devices and techniques of chainable DMFTs such that a device transform manager may select a set of functionalities (e.g., face recognition, color effects, etc.) from multiple vendors to customize the camera's capabilities according to the OEM specification.