Abstract: Techniques are provided for re-orienting a field of view of a depth camera having one or more sensors. The depth camera may have one or more sensors for generating a depth image and may also have an RGB camera. In some embodiments, the field of view is re-oriented based on the depth image. The position of the sensor(s) may be altered to change the field of view automatically based on an analysis of objects in the depth image. The re-orientation process may be repeated until a desired orientation of the sensor is determined. Input from the RGB camera might be used to validate a final orientation of the depth camera, but is not required to during the process of determining new possible orientation of the field of view.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may be determined and a model may be adjusted based on the location or position of the one or more extremities.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Abstract: Systems and associated methods for processing textures in a graphical processing unit (GPU) are disclosed. Textures may be managed on a per region (e.g., tile) basis, which allows efficient use of texture memory. Moreover, very large textures may be used. Techniques provide for both texture streaming, as well as sparse textures. A GPU texture unit may be used to intelligently clamp LOD based on a shader specified value. The texture unit may provide feedback to the shader to allow the shader to react conditionally based on whether clamping was used, etc. Per region (e.g., per-tile) independent mipmap stacks may be used to allow very large textures.

Abstract: The architecture implements A-buffer in hardware by extending hardware to efficiently store a variable amount of data for each pixel. In operation, a prepass is performed to generate the counts of the fragments per pixel in a count buffer, followed by a prefix sum pass on the generated count buffer to calculate locations in a fragment buffer in which to store all the fragments linearly. An index is generated for a given pixel in the prefix sum pass and stored in a location buffer. Access to the pixel fragments is then accomplished using the index. Linear storage of the data allows for a fast rendering pass that stores all the fragments to a memory buffer without needing to look at the contents of the fragments. This is then followed by a resolve pass on the fragment buffer to generate the final image.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Abstract: A sensor system creates a sequence of depth images that are used to detect and track motion of objects within range of the sensor system. A reference image is created and updated based on a moving average (or other function) of a set of depth images. A new depth images is compared to the reference image to create a motion image, which is an image file (or other data structure) with data representing motion. The new depth image is also used to update the reference image. The data in the motion image is grouped and associated with one or more objects being tracked. The tracking of the objects is updated by the grouped data in the motion image. The new positions of the objects are used to update an application. For example, a video game system will update the position of images displayed in the video based on the new positions of the objects. In one implementation, avatars can be moved based on movement of the user in front of a camera.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. The image may then be processed. For example, the image may be downsampled, a shadow, noise, and/or a missing potion in the image may be determined, pixels in the image that may be outside a range defined by a capture device associated with the image may be determined, a portion of the image associated with a floor may be detected. Additionally, a target in the image may be determined and scanned. A refined image may then be rendered based on the processed image. The refined image may then be processed to, for example, track a user.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may be determined and a model may be adjusted based on the location or position of the one or more extremities.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may then be discarded to isolate one or more voxels associated with a foreground object such as a human target and the isolated voxels associated with the foreground object may be processed.

Abstract: An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may then be determined.

Abstract: A method of tracking a target includes receiving from a source an observed depth image of a scene including the target. Each pixel of the observed depth image is labeled as either a foreground pixel belonging to the target or a background pixel not belonging to the target. Each foreground pixel is labeled with body part information indicating a likelihood that that foreground pixel belongs to one or more body parts of the target. The target is modeled with a skeleton including a plurality of skeletal points, each skeletal point including a three dimensional position derived from body part information of one or more foreground pixels.

Abstract: Allocation of memory registers for shaders by a processor is described herein. For each shader, registers are allocated based on the shader's level of complexity. Simpler shader instances are restricted to a smaller number of memory registers. More complex shader instances are allotted more registers. To do so, developers' high level shading level (HLSL) language includes template classes of shaders that can later be replaced by complex or simple versions of the shader. The HLSL is converted to bytecode that can be used to rasterize pixels on a computing device.

Abstract: A high level shader language compiler incorporates transforms to optimize shader code for graphics processing hardware. An instruction reordering transform determines instruction encapsulations of dependent instructions that reduce concurrent register usage by the shader. A phase pulling transform re-organizes the shader's instructions into phases that reduce a measure of depth of texture loads. A register assigning transform assigns registers to lower register usage by the shader.

Abstract: A high level shader language compiler incorporates transforms to optimize shader code for graphics processing hardware. An instruction reordering transform determines instruction encapsulations of dependent instructions that reduce concurrent register usage by the shader. A phase pulling transform re-organizes the shader's instructions into phases that reduce a measure of depth of texture loads. A register assigning transform assigns registers to lower register usage by the shader.

Abstract: The architecture implements A-buffer in hardware by extending hardware to efficiently store a variable amount of data for each pixel. In operation, a prepass is performed to generate the counts of the fragments per pixel in a count buffer, followed by a prefix sum pass on the generated count buffer to calculate locations in a fragment buffer in which to store all the fragments linearly. An index is generated for a given pixel in the prefix sum pass and stored in a location buffer. Access to the pixel fragments is then accomplished using the index. Linear storage of the data allows for a fast rendering pass that stores all the fragments to a memory buffer without needing to look at the contents of the fragments. This is then followed by a resolve pass on the fragment buffer to generate the final image.

Abstract: A shader program capable of execution on a GPU is analyzed for constant expressions. These constant expressions are replaced with references to registers or memory addresses on the GPU. A preshader is created that comprises two executable files. The first executable file contains the shader program with the each constant expression removed and replaced with a unique reference accessible by the GPU. The first file is executable at the GPU. A second file contains the removed constant expressions along with instructions to place the values generated at the associated reference. The second executable file is executable at a CPU. When the preshader is executed, an instance of the first file is executed at the GPU for each vertex or pixel that is displayed. One instance of the second file is executed at the CPU. As the preshader is executed, the constant expressions in the second file are evaluated and the resulting intermediate values are passed to each instance of the first file on the GPU.

Abstract: The discrete cosine transform (DCT) is mapped to a graphics processing unit (GPU) instead of a central processing unit (CPU). The DCT can be implemented using a shader-based process or a host-based process. A matrix is applied to a set of pixel samples. The samples are processed in either rows or columns first, and then the processing is performed in the opposite direction. The number of times a shader program is changed is minimized by processing all samples that use a particular shader (e.g., the first shader) at the same time (e.g., in sequence).