Abstract: Image registration is described. In an embodiment an image registration system executes automatic registration of images, for example medical images. In an example, semantic information is computed for each of the images to be registered comprising information about the types of objects in the images and the certainty of that information. In an example a mapping is found to register the images which takes into account the intensities of the image elements as well as the semantic information in a manner which is weighted by the certainty of that semantic information. For example, the semantic information is computed by estimating posterior distributions for the locations of anatomical structures by using a regression forest and transforming the posterior distributions into a probability map. In an example the mapping is found as a global point of inflection of an energy function, the energy function having a term related to the semantic information.

Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.

Abstract: A method of sensing depth using an RGB camera. In an example method, a color image of a scene is received from an RGB camera. The color image is applied to a trained machine learning component which uses features of the image elements to assign all or some of the image elements a depth value which represents the distance between the surface depicted by the image element and the RGB camera. In various examples, the machine learning component comprises one or more entangled geodesic random decision forests.

Abstract: Semi-supervised random decision forests for machine learning are described, for example, for interactive image segmentation, medical image analysis, and many other applications. In examples, a random decision forest comprising a plurality of hierarchical data structures is trained using both unlabeled and labeled observations. In examples, a training objective is used which seeks to cluster the observations based on the labels and similarity of the observations. In an example, a transducer assigns labels to the unlabeled observations on the basis of the clusters and certainty information. In an example, an inducer forms a generic clustering function by counting examples of class labels at leaves of the trees in the forest. In an example, an active learning module identifies regions in a feature space from which the observations are drawn using the clusters and certainty information; new observations from the identified regions are used to train the random decision forest.

Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.

Abstract: Video processing for motor task analysis is described. In various examples, a video of at least part of a person or animal carrying out a motor task, such as placing the forefinger on the nose, is input to a trained machine learning system to classify the motor task into one of a plurality of classes. In an example, motion descriptors such as optical flow are computed from pairs of frames of the video and the motion descriptors are input to the machine learning system. For example, during training the machine learning system identifies time-dependent and/or location-dependent acceleration or velocity features which discriminate between the classes of the motor task. In examples, the trained machine learning system computes, from the motion descriptors, the location dependent acceleration or velocity features which it has learned as being good discriminators. In various examples, a feature is computed using sub-volumes of the video.

Abstract: Foreground and background image segmentation is described. In an example, a seed region is selected in a foreground portion of an image, and a geodesic distance is calculated from each image element to the seed region. A subset of the image elements having a geodesic distance less than a threshold is determined, and this subset of image elements are labeled as foreground. In another example, an image element from an image showing at least a user, a foreground object in proximity to the user, and a background is applied to trained decision trees to obtain probabilities of the image element representing one of these items, and a corresponding classification assigned to the image element. This is repeated for each image element. Image elements classified as belonging to the user are labeled as foreground, and image elements classified as foreground objects or background are labeled as background.

Abstract: Medical image rendering is described. In an embodiment a medical image visualization engine receives results from an organ recognition system which provide estimated organ centers, bounding boxes and organ classification labels for a given medical image. In examples the visualization engine uses the organ recognition system results to select appropriate transfer functions, bounding regions, clipping planes and camera locations in order to optimally view an organ. For example, a rendering engine uses the selections to render a two-dimensional image of medical diagnostic quality with minimal user input. In an embodiment a graphical user interface populates a list of organs detected in a medical image and a clinician is able to select one organ and immediately be presented with the optimal view of that organ. In an example opacity of background regions of the medical image may be adjusted to provide context for organs presented in a foreground region.

Abstract: Filtering sensor data is described, for example, where filters conditioned on a local appearance of the signal are predicted by a machine learning system, and used to filter the sensor data. In various examples the sensor data is a stream of noisy video image data and the filtering process denoises the video stream. In various examples the sensor data is a depth image and the filtering process refines the depth image which may then be used for gesture recognition or other purposes. In various examples the sensor data is one dimensional measurement data from an electric motor and the filtering process denoises the measurements. In examples the machine learning system comprises a random decision forest where trees of the forest store filters at their leaves. In examples, the random decision forest is trained using a training objective with a data dependent regularization term.

Abstract: A method of sensing depth using an RGB camera. In an example method, a color image of a scene is received from an RGB camera. The color image is applied to a trained machine learning component which uses features of the image elements to assign all or some of the image elements a depth value which represents the distance between the surface depicted by the image element and the RGB camera. In various examples, the machine learning component comprises one or more entangled geodesic random decision forests.

Abstract: Methods and systems for controlling a computing-based device based on gestures made within a predetermined range of a camera wherein the predetermined range is a subset of the field of view of the camera. Any gestures made outside of the predetermined range are ignored and do not cause the computing-based device to perform any action. In some examples, the gestures are used to control a drawing canvas that is implemented in a video conference session. In these examples, a single camera may be used to generate an image of a video conference user which is used to detect gestures in the predetermined range and provide other parties to the video conference session a visual image of the user.

Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.

Abstract: Density estimation and/or manifold learning are described, for example, for computer vision, medical image analysis, text document clustering. In various embodiments a density forest is trained using unlabeled data to estimate the data distribution. In embodiments the density forest comprises a plurality of random decision trees each accumulating portions of the training data into clusters at their leaves. In embodiments probability distributions representing the clusters at each tree are aggregated to form a forest density which is an estimate of a probability density function from which the unlabeled data may be generated. A mapping engine may use the clusters at the leaves of the density forest to estimate a mapping function which maps the unlabeled data to a lower dimensional space whilst preserving relative distances or other relationships between the unlabeled data points. A sampling engine may use the density forest to randomly sample data from the forest density.

Abstract: Automatic organ localization is described. In an example, an organ in a medical image is localized using one or more trained regression trees. Each image element of the medical image is applied to the trained regression trees to compute probability distributions that relate to a distance from each image element to the organ. At least a subset of the probability distributions are selected and aggregated to calculate a localization estimate for the organ. In another example, the regression trees are trained using training images having a predefined organ location. At each node of the tree, test parameters are generated that determine which subsequent node each training image element is passed to. This is repeated until each image element reaches a leaf node of the tree. A probability distribution is generated and stored at each leaf node, based on the distance from the leaf node's image elements to the organ.

Abstract: Image labeling is described, for example, to recognize body organs in a medical image, to label body parts in a depth image of a game player, to label objects in a video of a scene. In various embodiments an automated classifier uses geodesic features of an image, and optionally other types of features, to semantically segment an image. For example, the geodesic features relate to a distance between image elements, the distance taking into account information about image content between the image elements. In some examples the automated classifier is an entangled random decision forest in which data accumulated at earlier tree levels is used to make decisions at later tree levels. In some examples the automated classifier has auto-context by comprising two or more random decision forests. In various examples parallel processing and look up procedures are used.

Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Abstract: Parallel processing for distance transforms is described. In an embodiment a raster scan algorithm is used to compute a distance transform such that each image element of a distance image is assigned a distance value. This distance value is a shortest distance from the image element to the seed region. In an embodiment two threads execute in parallel with a first thread carrying out a forward raster scan over the distance image and a second thread carrying out a backward raster scan over the image. In an example, a thread pauses when a cross-over condition is met until the other thread meets the condition after which both threads continue. In embodiments distances may be computed in Euclidean space or along geodesics defined on a surface. In an example, four threads execute two passes in parallel with each thread carrying out a raster scan over a different quarter of the image.

Abstract: Computing high dynamic range photographs is described for example, to enable high ranges of intensities to be represented in a single image. In various embodiments two or more photographs of the same scene taken at different exposure levels are combined in a way which takes into account intensity or other gradients in the images to form a high dynamic range image. In embodiments geodesic distances (which take into account intensity or other image gradients) are computed and used to form weights for a weighted aggregation of the photographs. In some embodiments a user configurable parameter is operable to control a degree of mixing of the photographs as the high dynamic range image is formed.

Abstract: Foreground and background image segmentation is described. In an example, a seed region is selected in a foreground portion of an image, and a geodesic distance is calculated from each image element to the seed region. A subset of the image elements having a geodesic distance less than a threshold is determined, and this subset of image elements are labeled as foreground. In another example, an image element from an image showing at least a user, a foreground object in proximity to the user, and a background is applied to trained decision trees to obtain probabilities of the image element representing one of these items, and a corresponding classification assigned to the image element. This is repeated for each image element. Image elements classified as belonging to the user are labeled as foreground, and image elements classified as foreground objects or background are labeled as background.

Abstract: A system and method for detecting and tracking targets including body parts and props is described. In one aspect, the disclosed technology acquires one or more depth images, generates one or more classification maps associated with one or more body parts and one or more props, tracks the one or more body parts using a skeletal tracking system, tracks the one or more props using a prop tracking system, and reports metrics regarding the one or more body parts and the one or more props. In some embodiments, feedback may occur between the skeletal tracking system and the prop tracking system.