Abstract: Systems and techniques for producing image-based radiology reports including contextual cropping of image data and radiologist supplied notes and annotations are provided herein. Computer vision and natural language processing algorithms may enable processing of image data and language inputs to identify objects associated with annotations, aid in cropping the image data according to the annotations and object identification and in producing a final text and image laden report.

Abstract: Described are systems and methods directed to generation and subsequent update of a dimensionally accurate body model of a body, such as a human body, based on two-dimensional (“2D”) images of at least a portion of that body and/or face images of a face of the body. A user may use a 2D camera, such as a digital camera typically included in many of today's portable devices (e.g., cell phones, tablets, laptops, etc.) to produce body images that are used to generate a body model of the body of the user. Subsequently, the body model may be updated based on a face image of the face of the user, without requiring the user to provide another body image.

Abstract: Described are systems and methods to determine one or more body dimensions of a body based on a processing of one or more two-dimensional images that include a representation of the body. Body dimensions include any length, circumference, etc., of any part of a body, such as shoulder circumference, chest circumference, waist circumference, hip circumference, inseam length, bicep circumference, leg circumference, etc.

Abstract: A predictor has a memory which stores at least one example for which an associated outcome is not known. The memory stores at least one decision tree comprising a plurality of nodes connected by edges, the nodes comprising a root node, internal nodes and leaf nodes. Individual ones of the nodes and individual ones of the edges each have an assigned module, comprising parameterized, differentiable operations, such that for each of the internal nodes the module computes a binary outcome for selecting a child node of the internal node. The predictor has a processor configured to compute the prediction by processing the example using a plurality of the differentiable operations selected according to a path through the tree from the root node to a leaf node.

Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Abstract: A predictor has a memory which stores at least one example for which an associated outcome is not known. The memory stores at least one decision tree comprising a plurality of nodes connected by edges, the nodes comprising a root node, internal nodes and leaf nodes. Individual ones of the nodes and individual ones of the edges each have an assigned module, comprising parameterized, differentiable operations, such that for each of the internal nodes the module computes a binary outcome for selecting a child node of the internal node. The predictor has a processor configured to compute the prediction by processing the example using a plurality of the differentiable operations selected according to a path through the tree from the root node to a leaf node.

Abstract: Described are systems and methods to determine one or more body dimensions of a body based on a processing of one or more two-dimensional images that include a representation of the body. Body dimensions include any length, circumference, etc., of any part of a body, such as shoulder circumference, chest circumference, waist circumference, hip circumference, inseam length, bicep circumference, leg circumference, etc.

Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.

Abstract: A predictor has a memory which stores at least one example for which an associated outcome is not known. The memory stores at least one decision tree comprising a plurality of nodes connected by edges, the nodes comprising a root node, internal nodes and leaf nodes. Individual ones of the nodes and individual ones of the edges each have an assigned module, comprising parameterized, differentiable operations, such that for each of the internal nodes the module computes a binary outcome for selecting a child node of the internal node. The predictor has a processor configured to compute the prediction by processing the example using a plurality of the differentiable operations selected according to a path through the tree from the root node to a leaf node.

Abstract: Image labeling is described, for example, to recognize body organs in a medical image, to label body parts in a depth image of a game player, to label objects in a video of a scene. In various embodiments an automated classifier uses geodesic features of an image, and optionally other types of features, to semantically segment an image. For example, the geodesic features relate to a distance between image elements, the distance taking into account information about image content between the image elements. In some examples the automated classifier is an entangled random decision forest in which data accumulated at earlier tree levels is used to make decisions at later tree levels. In some examples the automated classifier has auto-context by comprising two or more random decision forests. In various examples parallel processing and look up procedures are used.

Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Abstract: A sensor data processor is described comprising a memory storing a plurality of trained expert models. The machine learning system has a processor configured to receive an unseen sensor data example and, for each trained expert model, compute a prediction from the unseen sensor data example using the trained expert model. The processor is configured to aggregate the predictions to form an aggregated prediction, receive feedback about the aggregated prediction and update, for each trained expert, a weight associated with that trained expert, using the received feedback. The processor is configured to compute a second aggregated prediction by computing an aggregation of the predictions which takes into account the weights.

Abstract: Video processing for motor task analysis is described. In various examples, a video of at least part of a person or animal carrying out a motor task, such as placing the forefinger on the nose, is input to a trained machine learning system to classify the motor task into one of a plurality of classes. In an example, motion descriptors such as optical flow are computed from pairs of frames of the video and the motion descriptors are input to the machine learning system. For example, during training the machine learning system identifies time-dependent and/or location-dependent acceleration or velocity features which discriminate between the classes of the motor task. In examples, the trained machine learning system computes, from the motion descriptors, the location dependent acceleration or velocity features which it has learned as being good discriminators. In various examples, a feature is computed using sub-volumes of the video.

Abstract: A method of training a random decision tree to give improved generalization ability is described. At a split node of the random decision tree a plurality of training sensor data elements available at the split node are divided into a tuning set and a validation set. A plurality of models is formed using the tuning set, each model using different values of parameters of the split node. Performance of the models at splitting the validation set between left and right child nodes of the split node is computed and used to select one of the models.

Abstract: A machine learning system is described which has a memory storing at least one trained random decision tree and parameters of a plurality of clusters associated with the trained random decision tree. A processor of the machine learning system pushes a sensor data element through the trained random decision tree to compute a prediction and to obtain values of features associated with the sensor data element. The processor selects one of the clusters by comparing the features associated with the received sensor data element and the parameters of the clusters. The memory stores at least one cluster-specific random decision tree, which has been trained using data from the selected cluster. The processor is configured to push the prediction through the cluster-specific random decision tree to compute another prediction. The clusters group together sensor data elements which give rise to similar pathways when pushed through the trained random decision tree.

Abstract: Methods and systems for controlling a computing-based device based on gestures made within a predetermined range of a camera wherein the predetermined range is a subset of the field of view of the camera. Any gestures made outside of the predetermined range are ignored and do not cause the computing-based device to perform any action. In some examples, the gestures are used to control a drawing canvas that is implemented in a video conference session. In these examples, a single camera may be used to generate an image of a video conference user which is used to detect gestures in the predetermined range and provide other parties to the video conference session a visual image of the user.

Abstract: A training engine is described which has a memory arranged to access a neural network image classifier, the neural network image classifier having been trained using a plurality of training images from an input space, the training images being labeled for a plurality of classes. The training engine has an adversarial example generator which computes a plurality of adversarial images by, for each adversarial image, searching a region in the input space around one of the training images, the region being one in which the neural network is linear, to find an image which is incorrectly classified into the plurality of classes by the neural network. The training engine has a processor which further trains the neural network image classifier using at least the adversarial images.

Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Abstract: A training engine is described which has a memory arranged to access a neural network image classifier, the neural network image classifier having been trained using a plurality of training images from an input space, the training images being labeled for a plurality of classes. The training engine has an adversarial example generator which computes a plurality of adversarial images by, for each adversarial image, searching a region in the input space around one of the training images, the region being one in which the neural network is linear, to find an image which is incorrectly classified into the plurality of classes by the neural network. The training engine has a processor which further trains the neural network image classifier using at least the adversarial images.

Abstract: Filtering sensor data is described, for example, where filters conditioned on a local appearance of the signal are predicted by a machine learning system, and used to filter the sensor data. In various examples the sensor data is a stream of noisy video image data and the filtering process denoises the video stream. In various examples the sensor data is a depth image and the filtering process refines the depth image which may then be used for gesture recognition or other purposes. In various examples the sensor data is one dimensional measurement data from an electric motor and the filtering process denoises the measurements. In examples the machine learning system comprises a random decision forest where trees of the forest store filters at their leaves. In examples, the random decision forest is trained using a training objective with a data dependent regularization term.