Abstract: Systems and methods for categorizing an object captured in an image are disclosed. An example method includes providing a neural network configured to receive the image and to provide a corresponding output. The method additionally includes defining a plurality of known object classes, each corresponding to a real-world object class and being defined by a class-specific subset of visual features identified by the neural network. The method includes acquiring a first two-dimensional (2-D) image including a first object and providing the first 2-D image to the neural network. The neural network identifies a particular subset of the visual features corresponding to the first object in the first 2-D image. The method also includes identifying a first known object class most likely to include the first object, and identifying a second known object class that is next likeliest to include the first object.

Abstract: A system for training a machine learning framework to estimate depths of objects captured in 2-D images includes a first trained machine learning network and a second untrained or minimally trained machine learning framework. The first trained machine learning network is configured to analyze 2-D images of target spaces including target objects and to provide output indicative of 3-D positions of the target objects in the target spaces. The second machine learning network can be configured to provide an output responsive to receiving a 2-D input image. A comparator receives the outputs from the first and second machine learning networks based on a particular 2-D image. The comparator compares the output of the first trained machine learning network with the output of the second machine learning network. A feedback mechanism is operative to alter the second machine learning network based at least in part on the output of the comparator.

Abstract: Multi-object tracking in autonomous vehicles uses both camera data and LiDAR data for training, but not LiDAR data at query time. Thus, no LiDAR sensor is on a piloted autonomous vehicle. Example systems and methods rely on camera 2D object detections alone, rather than 3D annotations. Example systems/methods utilize a single network that is given a camera image as input and can learn both object detection and dense depth in a multimodal regression setting, where the ground truth LiDAR data is used only at training time to compute depth regression loss. The network uses the camera image alone as input at test time (i.e., when deployed for piloting an autonomous vehicle) and can predict both object detections and dense depth of the scene. LiDAR is only used for data acquisition and is not required for drawing 3D annotations or for piloting the vehicle.