Abstract: The present disclosure relates to systems and methods that provide both an image of a scene and depth information for the scene. An example system includes at least one time-of-flight (ToF) sensor and an imaging sensor. The ToF sensor and the imaging sensor are configured to receive light from a scene. The system also includes at least one light source and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with illumination light according to an illumination schedule. The operations also include causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the illumination light. The operations additionally include causing the imaging sensor to provide information indicative of an image of the scene based on the illumination light.

Abstract: Disclosed herein are neural networks for generating target classifications for an object from a set of input sequences. Each input sequence includes a respective input at each of multiple time steps, and each input sequence corresponds to a different sensing subsystem of multiple sensing subsystems. For each time step in the multiple time steps and for each input sequence in the set of input sequences, a respective feature representation is generated for the input sequence by processing the respective input from the input sequence at the time step using a respective encoder recurrent neural network (RNN) subsystem for the sensing subsystem that corresponds to the input sequence. For each time step in at least a subset of the multiple time steps, the respective feature representations are processed using a classification neural network subsystem to select a respective target classification for the object at the time step.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.

Abstract: The present disclosure relates to systems and methods that provide both an image of a scene and depth information for the scene. An example system includes at least one time-of-flight (ToF) sensor and an imaging sensor. The ToF sensor and the imaging sensor are configured to receive light from a scene. The system also includes at least one light source and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with illumination light according to an illumination schedule. The operations also include causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the illumination light. The operations additionally include causing the imaging sensor to provide information indicative of an image of the scene based on the illumination light.

Abstract: The present disclosure relates to systems and methods that provide both an image of a scene and depth information for the scene. An example system includes at least one time-of-flight (ToF) sensor and an imaging sensor. The ToF sensor and the imaging sensor are configured to receive light from a scene. The system also includes at least one light source and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with illumination light according to an illumination schedule. The operations also include causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the illumination light. The operations additionally include causing the imaging sensor to provide information indicative of an image of the scene based on the illumination light.

Abstract: The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

Abstract: The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.

Abstract: Disclosed herein are neural networks for generating target classifications for an object from a set of input sequences. Each input sequence includes a respective input at each of multiple time steps, and each input sequence corresponds to a different sensing subsystem of multiple sensing subsystems. For each time step in the multiple time steps and for each input sequence in the set of input sequences, a respective feature representation is generated for the input sequence by processing the respective input from the input sequence at the time step using a respective encoder recurrent neural network (RNN) subsystem for the sensing subsystem that corresponds to the input sequence. For each time step in at least a subset of the multiple time steps, the respective feature representations are processed using a classification neural network subsystem to select a respective target classification for the object at the time step.

Abstract: The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

Abstract: The present disclosure relates to systems and methods that provide both an image of a scene and depth information for the scene. An example system includes at least one time-of-flight (ToF) sensor and an imaging sensor. The ToF sensor and the imaging sensor are configured to receive light from a scene. The system also includes at least one light source and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with illumination light according to an illumination schedule. The operations also include causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the illumination light. The operations additionally include causing the imaging sensor to provide information indicative of an image of the scene based on the illumination light.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object detection predictions from a neural network. In some implementations, an input characterizing a first region of an environment is obtained. The input includes a projected laser image generated from a three-dimensional laser sensor reading of the first region, a camera image patch generated from a camera image of the first region, and a feature vector of features characterizing the first region. The input is processed using a high precision object detection neural network to generate a respective object score for each object category in a first set of one or more object categories. Each object score represents a respective likelihood that an object belonging to the object category is located in the first region of the environment.