Abstract: Methods and systems for jointly estimating a pose and a shape of an object perceived by an autonomous vehicle are described. The system includes data and program code collectively defining a neural network which has been trained to jointly estimate a pose and a shape of a plurality of objects from incomplete point cloud data. The neural network includes a trained shared encoder neural network, a trained pose decoder neural network, and a trained shape decoder neural network. The method includes receiving an incomplete point cloud representation of an object, inputting the point cloud data into the trained shared encoder, outputting a code representative of the point cloud data. The method also includes generating an estimated pose and shape of the object based on the code. The pose includes at least a heading or a translation and the shape includes a denser point cloud representation of the object.

Abstract: Systems and methods for complementary control of an autonomous vehicle are disclosed. A primary controller provides a first plurality of instructions to an AV platform for operating the AV in an autonomous mode along a planned path based on sensor data from a primary sensor system and a secondary sensor system, and provides information that includes a fallback monitoring region to a complementary controller. The complementary controller receives sensor data from the secondary sensor system that includes sensed data for a fallback monitoring region, analyzes the received sensor data to determine whether a collision is imminent with an object detected in the fallback monitoring region, and cause the AV platform to initiate a collision mitigation action if a collision is determined to be imminent.

Abstract: A system receives a 3D image having multiple data points, and uses one or more filters, such as a distance filter, map filter, and/or height filter to remove certain 3D data points from the image. The system may group the data points and annotate them to identify unknown or unclassified objects within the image.

Abstract: Systems and methods for complementary control of an autonomous vehicle are disclosed. A primary controller provides a first plurality of instructions to an AV platform for operating the AV in an autonomous mode along a planned path based on sensor data from a primary sensor system and a secondary sensor system, and provides information that includes a fallback monitoring region to a complementary controller. The complementary controller receives sensor data from the secondary sensor system that includes sensed data for a fallback monitoring region, analyzes the received sensor data to determine whether a collision is imminent with an object detected in the fallback monitoring region, and cause the AV platform to initiate a collision mitigation action if a collision is determined to be imminent.

Abstract: Systems and methods for object detection. The methods include, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Abstract: Methods and systems for jointly estimating a pose and a shape of an object perceived by an autonomous vehicle are described. The system includes data and program code collectively defining a neural network which has been trained to jointly estimate a pose and a shape of a plurality of objects from incomplete point cloud data. The neural network includes a trained shared encoder neural network, a trained pose decoder neural network, and a trained shape decoder neural network. The method includes receiving an incomplete point cloud representation of an object, inputting the point cloud data into the trained shared encoder, outputting a code representative of the point cloud data. The method also includes generating an estimated pose and shape of the object based on the code. The pose includes at least a heading or a translation and the shape includes a denser point cloud representation of the object.

Abstract: Systems and methods for object detection. The methods include, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Abstract: Methods and systems for jointly estimating a pose and a shape of an object perceived by an autonomous vehicle are described. The system includes data and program code collectively defining a neural network which has been trained to jointly estimate a pose and a shape of a plurality of objects from incomplete point cloud data. The neural network includes a trained shared encoder neural network, a trained pose decoder neural network, and a trained shape decoder neural network. The method includes receiving an incomplete point cloud representation of an object, inputting the point cloud data into the trained shared encoder, outputting a code representative of the point cloud data. The method also includes generating an estimated pose and shape of the object based on the code. The pose includes at least a heading or a translation and the shape includes a denser point cloud representation of the object.

Abstract: A system detects multiple instances of an object in a digital image by receiving a two-dimensional (2D) image that includes a plurality of instances of an object in an environment. For example, the system may receive the 2D image from a camera or other sensing modality of an autonomous vehicle (AV). The system uses a first object detection network to generate a plurality of predicted object instances in the image. The system then receives a data set that comprises depth information corresponding to the plurality of instances of the object in the environment. The data set may be received, for example, from a stereo camera of an AV, and the depth information may be in the form of a disparity map. The system may use the depth information to identify an individual instance from the plurality of predicted object instances in the image.

Abstract: A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, generating a reconstruction dataset having a higher resolution than the 3D point cloud in one or more regions corresponding to the surface from the 3D point cloud, and generate a polygon mesh representation of the surface by using a fine-to-coarse hash map for building polygons at a highest resolution first followed by progressively coarser resolution polygons, using the reconstruction dataset.

Abstract: Systems and methods for object detection. The methods comprise, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Abstract: Systems and methods for complementary control of an autonomous vehicle are disclosed. A primary controller provides a first plurality of instructions to an AV platform for operating the AV in an autonomous mode along a planned path based on sensor data from a primary sensor system and a secondary sensor system, and provides information that includes a fallback monitoring region to a complementary controller. The complementary controller receives sensor data from the secondary sensor system that includes sensed data for a fallback monitoring region, analyzes the received sensor data to determine whether a collision is imminent with an object detected in the fallback monitoring region, and cause the AV platform to initiate a collision mitigation action if a collision is determined to be imminent.

Abstract: Systems and methods for object detection. The methods comprise, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Abstract: Systems and methods for operating a vehicle. The methods comprise, by a processor: obtaining a pair of stereo images captured by a stereo camera; processing the pair of stereo images to generate a disparity map comprising a plurality of pixels defined by intensity values; converting each intensity value to a 3D position in a map (each 3D position defining a location of a data point in a point cloud); performing a hierarchical decision tree classification to determine a classification for each data point in the point cloud (the classification being a foreground classification or a background classification); and using the classifications to facilitate autonomous control of the vehicle.

Abstract: Systems and methods for operating a vehicle. The methods comprise, by a processor: obtaining a pair of stereo images captured by a stereo camera; processing the pair of stereo images to generate a disparity map comprising a plurality of pixels defined by intensity values; converting each intensity value to a 3D position in a map (each 3D position defining a location of a data point in a point cloud); performing a hierarchical decision tree classification to determine a classification for each data point in the point cloud (the classification being a foreground classification or a background classification); and using the classifications to facilitate autonomous control of the vehicle.

Abstract: A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, generating a reconstruction dataset having a higher resolution than the 3D point cloud in one or more regions corresponding to the surface from the 3D point cloud, and generate a polygon mesh representation of the surface by using a fine-to-coarse hash map for building polygons at a highest resolution first followed by progressively coarser resolution polygons, using the reconstruction dataset.

Abstract: A system detects multiple instances of an object in a digital image by receiving a two-dimensional (2D) image that includes a plurality of instances of an object in an environment. For example, the system may receive the 2D image from a camera or other sensing modality of an autonomous vehicle (AV). The system uses a first object detection network to generate a plurality of predicted object instances in the image. The system then receives a data set that comprises depth information corresponding to the plurality of instances of the object in the environment. The data set may be received, for example, from a stereo camera of an AV, and the depth information may be in the form of a disparity map. The system may use the depth information to identify an individual instance from the plurality of predicted object instances in the image.

Abstract: A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, identifying and discarding one or more outliers in the 3D point cloud to generate a filtered point cloud using a Gaussian process, adding one or more additional points to the filtered point cloud to generate a reconstruction dataset, and using Poisson surface reconstruction to generate an implicit surface corresponding to the surface from the reconstruction dataset.

Abstract: A system detects multiple instances of an object in a digital image by receiving a two-dimensional (2D) image that includes a plurality of instances of an object in an environment. For example, the system may receive the 2D image from a camera or other sensing modality of an autonomous vehicle (AV). The system uses a first object detection network to generate a plurality of predicted object instances in the image. The system then receives a data set that comprises depth information corresponding to the plurality of instances of the object in the environment. The data set may be received, for example, from a stereo camera of an AV, and the depth information may be in the form of a disparity map. The system may use the depth information to identify an individual instance from the plurality of predicted object instances in the image.

Abstract: A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, identifying and discarding one or more outliers in the 3D point cloud to generate a filtered point cloud using a Gaussian process, adding one or more additional points to the filtered point cloud to generate a reconstruction dataset, and using Poisson surface reconstruction to generate an implicit surface corresponding to the surface from the reconstruction dataset.