Abstract: A system and method for large-scale lane marking detection using multimodal sensor data are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on a vehicle; receiving point cloud data from a distance and intensity measuring device mounted on the vehicle; fusing the image data and the point cloud data to produce a set of lane marking points in three-dimensional (3D) space that correlate to the image data and the point cloud data; and generating a lane marking map from the set of lane marking points.

Abstract: A data-driven prediction-based system and method for trajectory planning of autonomous vehicles are disclosed. A particular embodiment includes: generating a first suggested trajectory for an autonomous vehicle; generating predicted resulting trajectories of proximate agents using a prediction module; scoring the first suggested trajectory based on the predicted resulting trajectories of the proximate agents; generating a second suggested trajectory for the autonomous vehicle and generating corresponding predicted resulting trajectories of proximate agents, if the score of the first suggested trajectory is below a minimum acceptable threshold; and outputting a suggested trajectory for the autonomous vehicle wherein the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold.

Abstract: A system and method for determining car to lane distance is provided. In one aspect, the system includes a camera configured to generate an image, a processor, and a computer-readable memory. The processor is configured to receive the image from the camera, generate a wheel segmentation map representative of one or more wheels detected in the image, and generate a lane segmentation map representative of one or more lanes detected in the image. For at least one of the wheels in the wheel segmentation map, the processor is also configured to determine a distance between the wheel and at least one nearby lane in the lane segmentation map. The processor is further configured to determine a distance between a vehicle in the image and the lane based on the distance between the wheel and the lane.

Abstract: A method of localization for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform by one or more autonomous vehicle driving modules execution of processing of images from a camera and data from a LiDAR using the following steps comprising: computing, in response to features from a 3D submap and features from a global map, matching score between corresponding features of a same class between the 3D submap and the global map; selecting, for each feature in the 3D submap, a corresponding feature with the highest matching score from the global map; determining a feature correspondence to be invalid if a distance between corresponding features is larger than a threshold; and removing the invalid feature correspondence.

Abstract: Disclosed are devices, systems and methods for incorporating a smoothness constraint for camera pose estimation. One method for robust camera pose estimation includes determining a first bounding box based on a previous frame, determining a second bounding box based on a current frame that is temporally subsequent to the previous frame, estimating the camera pose by minimizing a weighted sum of a camera pose function and a constraint function, where the camera pose function tracks a position and an orientation of the camera in time, and where the constraint function is based on coordinates of the first bounding box and coordinates of the second bounding box, and using the camera pose for navigating the vehicle. The method may further include generating an initial estimate of the camera pose is based on a Global Positioning System (GPS) sensor or an Inertial Measurement Unit (IMU).

Abstract: A system and method for image localization based on semantic segmentation are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on an autonomous vehicle; performing semantic segmentation or other object detection on the received image data to identify and label objects in the image data and produce semantic label image data; identifying extraneous objects in the semantic label image data; removing the extraneous objects from the semantic label image data; comparing the semantic label image data to a baseline semantic label map; and determining a vehicle location of the autonomous vehicle based on information in a matching baseline semantic label map.

Abstract: A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs includes instructions, which when executed by a computing device, causes the computing device to perform the following steps comprising: performing data alignment among sensors including a LiDAR, cameras and an IMU-GPS module; collecting image data and generating point clouds; processing, in the IMU-GPS module, a pair of consecutive images in the image data to recognize pixels corresponding to a same point in the point clouds; and establishing an optical flow for visual odometry.

Abstract: A vehicle position and velocity estimation based on camera and LIDAR data are disclosed.

Abstract: A system and method for large-scale lane marking detection using multimodal sensor data are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on a vehicle; receiving point cloud data from a distance and intensity measuring device mounted on the vehicle; fusing the image data and the point cloud data to produce a set of lane marking points in three-dimensional (3D) space that correlate to the image data and the point cloud data; and generating a lane marking map from the set of lane marking points.

Abstract: A system and method for drivable road surface representation generation using multimodal sensor data are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on a vehicle and receiving three dimensional (3D) point cloud data from a distance measuring device mounted on the vehicle; projecting the 3D point cloud data onto the 2D image data to produce mapped image and point cloud data; performing post-processing operations on the mapped image and point cloud data; and performing a smoothing operation on the processed mapped image and point cloud data to produce a drivable road surface map or representation.

Abstract: A system and method for adaptive cruise control for defensive driving are disclosed. A particular embodiment includes: receiving input object data from a subsystem of an autonomous vehicle, the input object data including distance data and velocity data relative to a lead vehicle; generating a weighted distance differential corresponding to a weighted difference between an actual distance between the autonomous vehicle and the lead vehicle and a desired distance between the autonomous vehicle and the lead vehicle; generating a weighted velocity differential corresponding to a weighted difference between a velocity of the autonomous vehicle and a velocity of the lead vehicle; combining the weighted distance differential and the weighted velocity differential with the velocity of the lead vehicle to produce a velocity command for the autonomous vehicle; and controlling the autonomous vehicle to conform to the velocity command.

Abstract: A method of generating a ground truth dataset for motion planning is disclosed. The method includes performing data alignment, collecting data in an environment, using sensors, calculating, among other sensors, light detecting and ranging (LiDAR)'s poses, stitching multiple LiDAR scans to form a local map, refining positions in the local map based on a matching algorithm, and projecting 3D points in the local map onto corresponding images.

Abstract: A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: receiving perception data from a plurality of sensors of an autonomous vehicle; configuring the perception simulation operation based on a comparison of the perception data against ground truth data; generating simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle.

Abstract: A method of generating a ground truth dataset for motion planning of a vehicle is disclosed. The method includes: obtaining undistorted LiDAR scans; identifying, for a pair of undistorted LiDAR scans, points belonging to a static object in an environment; aligning the close points based on pose estimates; and transforming a reference scan that is close in time to a target undistorted LiDAR scan so as to align the reference scan with the target undistorted LiDAR scan.

Abstract: A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs include instructions, which when executed by a computing device, cause the computing device to perform the following steps comprising: receiving a lane marking, expressed in god's view, associated with a current view; fitting, in a post-processing module, the lane marking in an arc by using a set of parameters; generating a lane template, using the set of parameters, the lane template including features of the lane marking associated with the current view and features of the arc; and feeding the lane template associated with the current view for detection of a next view.

Abstract: A system and method for large scale distributed simulation for realistic multiple-agent interactive environments are disclosed. A particular embodiment includes: generating a vicinal scenario for each simulated vehicle in an iteration of a simulation, the vicinal scenarios corresponding to different locations, traffic patterns, or environmental conditions being simulated; assigning a processing task to at least one of a plurality of distributed computing devices to generate vehicle trajectories for each of a plurality of simulated vehicles of the simulation based on the vicinal scenario; and updating a state and trajectory of each of the plurality of simulated vehicles based on processed data received from the plurality of distributed computing devices.

Abstract: A system and method for transitioning between an autonomous and manual driving mode based on detection of a driver's capacity to control a vehicle are disclosed. A particular embodiment includes: receiving sensor data related to a vehicle driver's capacity to take manual control of an autonomous vehicle; determining, based on the sensor data, if the driver has the capacity to take manual control of the autonomous vehicle, the determining including prompting the driver to perform an action or provide an input; and outputting a vehicle control transition signal to a vehicle subsystem to cause the vehicle subsystem to take action based on the driver's capacity to take manual control of the autonomous vehicle.

Abstract: A system and method for aerial video traffic analysis are disclosed. A particular embodiment is configured to: receive a captured video image sequence from an unmanned aerial vehicle (UAV); clip the video image sequence by removing unnecessary images; stabilize the video image sequence by choosing a reference image and adjusting other images to the reference image; extract a background image of the video image sequence for vehicle segmentation; perform vehicle segmentation to identify vehicles in the video image sequence on a pixel by pixel basis; determine a centroid, heading, and rectangular shape of each identified vehicle; perform vehicle tracking to detect a same identified vehicle in multiple image frames of the video image sequence; and produce output and visualization of the video image sequence including a combination of the background image and the images of each identified vehicle.

Abstract: A system method for detecting taillight signals of a vehicle using a convolutional neural network is disclosed.