Patents Assigned to DeepMap Inc.
-
Publication number: 20210349923Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.Type: ApplicationFiled: July 20, 2021Publication date: November 11, 2021Applicant: DEEPMAP INC.Inventor: Mark Damon WHEELER
-
Patent number: 11068516Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.Type: GrantFiled: February 10, 2020Date of Patent: July 20, 2021Assignee: DEEPMAP INC.Inventor: Mark Damon Wheeler
-
Patent number: 11041730Abstract: The autonomous vehicle generates an overlapped image by overlaying HD map data over sensor data and rendering the overlaid images. The visualization process is repeated as the vehicle drives along the route. The visualization may be displayed on a screen within the vehicle or at a remote device. The system performs reverse rendering of a scene based on map data from a selected point. For each line of sight originating at the selected point, the system identifies the farthest object in the map data. Accordingly, the system eliminates objects obstructing the view of the farthest objects in the HD map as viewed from the selected point. The system further allows filtering of objects using filtering criteria based on semantic labels. The system generates a view from the selected point such that 3D objects matching the filtering criteria are eliminated from the view.Type: GrantFiled: March 1, 2019Date of Patent: June 22, 2021Assignee: DEEPMAP INC.Inventors: Gil Colgate, Mark Damon Wheeler
-
Patent number: 10999511Abstract: A method and system for synchronizing a lidar and a camera on an autonomous vehicle. The system instructs the camera to detect light columns transmitted by the lidar. The system iterates through various start times for the camera. The system instructs the lidar to emit a plurality of light columns at a lidar frequency. The system instructs the camera to capture images at a camera frequency starting at each start time. The system analyzes the image data received from the cameras to identify light columns captured in the images. The system calculates an alignment score for each of the many start times based on the identified light columns. The start time with the optimal alignment score is selected and used to synchronize the lidar and the camera. With lidar data detected by the synchronized lidar and image data captured by the synchronized camera, the system may navigate the autonomous vehicle.Type: GrantFiled: September 24, 2019Date of Patent: May 4, 2021Assignee: DEEPMAP INC.Inventors: Lin Yang, Mark Damon Wheeler
-
Patent number: 10962366Abstract: As an autonomous vehicle moves through a local area, pairwise alignment may be performed to calculate changes in the pose of the vehicle between different points in time. The vehicle comprises an imaging system configured to capture image frames depicting a portion of the surrounding area. Features are identified from the captured image frames, and a 3-D location is determined for each identified feature. The features of different image frames corresponding to different points in time are analyzed to determine a transformation in the pose of the vehicle during the time period between the image frames. The determined poses of the vehicle are used to generate an HD map of the local area.Type: GrantFiled: February 11, 2020Date of Patent: March 30, 2021Assignee: DEEPMAP INC.Inventors: Ronghua Zhang, Chen Chen, Di Zeng, Mark Damon Wheeler
-
Patent number: 10928523Abstract: A vehicle, for example, an autonomous vehicle receives signals from a global navigation satellite system (GNSS) and determines accurate location of the vehicle using the GNSS signal. The vehicle performs localization to determine the location of the vehicle as it drives. The autonomous vehicle uses sensor data and a high definition map to determine an accurate location of the autonomous vehicle. The autonomous vehicle uses accurate location of the vehicle to determine RTK corrections that is used for improving GNSS location estimates at a future location. The RTK corrections may be transmitted to other vehicles.Type: GrantFiled: January 6, 2020Date of Patent: February 23, 2021Assignee: DEEPMAP INC.Inventor: Jeffrey Minoru Adachi
-
Patent number: 10928207Abstract: Camera based localization performed to determine a current pose of an autonomous vehicle without the aid of depth sensors such as LiDAR. The vehicle comprises an imaging system configured to capture image frames depicting portions of the surrounding area. Based on an initial pose of the vehicle, edgels corresponding to three-dimensional locations are loaded and mapped to corresponding edge pixels of the captured image frame. A pose of the vehicle is optimized based upon the determined correspondences by identifying a transformation that minimizes a distance between the edgels and their corresponding edge pixels. The determined transformation can be applied to the initial pose to determine an updated pose of the vehicle.Type: GrantFiled: March 1, 2019Date of Patent: February 23, 2021Assignee: DEEPMAP INC.Inventors: Ronghua Zhang, Lin Yang
-
Patent number: 10897575Abstract: A system performs calibration of sensors mounted on a vehicle, for example, lidar and camera sensors mounted on a vehicle, for example, an autonomous vehicle. The system receives a lidar scan and camera image of a view and determines a lidar-to-camera transform based on the lidar scan and the camera image. The system may use a pattern, for example, a checkerboard pattern in the view for calibration. The pattern is placed close to the vehicle to determine an approximate lidar-to-camera transform and then placed at a distance from the vehicle to determine an accurate lidar-to-camera transform. Alternatively, the system determines edges in the lidar scan and the camera image and aligns features based on real-world objects in the scene by comparing edges.Type: GrantFiled: January 2, 2020Date of Patent: January 19, 2021Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Lin Yang
-
Patent number: 10859395Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.Type: GrantFiled: December 29, 2017Date of Patent: December 8, 2020Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Lin Yang, Dongzhen Piao, Yu Zhang
-
Patent number: 10845820Abstract: A system generates a high definition map for an autonomous vehicle to travel from a source location to a destination location. The system determines a low resolution route and receives high definition map data for a set of geographical regions overlaying the low resolution route. The system uses lane elements within the geographical regions to form a set of potential partial routes. The system calculates the error between the potential partial route and the low resolution route and removes potential partial routes with errors above the threshold. Once completed, the system selects a final route and sends signals to the controls of the autonomous vehicle to follow the final route. The system determines whether surface areas adjacent to a lane that are not part of the road are safe for the vehicle to drive in case of emergency. The system stores information describing navigable surface areas with representations of lanes.Type: GrantFiled: December 22, 2017Date of Patent: November 24, 2020Assignee: DEEPMAP INC.Inventor: Mark Damon Wheeler
-
Patent number: 10841496Abstract: A system performs calibration of sensors mounted on a vehicle, for example, lidar and camera sensors mounted on a vehicle, for example, an autonomous vehicle. The system receives a lidar scan and camera image of a view and determines a lidar-to-camera transform based on the lidar scan and the camera image. The system may use a pattern, for example, a checkerboard pattern in the view for calibration. The pattern is placed close to the vehicle to determine an approximate lidar-to-camera transform and then placed at a distance from the vehicle to determine an accurate lidar-to-camera transform. Alternatively, the system determines edges in the lidar scan and the camera image and aligns features based on real-world objects in the scene by comparing edges.Type: GrantFiled: October 15, 2018Date of Patent: November 17, 2020Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Lin Yang
-
Patent number: 10837773Abstract: A vehicle computing system performs enhances relatively sparse data collected by a LiDAR sensor by increasing the density of points in certain portions of the scan. For instance, the system generates 3D triangles based on a point cloud collected by the LiDAR sensor and filters the 3D triangles to identify a subset of 3D triangles that are proximate to the ground. The system interpolates points within the subset of 3D triangles to identify additional points on the ground. As another example, the system uses data collected by the LiDAR sensor to identify vertical structures and interpolate additional points on those vertical structures. The enhanced data can be used for a variety of applications related to autonomous vehicle navigation and HD map generation, such as detecting lane markings on the road in front of the vehicle or determining a change in the vehicle's position and orientation.Type: GrantFiled: December 27, 2017Date of Patent: November 17, 2020Assignee: DEEPMAP INC.Inventor: Lin Yang
-
Patent number: 10816346Abstract: An online system builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region. The autonomous vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system updates existing occupancy maps to improve the accuracy of the occupancy maps (OMaps), and to thereby improve passenger and pedestrian safety. While vehicles are in motion, they can continuously collect data about their surroundings. When new data is available from the various vehicles within a fleet, this can be updated in a local representation of the occupancy map and can be passed to the online HD map system (e.g., in the cloud) for updating the master occupancy map shared by all of the vehicles.Type: GrantFiled: December 29, 2017Date of Patent: October 27, 2020Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Xiaqing Wu
-
Patent number: 10801845Abstract: An online system builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region. The autonomous vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system ranks the autonomous vehicles based on factors including an upload rate indicating how often the vehicle was used providing data to the online system. The sensor data from vehicles is uploaded to the online system (e.g., in the cloud) to create the HD map while spreading the burden of uploading this data as evenly as possible across a fleet of vehicles. Data uploads are expensive and time consuming, so the system makes this negligible for each vehicle by balancing/managing the uploads carefully across the fleet.Type: GrantFiled: August 17, 2019Date of Patent: October 13, 2020Assignee: DEEPMAP INC.Inventor: Mark Damon Wheeler
-
Patent number: 10794711Abstract: An online system build a high definition map for geographical regions based on sensor data captured by a plurality of vehicles driving through the geographical regions. The high definition map comprises landmark map including representations of driving paths and landmarks and occupancy map comprising spatial 3D representation of the road and all physical objects around the road. The vehicles provide information to update the landmark map and the occupancy map to the online system. The vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system ranks the vehicles based on factors including an upload rate of the vehicle and requests data describing high definition map from vehicles selected based on the ranking.Type: GrantFiled: December 29, 2017Date of Patent: October 6, 2020Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Xiaqing James Wu, Wei Luo
-
Publication number: 20200284590Abstract: According to an aspect of an embodiment, operations may comprise obtaining a pose graph that comprises a plurality of nodes. The operations may also comprise dividing the pose graph into a plurality of pose subgraphs, each pose subgraph comprising one or more respective pose subgraph interior nodes and one or more respective pose subgraph boundary nodes. The operations may also comprise generating one or more boundary subgraphs based on the plurality of pose subgraphs, each of the one or more boundary subgraphs comprising one or more respective boundary subgraph boundary nodes and comprising one or more respective boundary subgraph interior nodes. The operations may also comprise obtaining an optimized pose graph by performing a pose graph optimization. The pose graph optimization may comprise performing a pose subgraph optimization of the plurality of pose subgraphs and performing a boundary subgraph optimization of the plurality of boundary subgraphs.Type: ApplicationFiled: March 5, 2020Publication date: September 10, 2020Applicant: DEEPMAP INC.Inventor: Chen Chen
-
Patent number: 10670416Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.Type: GrantFiled: December 29, 2017Date of Patent: June 2, 2020Assignee: DEEPMAP INC.Inventors: Mark Damon Wheeler, Lin Yang, Derek Thomas Miller, Yu Zhang, Lenord Melvix Joseph Stephen Max
-
Patent number: 10598489Abstract: As an autonomous vehicle moves through a local area, pairwise alignment may be performed to calculate changes in the pose of the vehicle between different points in time. The vehicle comprises an imaging system configured to capture image frames depicting a portion of the surrounding area. Features are identified from the captured image frames, and a 3-D location is determined for each identified feature. The features of different image frames corresponding to different points in time are analyzed to determine a transformation in the pose of the vehicle during the time period between the image frames. The determined poses of the vehicle are used to generate an HD map of the local area.Type: GrantFiled: May 30, 2019Date of Patent: March 24, 2020Assignee: DeepMap Inc.Inventors: Ronghua Zhang, Chen Chen, Di Zeng, Mark Damon Wheeler
-
Patent number: 10572514Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.Type: GrantFiled: May 9, 2019Date of Patent: February 25, 2020Assignee: DeepMap Inc.Inventor: Mark Damon Wheeler
-
Patent number: 10545029Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.Type: GrantFiled: December 29, 2017Date of Patent: January 28, 2020Assignee: DeepMap Inc.Inventors: Lin Yang, Ming Cui