Patents Assigned to DeepMap Inc.
  • Publication number: 20210349923
    Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.
    Type: Application
    Filed: July 20, 2021
    Publication date: November 11, 2021
    Applicant: DEEPMAP INC.
    Inventor: Mark Damon WHEELER
  • Patent number: 11068516
    Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.
    Type: Grant
    Filed: February 10, 2020
    Date of Patent: July 20, 2021
    Assignee: DEEPMAP INC.
    Inventor: Mark Damon Wheeler
  • Patent number: 11041730
    Abstract: The autonomous vehicle generates an overlapped image by overlaying HD map data over sensor data and rendering the overlaid images. The visualization process is repeated as the vehicle drives along the route. The visualization may be displayed on a screen within the vehicle or at a remote device. The system performs reverse rendering of a scene based on map data from a selected point. For each line of sight originating at the selected point, the system identifies the farthest object in the map data. Accordingly, the system eliminates objects obstructing the view of the farthest objects in the HD map as viewed from the selected point. The system further allows filtering of objects using filtering criteria based on semantic labels. The system generates a view from the selected point such that 3D objects matching the filtering criteria are eliminated from the view.
    Type: Grant
    Filed: March 1, 2019
    Date of Patent: June 22, 2021
    Assignee: DEEPMAP INC.
    Inventors: Gil Colgate, Mark Damon Wheeler
  • Patent number: 10999511
    Abstract: A method and system for synchronizing a lidar and a camera on an autonomous vehicle. The system instructs the camera to detect light columns transmitted by the lidar. The system iterates through various start times for the camera. The system instructs the lidar to emit a plurality of light columns at a lidar frequency. The system instructs the camera to capture images at a camera frequency starting at each start time. The system analyzes the image data received from the cameras to identify light columns captured in the images. The system calculates an alignment score for each of the many start times based on the identified light columns. The start time with the optimal alignment score is selected and used to synchronize the lidar and the camera. With lidar data detected by the synchronized lidar and image data captured by the synchronized camera, the system may navigate the autonomous vehicle.
    Type: Grant
    Filed: September 24, 2019
    Date of Patent: May 4, 2021
    Assignee: DEEPMAP INC.
    Inventors: Lin Yang, Mark Damon Wheeler
  • Patent number: 10962366
    Abstract: As an autonomous vehicle moves through a local area, pairwise alignment may be performed to calculate changes in the pose of the vehicle between different points in time. The vehicle comprises an imaging system configured to capture image frames depicting a portion of the surrounding area. Features are identified from the captured image frames, and a 3-D location is determined for each identified feature. The features of different image frames corresponding to different points in time are analyzed to determine a transformation in the pose of the vehicle during the time period between the image frames. The determined poses of the vehicle are used to generate an HD map of the local area.
    Type: Grant
    Filed: February 11, 2020
    Date of Patent: March 30, 2021
    Assignee: DEEPMAP INC.
    Inventors: Ronghua Zhang, Chen Chen, Di Zeng, Mark Damon Wheeler
  • Patent number: 10928523
    Abstract: A vehicle, for example, an autonomous vehicle receives signals from a global navigation satellite system (GNSS) and determines accurate location of the vehicle using the GNSS signal. The vehicle performs localization to determine the location of the vehicle as it drives. The autonomous vehicle uses sensor data and a high definition map to determine an accurate location of the autonomous vehicle. The autonomous vehicle uses accurate location of the vehicle to determine RTK corrections that is used for improving GNSS location estimates at a future location. The RTK corrections may be transmitted to other vehicles.
    Type: Grant
    Filed: January 6, 2020
    Date of Patent: February 23, 2021
    Assignee: DEEPMAP INC.
    Inventor: Jeffrey Minoru Adachi
  • Patent number: 10928207
    Abstract: Camera based localization performed to determine a current pose of an autonomous vehicle without the aid of depth sensors such as LiDAR. The vehicle comprises an imaging system configured to capture image frames depicting portions of the surrounding area. Based on an initial pose of the vehicle, edgels corresponding to three-dimensional locations are loaded and mapped to corresponding edge pixels of the captured image frame. A pose of the vehicle is optimized based upon the determined correspondences by identifying a transformation that minimizes a distance between the edgels and their corresponding edge pixels. The determined transformation can be applied to the initial pose to determine an updated pose of the vehicle.
    Type: Grant
    Filed: March 1, 2019
    Date of Patent: February 23, 2021
    Assignee: DEEPMAP INC.
    Inventors: Ronghua Zhang, Lin Yang
  • Patent number: 10897575
    Abstract: A system performs calibration of sensors mounted on a vehicle, for example, lidar and camera sensors mounted on a vehicle, for example, an autonomous vehicle. The system receives a lidar scan and camera image of a view and determines a lidar-to-camera transform based on the lidar scan and the camera image. The system may use a pattern, for example, a checkerboard pattern in the view for calibration. The pattern is placed close to the vehicle to determine an approximate lidar-to-camera transform and then placed at a distance from the vehicle to determine an accurate lidar-to-camera transform. Alternatively, the system determines edges in the lidar scan and the camera image and aligns features based on real-world objects in the scene by comparing edges.
    Type: Grant
    Filed: January 2, 2020
    Date of Patent: January 19, 2021
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Lin Yang
  • Patent number: 10859395
    Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: December 8, 2020
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Lin Yang, Dongzhen Piao, Yu Zhang
  • Patent number: 10845820
    Abstract: A system generates a high definition map for an autonomous vehicle to travel from a source location to a destination location. The system determines a low resolution route and receives high definition map data for a set of geographical regions overlaying the low resolution route. The system uses lane elements within the geographical regions to form a set of potential partial routes. The system calculates the error between the potential partial route and the low resolution route and removes potential partial routes with errors above the threshold. Once completed, the system selects a final route and sends signals to the controls of the autonomous vehicle to follow the final route. The system determines whether surface areas adjacent to a lane that are not part of the road are safe for the vehicle to drive in case of emergency. The system stores information describing navigable surface areas with representations of lanes.
    Type: Grant
    Filed: December 22, 2017
    Date of Patent: November 24, 2020
    Assignee: DEEPMAP INC.
    Inventor: Mark Damon Wheeler
  • Patent number: 10841496
    Abstract: A system performs calibration of sensors mounted on a vehicle, for example, lidar and camera sensors mounted on a vehicle, for example, an autonomous vehicle. The system receives a lidar scan and camera image of a view and determines a lidar-to-camera transform based on the lidar scan and the camera image. The system may use a pattern, for example, a checkerboard pattern in the view for calibration. The pattern is placed close to the vehicle to determine an approximate lidar-to-camera transform and then placed at a distance from the vehicle to determine an accurate lidar-to-camera transform. Alternatively, the system determines edges in the lidar scan and the camera image and aligns features based on real-world objects in the scene by comparing edges.
    Type: Grant
    Filed: October 15, 2018
    Date of Patent: November 17, 2020
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Lin Yang
  • Patent number: 10837773
    Abstract: A vehicle computing system performs enhances relatively sparse data collected by a LiDAR sensor by increasing the density of points in certain portions of the scan. For instance, the system generates 3D triangles based on a point cloud collected by the LiDAR sensor and filters the 3D triangles to identify a subset of 3D triangles that are proximate to the ground. The system interpolates points within the subset of 3D triangles to identify additional points on the ground. As another example, the system uses data collected by the LiDAR sensor to identify vertical structures and interpolate additional points on those vertical structures. The enhanced data can be used for a variety of applications related to autonomous vehicle navigation and HD map generation, such as detecting lane markings on the road in front of the vehicle or determining a change in the vehicle's position and orientation.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: November 17, 2020
    Assignee: DEEPMAP INC.
    Inventor: Lin Yang
  • Patent number: 10816346
    Abstract: An online system builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region. The autonomous vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system updates existing occupancy maps to improve the accuracy of the occupancy maps (OMaps), and to thereby improve passenger and pedestrian safety. While vehicles are in motion, they can continuously collect data about their surroundings. When new data is available from the various vehicles within a fleet, this can be updated in a local representation of the occupancy map and can be passed to the online HD map system (e.g., in the cloud) for updating the master occupancy map shared by all of the vehicles.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: October 27, 2020
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Xiaqing Wu
  • Patent number: 10801845
    Abstract: An online system builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region. The autonomous vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system ranks the autonomous vehicles based on factors including an upload rate indicating how often the vehicle was used providing data to the online system. The sensor data from vehicles is uploaded to the online system (e.g., in the cloud) to create the HD map while spreading the burden of uploading this data as evenly as possible across a fleet of vehicles. Data uploads are expensive and time consuming, so the system makes this negligible for each vehicle by balancing/managing the uploads carefully across the fleet.
    Type: Grant
    Filed: August 17, 2019
    Date of Patent: October 13, 2020
    Assignee: DEEPMAP INC.
    Inventor: Mark Damon Wheeler
  • Patent number: 10794711
    Abstract: An online system build a high definition map for geographical regions based on sensor data captured by a plurality of vehicles driving through the geographical regions. The high definition map comprises landmark map including representations of driving paths and landmarks and occupancy map comprising spatial 3D representation of the road and all physical objects around the road. The vehicles provide information to update the landmark map and the occupancy map to the online system. The vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system ranks the vehicles based on factors including an upload rate of the vehicle and requests data describing high definition map from vehicles selected based on the ranking.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: October 6, 2020
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Xiaqing James Wu, Wei Luo
  • Publication number: 20200284590
    Abstract: According to an aspect of an embodiment, operations may comprise obtaining a pose graph that comprises a plurality of nodes. The operations may also comprise dividing the pose graph into a plurality of pose subgraphs, each pose subgraph comprising one or more respective pose subgraph interior nodes and one or more respective pose subgraph boundary nodes. The operations may also comprise generating one or more boundary subgraphs based on the plurality of pose subgraphs, each of the one or more boundary subgraphs comprising one or more respective boundary subgraph boundary nodes and comprising one or more respective boundary subgraph interior nodes. The operations may also comprise obtaining an optimized pose graph by performing a pose graph optimization. The pose graph optimization may comprise performing a pose subgraph optimization of the plurality of pose subgraphs and performing a boundary subgraph optimization of the plurality of boundary subgraphs.
    Type: Application
    Filed: March 5, 2020
    Publication date: September 10, 2020
    Applicant: DEEPMAP INC.
    Inventor: Chen Chen
  • Patent number: 10670416
    Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: June 2, 2020
    Assignee: DEEPMAP INC.
    Inventors: Mark Damon Wheeler, Lin Yang, Derek Thomas Miller, Yu Zhang, Lenord Melvix Joseph Stephen Max
  • Patent number: 10598489
    Abstract: As an autonomous vehicle moves through a local area, pairwise alignment may be performed to calculate changes in the pose of the vehicle between different points in time. The vehicle comprises an imaging system configured to capture image frames depicting a portion of the surrounding area. Features are identified from the captured image frames, and a 3-D location is determined for each identified feature. The features of different image frames corresponding to different points in time are analyzed to determine a transformation in the pose of the vehicle during the time period between the image frames. The determined poses of the vehicle are used to generate an HD map of the local area.
    Type: Grant
    Filed: May 30, 2019
    Date of Patent: March 24, 2020
    Assignee: DeepMap Inc.
    Inventors: Ronghua Zhang, Chen Chen, Di Zeng, Mark Damon Wheeler
  • Patent number: 10572514
    Abstract: High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.
    Type: Grant
    Filed: May 9, 2019
    Date of Patent: February 25, 2020
    Assignee: DeepMap Inc.
    Inventor: Mark Damon Wheeler
  • Patent number: 10545029
    Abstract: An HD map system represents landmarks on a high definition map for autonomous vehicle navigation, including describing spatial location of lanes of a road and semantic information about each lane, and along with traffic signs and landmarks. The system generates lane lines designating lanes of roads based on, for example, mapping of camera image pixels with high probability of being on lane lines into a three-dimensional space, and locating/connecting center lines of the lane lines. The system builds a large connected network of lane elements and their connections as a lane element graph. The system also represents traffic signs based on camera images and detection and ranging sensor depth maps. These landmarks are used in building a high definition map that allows autonomous vehicles to safely navigate through their environments.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: January 28, 2020
    Assignee: DeepMap Inc.
    Inventors: Lin Yang, Ming Cui