Patents by Inventor Shiyu Song

Shiyu Song has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11493926
    Abstract: In one embodiment, a system generates a plurality of driving scenarios to train a reinforcement learning (RL) agent and replays each of the driving scenarios to train the RL agent by: applying a RL algorithm to an initial state of a driving scenario to determine a number of control actions from a number of discretized control/action options for the ADV to advance to a number of trajectory states which are based on a number of discretized trajectory state options, determining a reward prediction by the RL algorithm for each of the controls/actions, determining a judgment score for the trajectory states, and updating the RL agent based on the judgment score.
    Type: Grant
    Filed: May 15, 2019
    Date of Patent: November 8, 2022
    Assignee: BAIDU USA LLC
    Inventors: Runxin He, Jinyun Zhou, Qi Luo, Shiyu Song, Jinghao Miao, Jiangtao Hu, Yu Wang, Jiaxuan Xu, Shu Jiang
  • Patent number: 11486713
    Abstract: Embodiments of the present disclosure disclose a method and apparatus for positioning an autonomous vehicle. The method includes: matching a current point cloud projected image of a first resolution with a map of the first resolution to generate a first histogram filter based on the matching result; determining at least two first response areas in the first histogram filter based on a probability value of an element in the first histogram filter; generating a second histogram filter based on a result of matching a current point cloud projected image of a second resolution with a map of the second resolution and the at least two first response areas, the first resolution being less than the second resolution; and calculating a weighted average of probability values of target elements in the second histogram filter to determine a positioning result of the autonomous vehicle in the map of the second resolution.
    Type: Grant
    Filed: July 10, 2019
    Date of Patent: November 1, 2022
    Assignee: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
    Inventors: Hao Li, Guowei Wan, Yao Zhou, Shiyu Song, Fangfang Dong
  • Patent number: 11468690
    Abstract: In one embodiment, a system identifies a road to be navigated by an ADV, the road being captured by one or more point clouds from one or more LIDAR sensors. The system extracts road marking information of the identified road from the point clouds, the road marking information describing one or more road markings of the identified road. The system partitions the road into one or more road partitions based on the road markings. The system generates a point cloud map based on the road partitions, where the point cloud map is utilized to perceive a driving environment surrounding the ADV.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: October 11, 2022
    Assignees: BAIDU USA LLC, BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO. LTD.
    Inventors: Pengfei Yuan, Yong Xiao, Runxin He, Li Yu, Shiyu Song
  • Patent number: 11466992
    Abstract: The present disclosure provides a method, an apparatus, a computing device and a computer readable storage medium for detecting an environmental change, and relates to the field of autonomous driving. The method obtains a global map for an area and a first local map built in real time for a sub-area in the area; and determines an environmental change in the first sub-area by comparing the first local map and the global map and determining a first probability of the environmental change. Techniques of the present disclosure can automatically detect environmental changes that affect the positioning of autonomous driving, thereby facilitating the updating of positioning maps.
    Type: Grant
    Filed: March 2, 2020
    Date of Patent: October 11, 2022
    Assignee: Beijing Baidu Netcom Science and Technology Co., Ltd.
    Inventors: Hang Gao, Wendong Ding, Shenhua Hou, Guowei Wan, Shiyu Song
  • Patent number: 11467591
    Abstract: In one embodiment, a system uses an actor-critic reinforcement learning model to generate a trajectory for an autonomous driving vehicle (ADV) in an open space. The system perceives an environment surrounding an ADV. The system applies a RL algorithm to an initial state of a planning trajectory based on the perceived environment to determine a plurality of controls for the ADV to advance to a plurality of trajectory states based on map and vehicle control information for the ADV. The system determines a reward prediction by the RL algorithm for each of the plurality of controls in view of a target destination state. The system generates a first trajectory from the trajectory states by maximizing the reward predictions to control the ADV autonomously according to the first trajectory.
    Type: Grant
    Filed: May 15, 2019
    Date of Patent: October 11, 2022
    Assignee: BAIDU USA LLC
    Inventors: Runxin He, Jinyun Zhou, Qi Luo, Shiyu Song, Jinghao Miao, Jiangtao Hu, Yu Wang, Jiaxuan Xu, Shu Jiang
  • Patent number: 11465642
    Abstract: In one embodiment, a system receives a stream of frames of point clouds from one or more LIDAR sensors of an ADV and corresponding poses in real-time. The system extracts segment information for each frame of the stream based on geometric or spatial attributes of points in the frame, where the segment information includes one or more segments of at least a first frame corresponding to a first pose. The system registers the stream of frames based on the segment information. The system generates a first point cloud map for the stream of frames based on the frame registration.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: October 11, 2022
    Assignees: BAIDU USA LLC, Baidu.com Times Technology (Beijing) Co., Ltd.
    Inventors: Yong Xiao, Runxin He, Pengfei Yuan, Li Yu, Shiyu Song
  • Patent number: 11448771
    Abstract: Embodiments of the present disclosure provide a method and apparatus for evaluating data, a device, and a computer-readable storage medium in the field of autonomous driving. The method may include: generating, at least based on satellite navigation data related to a vehicle, a reference position and posture data set of the vehicle; generating, at least based on a part of the satellite navigation data, a test position and posture data set of the vehicle, the accuracy of the test position and posture data set being lower than that of the reference position and posture data set; obtaining a revised position and posture data set for the test position and posture data set; and evaluating, based on the reference position and posture data set and the revised position and posture data set, a correction performance for the test position and posture data set.
    Type: Grant
    Filed: March 10, 2020
    Date of Patent: September 20, 2022
    Assignee: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
    Inventors: Renlan Cai, Wenjie Liu, Xiaotao Li, Pengfei Yuan, Liang Peng, Shiyu Song
  • Patent number: 11447153
    Abstract: A method and apparatus for annotating a virtual lane at a crossing is provided, which relates to the field of intelligent transportation, and specifically includes: calculating a virtual connecting probability of various lanes that do not connected to each other at the crossing based on driving trajectory data of a vehicles that passing through the crossing within a detection time, and generating the virtual lane at the crossing for two disconnected lanes whose virtual connecting probability is greater than a threshold, and annotating the virtual lane on a map. In this process, the virtual lane at the crossing can be automatically generated according to the driving trajectory data of the vehicles passing through the crossing, and because the driving trajectory data of the vehicles passing through the crossing is real trajectory data, which is more in conformity with actual driving rules of the vehicles, and has a higher annotation accuracy.
    Type: Grant
    Filed: June 4, 2021
    Date of Patent: September 20, 2022
    Assignee: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.
    Inventors: Yang Liu, Shiyu Song
  • Patent number: 11409284
    Abstract: In one embodiment, an open space model is generated for a system to plan trajectories for an ADV in an open space. The system perceives an environment surrounding an ADV including one or more obstacles. The system determines a target function for the open space model based on constraints for the one or more obstacles and map information. The system iteratively, performs a first quadratic programming (QP) optimization on the target function based on a first trajectory while fixing a first set of variables, and performs a second QP optimization on the target function based on a result of the first QP optimization while fixing a second set of variables. The system generates a second trajectory based on results of the first and the second QP optimizations to control the ADV autonomously according to the second trajectory.
    Type: Grant
    Filed: May 15, 2019
    Date of Patent: August 9, 2022
    Assignee: BAIDU USA LLC
    Inventors: Runxin He, Jinyun Zhou, Qi Luo, Shiyu Song, Jinghao Miao, Jiangtao Hu, Yu Wang, Jiaxuan Xu, Shu Jiang
  • Patent number: 11373328
    Abstract: The disclosure provides a method, an apparatus, a device and a storage medium for positioning an object. The method includes: obtaining a map related to a region where the object is located, the map including a plurality of map layers having different height information; determining, based on the map and current point cloud data related to the object, an estimated position of the object, an estimated height corresponding to the estimated position and an estimated probability that the object is located at the estimated position with an estimated posture; and determining, at least based on the estimated position, the estimated height and the estimated probability, positioning information for the object, the positioning information indicating at least one of a current position of the object, a current height of the object and a current posture of the object.
    Type: Grant
    Filed: September 30, 2020
    Date of Patent: June 28, 2022
    Assignee: Apollo Intelligent Driving Technology (Beijing) Co., Ltd.
    Inventors: Guowei Wan, Shenhua Hou, Shiyu Song
  • Patent number: 11364931
    Abstract: In one embodiment, a method for temporal smoothness in localization results for an autonomous driving vehicle includes: creating a probability offset volume that represents an overall matching cost between a first set of keypoints from the online point cloud and a second set of keypoints from a pre-built point cloud map for each of a series of sequential light detection and ranging (LiDAR) frames in an online point cloud. The method also includes compressing the probability offset volume into multiple probability vectors across a X dimension, a Y dimension and a yaw dimension; providing each probability vector of the probability offset volume to a number of recurrent neural networks (RNNs); and generating, by the RNNs, a trajectory of location results across the plurality of sequential LiDAR frames.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: June 21, 2022
    Assignees: BAIDU USA LLC, BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO. LTD.
    Inventors: Weixin Lu, Yao Zhou, Guowei Wan, Shenhua Hou, Shiyu Song
  • Publication number: 20220164595
    Abstract: The present disclosure provides a method, an apparatus, an electronic device and a storage medium for vehicle localization, which relates to the technical fields of autonomous driving, electronic map, deep learning, image processing, and the like. In the method, a computing device obtains an image descriptor map corresponding to a captured image of an external environment of a vehicle and a predicted pose of the vehicle when the captured image is captured; obtains a set of reference descriptors and a set of spatial coordinates corresponding to a set of keypoints of a reference image of the external environment; determines a plurality of sets of image descriptors corresponding to the set of spatial coordinates when the vehicle is in a plurality of candidate poses, respectively; determines a plurality of similarities between the plurality of sets of image descriptors and the set of reference descriptors; and updates the predicted pose based on the plurality of candidate poses and the plurality of similarities.
    Type: Application
    Filed: November 25, 2020
    Publication date: May 26, 2022
    Inventors: Yao ZHOU, Guowei WAN, Shenhua HOU, Shiyu SONG
  • Publication number: 20220164603
    Abstract: The present disclosure provides a data processing method, an apparatus, an electronic device and a medium, which relates to the technical fields of autonomous driving, electronic maps, deep learning, image processing, and the like. The method includes: a computing device inputs a reference image and a captured image into a feature extraction model; obtain, a set of reference descriptors based on the first descriptor map; determine a plurality of sets of training descriptors; obtain a predicted pose of the vehicle by inputting the plurality of training poses and a plurality of similarities into a pose prediction model; and train the feature extraction model and the pose prediction model. When applied to a vehicle localization system, the trained feature extraction model and pose prediction model according to some embodiments of the present disclosure can improve accuracy and robustness of vehicle localization, thereby boosting the performance of the vehicle localization system.
    Type: Application
    Filed: November 25, 2020
    Publication date: May 26, 2022
    Inventors: Yao ZHOU, Guowei WAN, Shenhua HOU, Shiyu SONG
  • Patent number: 11328401
    Abstract: A stationary object detecting method, a stationary object apparatus, and an electronic device are disclosed in embodiments of the present disclosure, the method includes: obtaining point cloud data of a scene and feature information of each of data points in the point cloud data; performing a triangulation network connection on each of the data points in the point cloud data to generate a triangular network model, and taking a picture of the triangular network model using a preset camera to obtain an image; obtaining a first data point corresponding to each pixel point in the image, and obtaining a feature map of the point cloud data according to feature information of each first data point; inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the point cloud data.
    Type: Grant
    Filed: July 22, 2019
    Date of Patent: May 10, 2022
    Inventors: Weixin Lu, Shiyu Song, Fangfang Dong, Shengpan Xu
  • Patent number: 11315317
    Abstract: In one embodiment, a system generates an occupancy grid map based on an initial frame of point clouds. The system receives one or more subsequent frames of the point clouds. For each of the subsequent frames, the system updates an occupancy grid map based on the subsequent frame. The system identifies one or more problematic voxels based on the update, the system determines whether the problematic voxels belong to a wall object, and in response to determining that the problematic voxels belong to a wall object, the system flags the problematic voxels as ghost effect voxels for the subsequent frame.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: April 26, 2022
    Assignees: BAIDU USA LLC, BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO. LTD.
    Inventors: Li Yu, Pengfei Yuan, Yong Xiao, Runxin He, Shiyu Song
  • Publication number: 20220081000
    Abstract: In one embodiment, a system/method generates a driving trajectory for an autonomous driving vehicle (ADV). The system perceives an environment of an autonomous driving vehicle (ADV). The system determines one or more bounding conditions based on the perceived environment. The system generates a first trajectory using a neural network model, wherein the neural network model is trained to generate a driving trajectory. The system evaluates/determines if the first trajectory satisfies the one or more bounding conditions. If the first trajectory satisfies the one or more bounding conditions, the system controls the ADV autonomously according to the first trajectory. Otherwise, the system controls the ADV autonomously according to a second trajectory, where the second trajectory is generated based on an objective function, where the objective function is determined based on at least the one or more bounding conditions.
    Type: Application
    Filed: September 15, 2020
    Publication date: March 17, 2022
    Inventors: YIFEI JIANG, JINYUN ZHOU, JIAMING TAO, SHU JIANG, JIANGTAO HU, JINGHAO MIAO, SHIYU SONG
  • Patent number: 11255973
    Abstract: A method and an apparatus for extracting a lane line, a device, a computer readable-storage medium and a collection entity are provided. The method includes: obtaining a first group of lane lines of a road based on a first image generated from a point cloud collected by a laser radar; obtaining a second group of lane lines of the road based on a second image collected by a camera; and determining a lane line set of the road based on the first group of lane lines and the second group of lane lines.
    Type: Grant
    Filed: March 14, 2019
    Date of Patent: February 22, 2022
    Assignee: Apollo Intelligent Driving Technology (Beijing) Co., Ltd.
    Inventors: Lixia Shen, Ruijie Hou, Shiyu Song, Fangfang Dong
  • Patent number: 11243086
    Abstract: Embodiments of the present disclosure provide a method and a device for acquiring a map, a device, and a computer readable storage medium. The method includes: acquiring a reference map of an acquisition area; dividing the acquisition area into a plurality of sub-areas based on the reference map; generating a plurality of acquisition tasks corresponding to the plurality of sub-areas, wherein each of the plurality of acquisition tasks is configured to acquire a map of each of the plurality of sub-areas; and assigning each of the plurality of acquisition tasks to an acquisition entity, to enable the acquisition entity to acquire the map of each of the plurality of sub-areas.
    Type: Grant
    Filed: July 3, 2019
    Date of Patent: February 8, 2022
    Assignee: Apollo Intelligent Driving Technology (Beijing) Co., Ltd.
    Inventors: Li Yu, Shiyu Song, Fangfang Dong
  • Patent number: 11227398
    Abstract: In one embodiment, a system receives a number of point clouds captured by one or more LIDAR sensors of an ADV and corresponding poses. The system receives a number of RGB images from one or more image capturing sensors of the ADV. The system synchronizes the RGB images with the point clouds to obtain RGB point clouds. The system extracts features from the RGB point clouds, the features including contextual and spatial information of the RGB point clouds. The system registers the RGB point clouds based on the extracted features and generates a point cloud map based on the registration of the RGB point clouds.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: January 18, 2022
    Assignees: BAIDU USA LLC, BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO., LTD.
    Inventors: Yong Xiao, Runxin He, Pengfei Yuan, Li Yu, Shiyu Song
  • Publication number: 20210405200
    Abstract: The present application discloses a high-precision mapping method and device, which relates to the field of autonomous driving. A specific implementation includes: acquiring global initial poses of multiple point clouds, where the point clouds are point clouds of a location for which a map is to be built and are collected by a lidar using a multi-circle collection mode; dividing the multiple point clouds into multiple spatial submap graphs according to a spatial distribution relationship of the multiple point clouds; optimizing, for each spatial submap graph, global initial poses of point clouds belonging to the spatial submap graph to acquire global poses of the point clouds in each spatial submap graph; and stitching the multiple spatial submap graphs together according to global poses of the point clouds in the multiple spatial submap graphs to acquire a base graph of the map to be built.
    Type: Application
    Filed: December 30, 2020
    Publication date: December 30, 2021
    Applicant: Beijing Baidu Netcome Science Technology Co. Ltd.
    Inventors: Wendong DING, Fan YI, Li YU, Pengfei YUAN, Shiyu SONG