Patents by Inventor Shuqing Zeng

Shuqing Zeng has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10572748
    Abstract: An adaptive parallel imaging processing system in a vehicle is provided. The system may include, but is not limited to, a plurality of processors and a resource management system including, but not limited to, an execution monitor, the execution monitor configured to calculate an average utilization of each of the plurality of processors over a moving window, and a service scheduler controlling a request queue for each of the plurality of processors, the service scheduler scheduling image processing tasks in the respective request queue for the each of the plurality of processors based upon the average utilization of each of the plurality of processors, the capabilities of each of the plurality of processors, and a priority associated with each image processing task, wherein an autonomous vehicle control system is configured to generate the instructions to control the at least one vehicle system based upon the processed image processing tasks.
    Type: Grant
    Filed: December 6, 2017
    Date of Patent: February 25, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman Millett
  • Publication number: 20200027241
    Abstract: In various embodiments, methods, systems, and vehicles are provided for calibrating vehicle cameras. In certain embodiments, a vehicle includes a camera, a memory, and a processor. The camera is disposed onboard the vehicle, and is configured to generate a camera image in which an object is detected onboard the vehicle. The memory is configured to store map data relating to the detected object. The processor is disposed onboard the vehicle, and is configured to perform an initial projection using the map data and initial values of calibration parameters for the camera; and update values of the calibration parameters using a comparison of the initial projection with the camera image.
    Type: Application
    Filed: July 19, 2018
    Publication date: January 23, 2020
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Wei Tong, Shuqing Zeng
  • Patent number: 10534368
    Abstract: A crowdsourced virtual sensor generator is provided which could be generated by a vehicle or provided to the vehicle as, for example, a service. The crowdsourced virtual sensor generator may include, but is not limited to, a communication system configured to receive contributing vehicle sensor data from one or more contributing vehicles, a location of the one or more contributing vehicles and a target vehicle, and a processor, the processor configured to filter the received contributing vehicle sensor data based upon the location of the one or more contributing vehicles and the location of the target vehicle, aggregate the filtered contributing vehicle sensor data into at least one of a data-specific dataset and an application-specific data set, and generate a virtual sensor for the target vehicle, the virtual sensor processing the filtered and aggregated contributing vehicle sensor data to generate output data relative to the location of the target vehicle.
    Type: Grant
    Filed: June 26, 2017
    Date of Patent: January 14, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Ramesh Sethu, Arun Adiththan, Shuqing Zeng, Upali P. Mudalige
  • Publication number: 20190361439
    Abstract: Systems and methods are provided for controlling an autonomous vehicle (AV). A scene understanding module of a high-level controller selects a particular combination of sensorimotor primitive modules to be enabled and executed for a particular driving scenario from a plurality of sensorimotor primitive modules. Each one of the particular combination of the sensorimotor primitive modules addresses a sub-task in a sequence of sub-tasks that address a particular driving scenario. A primitive processor module executes the particular combination of the sensorimotor primitive modules such that each generates a vehicle trajectory and speed profile. An arbitration module selects one of the vehicle trajectory and speed profiles having the highest priority ranking for execution, and a vehicle control module processes the selected one of vehicle trajectory and speed profiles to generate control signals used to execute one or more control actions to automatically control the AV.
    Type: Application
    Filed: May 24, 2018
    Publication date: November 28, 2019
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Upali P. Mudalige
  • Publication number: 20190361454
    Abstract: Systems and methods are provided for controlling an autonomous vehicle (AV). A feature map generator module generates a feature map (FM). Based on the FM, a perception map generator module generates a perception map (PM). A scene understanding module selects from a plurality of sensorimotor primitive modules (SPMs), based on the FM, a particular combination of SPMs to be enabled and executed for the particular driving scenario (PDS). Each SPM maps information from either the FM or the PM to a vehicle trajectory and speed profile (VTSP) for automatically controlling the AV to cause the AV to perform a specific driving maneuver. Each one of the particular combination of the SPMs addresses a sub-task in a sequence of sub-tasks that address the PDS. Each of the particular combination of the SPMs are retrieved from memory and executed to generate a corresponding VTSP.
    Type: Application
    Filed: May 24, 2018
    Publication date: November 28, 2019
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Upali P. Mudalige
  • Publication number: 20190361456
    Abstract: Systems and methods are provided for controlling an autonomous vehicle (AV). A map generator module processes sensor data to generate a world representation of a particular driving scenario (PDS). A scene understanding module (SUM) processes navigation route data, position information and a feature map to define an autonomous driving task (ADT), and decomposes the ADT into a sequence of sub-tasks. The SUM selects a particular combination of sensorimotor primitive modules (SPMs) to be enabled and executed for the PDS. Each one of the SPMs addresses a sub-task in the sequence. A primitive processor module executes the particular combination of the SPMs such that each generates a vehicle trajectory and speed (VTS) profile.
    Type: Application
    Filed: May 24, 2018
    Publication date: November 28, 2019
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Upali P. Mudalige
  • Publication number: 20190354781
    Abstract: A system and method for determining an object location by using map information is disclosed. The method includes receiving, by a controller, an image data of a scene. An object is located within the scene. The method also includes determining, by the controller, a location of the object based on the image data of the scene. The method can also include receiving, by the controller, a map information. The map information includes at least one path information. The method also includes determining an association between the location of the object based on the image data and the map information of the scene. The method also includes determining a 3-dimensional location of the object based on the determined association.
    Type: Application
    Filed: May 17, 2018
    Publication date: November 21, 2019
    Inventors: Wei Tong, Yang Yang, Brent N. Bacchus, Shuqing Zeng
  • Patent number: 10482331
    Abstract: Methods and systems are provided for detecting an object. In one embodiment, a method includes: receiving, by a processor, image data from an image sensor; receiving, by a processor, radar data from a radar system; processing, by the processor, the image data from the image sensor and the radar data from the radar system using a deep learning method; and detecting, by the processor, an object based on the processing.
    Type: Grant
    Filed: November 16, 2016
    Date of Patent: November 19, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventor: Shuqing Zeng
  • Patent number: 10444342
    Abstract: A system and method of controlling operation of a host device in real-time, the host device operatively connected to an optical device and a radar device. The optical device is configured to obtain visual data of at least one object. The object is located at an incline, relative to the host device, the incline being characterized by an elevation angle (?) and an azimuth angle (?). The radar device is configured to obtain radar data, including a radial distance (r) of the object from the host device, the azimuth angle (?), and a range rate (dr/dt). The controller is programmed to determine a time-to-contact for the host device and the object based at least partially on a 3-D position and 3-D velocity vector. The operation of the host device is controlled based at least partially on the time-to-contact.
    Type: Grant
    Filed: March 8, 2017
    Date of Patent: October 15, 2019
    Assignee: GM Global Technology Operations LLC.
    Inventors: Shuqing Zeng, Jay H. Sim, Mohammad Naserian
  • Patent number: 10395144
    Abstract: Methods and apparatus are provided for controlling an autonomous vehicle. A sensor fusion system with a sensor system for providing environment condition information and a convolutional neural network (CNN) is provided. The CNN includes a receiving interface configured to receive the environment condition information from the sensor system, a common convolutional layer configured to extract traffic information from the received environment condition information, and a plurality of fully connected layers configured to detect objects belonging to different object classes based on the extracted traffic information, wherein the object classes include at least one of a road feature class, a static object class, and a dynamic object class.
    Type: Grant
    Filed: July 24, 2017
    Date of Patent: August 27, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Upali P. Mudalige
  • Patent number: 10391939
    Abstract: A system and method to determine trailer pose includes imaging first and second telltales affixed to a trailer to provide trailer image data. A controller is operable to determine trailer pose from the trailer image data. The trailer pose information may be provided to one or more controllers of a vehicle towing the trailer.
    Type: Grant
    Filed: September 1, 2016
    Date of Patent: August 27, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Mohammad Naserian, Shuqing Zeng, Jay H. Sim
  • Publication number: 20190258894
    Abstract: A system and method to perform detection based on sensor fusion includes obtaining data from two or more sensors of different types. The method also includes extracting features from the data from the two or more sensors and processing the features to obtain a vector associated with each of the two or more sensors. The method further includes concatenating the two or more vectors obtained from the two or more sensors to obtain a fused vector, and performing the detection based on the fused vector.
    Type: Application
    Filed: February 20, 2018
    Publication date: August 22, 2019
    Inventors: Shuqing Zeng, Wei Tong, Yasen Hu, Mohannad Murad, David R. Petrucci, Gregg R. Kittinger
  • Publication number: 20190258876
    Abstract: A system and method of operating a vehicle. The system includes a two-dimensional imager, a three-dimensional imager, and at least one processor. The two-dimensional imager obtains a two-dimensional image of an environment surrounding the vehicle, wherein the environment includes an object. The three-dimensional imager obtains a three-dimensional (3D) point cloud of the environment. The at least one processor identifies the object from the 2D image and assigns the identification of the object to a selected point of the 3D point cloud.
    Type: Application
    Filed: February 20, 2018
    Publication date: August 22, 2019
    Inventors: Hansi Liu, Fan Bai, Shuqing Zeng
  • Publication number: 20190244136
    Abstract: A system and method to perform inter-sensor learning obtains a detection of a target based on a first sensor. The method also includes determining whether a second sensor with an overlapping detection range with the first sensor also detects the target, and performing learning to update a detection algorithm used with the second sensor based on the second sensor failing to detect the target.
    Type: Application
    Filed: February 5, 2018
    Publication date: August 8, 2019
    Inventors: Yasen Hu, Shuqing Zeng, Wei Tong, Mohannad Murad, Gregg R. Kittinger, David R. Petrucci
  • Patent number: 10373000
    Abstract: A method of identifying a condition of a road surface includes capturing at least a first image of the road surface with a first camera, and a second image of the road surface with a second camera. The first image and the second image are tiled together to form a combined tile image. A feature vector is extracted from the combined tile image using a convolutional neural network, and a condition of the road surface is determined from the feature vector using a classifier.
    Type: Grant
    Filed: August 15, 2017
    Date of Patent: August 6, 2019
    Assignee: GM Global Technology Operations LLC
    Inventors: Wei Tong, Qingrong Zhao, Shuqing Zeng, Bakhtiar B. Litkouhi
  • Publication number: 20190232955
    Abstract: Technical solutions are described for controlling an automated driving system of a vehicle. An example method includes computing a complexity metric of an upcoming region along a route that the vehicle is traveling along. The method further includes, in response to the complexity metric being below a predetermined low-complexity threshold, determining a trajectory for the vehicle to travel in the upcoming region using a computing system of the vehicle. Further, the method includes in response to the complexity metric being above a predetermined high-complexity threshold, instructing an external computing system to determine the trajectory for the vehicle to travel in the upcoming region. If the trajectory cannot be determined by the external computing system a minimal risk condition maneuver of the vehicle is performed.
    Type: Application
    Filed: February 1, 2018
    Publication date: August 1, 2019
    Inventors: Donald K. Grimm, Wei Tong, Shuqing Zeng, Upali P. Mudalige
  • Publication number: 20190232898
    Abstract: A system and method to perform dynamic bandwidth adjustment among two or more vehicle sensors includes receiving input. The input includes data from each of the two or more vehicle sensors. The two or more vehicle sensors include a camera, an audio detector, a radar system, or a lidar system. The method also includes determining a bandwidth at which each of the two or more vehicle sensors should provide the data, and providing a control signal to each of the two or more vehicle sensors to adjust the bandwidth based on the determining.
    Type: Application
    Filed: February 1, 2018
    Publication date: August 1, 2019
    Inventors: Yasen Hu, Shuqing Zeng, Wei Tong, Mohannad Murad, David R. Petrucci, Gregg R. Kittinger
  • Publication number: 20190228504
    Abstract: A system and method for generating a range image using sparse depth data is disclosed. The method includes receiving, by a controller, image data of a scene. The image data includes a first set of pixels. The method also includes receiving, by the controller, a sparse depth data of the scene. The sparse depth data includes a second set of pixels, and the number of the second set of pixels is less than the number of first set of pixels. The method also includes combining the image data and the sparse depth data into a combined data. The method also includes generating a range image using the combined data.
    Type: Application
    Filed: January 24, 2018
    Publication date: July 25, 2019
    Inventors: Wei Tong, Shuqing Zeng, Upali P. Mudalige
  • Patent number: 10354368
    Abstract: A method and apparatus for determining ground clearance of a structure are provided. The method includes removing reflection points caused by noise from first reflection point information based on temporal persistency and generating second reflection point information, extracting visual features from an image of a camera based on convolutional neural network, projecting the second reflection point information onto the image, generating region proposals based on the projected second reflection point information and the image, the region proposals indicating potential horizontal structures above a path, detecting stationary horizontal structure above a path based on the generated region proposals, and determining distance between ground and the detected stationary horizontal structure based on the projected reflection point information and the image.
    Type: Grant
    Filed: July 31, 2017
    Date of Patent: July 16, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Mohammad Naserian, Shuqing Zeng, Allan Lewis
  • Patent number: 10346705
    Abstract: A method and apparatus for trailer recognition are provided. The method includes detecting a face of a trailer in a first and second image, taken at first and second distances respectively, generating a bounding box around the face of the trailer in the first and second images, determining a first and second set of feature points in the bounding boxes of the first and second images, determining a correspondence between the first set of feature points and the second set of feature points and a correspondence between corners of the bounding boxes and estimating at least one dimension of the trailer by performing a least squares analysis to solve for three-dimensional real world coordinates of the first and second set of features points and the corners of the bounding boxes. The method may assist an operator of a vehicle in aligning the vehicle hitch with a trailer coupler.
    Type: Grant
    Filed: June 20, 2017
    Date of Patent: July 9, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Mohammad Naserian, Shuqing Zeng, Allan Lewis