Patents by Inventor Shuqing Zeng

Shuqing Zeng has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10741081
    Abstract: Technical solutions are described for vehicle collision prevention for a vehicle when the vehicle is in a parked condition. An example method includes performing a stationary safety monitoring when the vehicle is in parked condition. The stationary safety monitoring includes detecting presence of a moving object within a predetermined region from the vehicle. Further, the method includes, in response to detecting the moving object in the predetermined region initiating a notification for the moving object to prevent collision with the vehicle.
    Type: Grant
    Filed: November 14, 2017
    Date of Patent: August 11, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Donald K. Grimm, Shuqing Zeng, Upali P. Mudalige, Robert A. Bordo, Perry L. Maniaci
  • Patent number: 10706505
    Abstract: A system and method for generating a range image using sparse depth data is disclosed. The method includes receiving, by a controller, image data of a scene. The image data includes a first set of pixels. The method also includes receiving, by the controller, a sparse depth data of the scene. The sparse depth data includes a second set of pixels, and the number of the second set of pixels is less than the number of first set of pixels. The method also includes combining the image data and the sparse depth data into a combined data. The method also includes generating a range image using the combined data.
    Type: Grant
    Filed: January 24, 2018
    Date of Patent: July 7, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Wei Tong, Shuqing Zeng, Upali P. Mudalige
  • Patent number: 10678253
    Abstract: Systems and methods are provided for controlling an autonomous vehicle (AV). A map generator module processes sensor data to generate a world representation of a particular driving scenario (PDS). A scene understanding module (SUM) processes navigation route data, position information and a feature map to define an autonomous driving task (ADT), and decomposes the ADT into a sequence of sub-tasks. The SUM selects a particular combination of sensorimotor primitive modules (SPMs) to be enabled and executed for the PDS. Each one of the SPMs addresses a sub-task in the sequence. A primitive processor module executes the particular combination of the SPMs such that each generates a vehicle trajectory and speed (VTS) profile. A selected one of the VTS profiles is then processed to generate the control signals, which are then processed at a low-level controller to generate commands that control one or more of actuators of the AV.
    Type: Grant
    Filed: May 24, 2018
    Date of Patent: June 9, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Upali P. Mudalige
  • Patent number: 10678247
    Abstract: An autonomic vehicle control system includes a perception module of a spatial monitoring system that is disposed to monitor a spatial environment proximal to the autonomous vehicle. A method for evaluating vehicle dynamics operation includes determining a desired trajectory for the autonomous vehicle, wherein the desired trajectory includes desired vehicle positions including an x-position, a y-position and a heading. Vehicle control commands are determined based upon the desired trajectory, and include a commanded steering angle, an acceleration command and a braking command. Actual vehicle states responsive to the vehicle control commands are determined. An estimated trajectory is determined based upon the actual vehicle states, and a trajectory error is determined based upon a difference between the desired trajectory and the estimated trajectory. The trajectory error is monitored over a time horizon, and a first state of health (SOH) is determined based upon the trajectory error over the time horizon.
    Type: Grant
    Filed: August 28, 2017
    Date of Patent: June 9, 2020
    Assignee: GM Global Technology Operations LLC
    Inventors: Shengbing Jiang, Mutasim A. Salman, Yilu Zhang, Shuqing Zeng
  • Patent number: 10671860
    Abstract: A system and method of operating a vehicle. The system includes a two-dimensional imager, a three-dimensional imager, and at least one processor. The two-dimensional imager obtains a two-dimensional image of an environment surrounding the vehicle, wherein the environment includes an object. The three-dimensional imager obtains a three-dimensional (3D) point cloud of the environment. The at least one processor identifies the object from the 2D image and assigns the identification of the object to a selected point of the 3D point cloud.
    Type: Grant
    Filed: February 20, 2018
    Date of Patent: June 2, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Hansi Liu, Fan Bai, Shuqing Zeng
  • Publication number: 20200167887
    Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.
    Type: Application
    Filed: November 26, 2018
    Publication date: May 28, 2020
    Applicant: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Publication number: 20200167941
    Abstract: Systems and methods for depth estimation of images from a mono-camera by use of radar data by: receiving, a plurality of input 2-D images from the mono-camera; generating, by the processing unit, an estimated depth image by supervised training of an image estimation model; generating, by the processing unit, a synthetic image from a first input image and a second input image from the mono-camera by applying an estimated transform pose; comparing, by the processing unit, an estimated three-dimensional (3-D) point cloud to radar data by applying another estimated transform pose to a 3-D point cloud wherein the 3-D point cloud is estimated from a depth image by supervised training of the image estimation model to radar distance and radar doppler measurement; correcting a depth estimation of the estimated depth image by losses derived from differences: of the synthetic image and original images; of an estimated depth image and a measured radar distance; and of an estimated doppler information and measured radar d
    Type: Application
    Filed: November 27, 2018
    Publication date: May 28, 2020
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Wei Tong, Shuqing Zeng, Mohannad Murad, Alaa M. Khamis
  • Patent number: 10663581
    Abstract: An ultra-short range radar (USRR) system of a vehicle includes an object detection module configured to, based on radar signals from USRR sensors of the vehicle: identify the presence of an object that is external to the vehicle; determine a location of the object; and determine at least one of a height, a length, and a width of the object. A remedial action module is configured to, based on the location of the object and the at least one dimension of the object, at least one of: selectively actuate an actuator of the vehicle; selectively generate an audible alert via at least one speaker of the vehicle; and selectively generate a visual alert via at least one light emitting device of the vehicle.
    Type: Grant
    Filed: July 13, 2017
    Date of Patent: May 26, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Stephen W. Decker, Jeremy P. Gray, Igal Bilik, Shuqing Zeng
  • Publication number: 20200160125
    Abstract: A signal processing system includes a central processing unit (CPU) in communication with an accelerator, and an instruction scheduler in communication with the accelerator. A first memory device including a first instruction set is configured to operate the accelerator, a second instruction set is configured to operate the CPU, and a second memory device is configured to receive a datafile. The accelerator includes a plurality of processing engines (PEs) and an instruction scheduler, the instruction set includes a plurality of operators, and the instruction scheduler is configured to implement the operators in the accelerator employing the PEs. The CPU employs the operators implemented in the accelerator to analyze the datafile to extract a feature therefrom.
    Type: Application
    Filed: November 16, 2018
    Publication date: May 21, 2020
    Applicant: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Patent number: 10657617
    Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.
    Type: Grant
    Filed: November 26, 2018
    Date of Patent: May 19, 2020
    Assignee: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Publication number: 20200142026
    Abstract: A disambiguating system for disambiguating between ambiguous detections is provided. The system includes a plurality of modules, wherein each module is configured to disambiguate between ambiguous detections by selecting, as a true detection, one candidate detection in a set of ambiguous detections and wherein each module is configured to apply a different selection technique. The system includes: one or more modules configured to select as the true detection, the candidate detection whose associated position is closer to a position indicated by other data and one or more modules configured to select as the true detection, the candidate detection with the highest probability of being true based on other sensor data.
    Type: Application
    Filed: November 1, 2018
    Publication date: May 7, 2020
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: LAWRENCE A. BUSH, BRENT N. BACCHUS, SHUQING ZENG, STEPHEN W. DECKER
  • Publication number: 20200134324
    Abstract: Examples of techniques for using fixed-point quantization in deep neural networks are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises quantizing a fixed-point value based on an activation input and a synapse weight. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.
    Type: Application
    Filed: October 25, 2018
    Publication date: April 30, 2020
    Inventors: Shuqing Zeng, Wei Tong, Shige Wang, Roman L. Millett
  • Publication number: 20200134459
    Abstract: In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.
    Type: Application
    Filed: October 25, 2018
    Publication date: April 30, 2020
    Inventors: Shuqing Zeng, Wei Tong, Shige Wang, Roman L. Millett
  • Patent number: 10600257
    Abstract: An autonomic vehicle control system is described, and includes a vehicle spatial monitoring system including a subject spatial sensor that is disposed to monitor a spatial environment proximal to the autonomous vehicle. A controller is in communication with the subject spatial sensor, and the controller includes a processor and a memory device including an instruction set. The instruction set is executable to evaluate the subject spatial sensor, which includes determining first, second, third, fourth and fifth SOH (state of health) parameters associated with the subject spatial sensor, and determining an integrated SOH parameter for the subject spatial sensor based thereupon.
    Type: Grant
    Filed: August 28, 2017
    Date of Patent: March 24, 2020
    Assignee: GM Global Technology Operations LLC
    Inventors: Shengbing Jiang, Mutasim A. Salman, Xinyu Du, Wen-Chiao Lin, Jinsong Wang, Shuqing Zeng
  • Patent number: 10602242
    Abstract: Apparatus, method and system for hybrid multi-mode data fusion to fuse data of multiple different formats by a centralized data fusion processing and send the fused data by a distributed data processing to a set of heterogeneous sensor devices. The hybrid multi-mode data fusion apparatus comprises a multi-mode fusion module configured to operate in a first mode to receive fused data of multiple different formats sensed from a set of heterogeneous devices and configured to operate in a second mode to send the fused data to each of the heterogeneous devices. An association module to fuse data of multiple different formats sensed from each of the heterogeneous devices and to send the fused data to the multi-mode fusion module. A fusion track data file to store prior fused data to enable the multi-mode fusion module to send cueing information based upon the stored prior fused data to the heterogeneous devices.
    Type: Grant
    Filed: June 14, 2017
    Date of Patent: March 24, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventor: Shuqing Zeng
  • Patent number: 10592784
    Abstract: A system and method to perform detection based on sensor fusion includes obtaining data from two or more sensors of different types. The method also includes extracting features from the data from the two or more sensors and processing the features to obtain a vector associated with each of the two or more sensors. The method further includes concatenating the two or more vectors obtained from the two or more sensors to obtain a fused vector, and performing the detection based on the fused vector.
    Type: Grant
    Filed: February 20, 2018
    Date of Patent: March 17, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Yasen Hu, Mohannad Murad, David R. Petrucci, Gregg R. Kittinger
  • Publication number: 20200065661
    Abstract: Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).
    Type: Application
    Filed: August 21, 2018
    Publication date: February 27, 2020
    Inventors: Wei Tong, Shuqing Zeng, Upali P. Mudalige, Shige Wang
  • Patent number: 10572748
    Abstract: An adaptive parallel imaging processing system in a vehicle is provided. The system may include, but is not limited to, a plurality of processors and a resource management system including, but not limited to, an execution monitor, the execution monitor configured to calculate an average utilization of each of the plurality of processors over a moving window, and a service scheduler controlling a request queue for each of the plurality of processors, the service scheduler scheduling image processing tasks in the respective request queue for the each of the plurality of processors based upon the average utilization of each of the plurality of processors, the capabilities of each of the plurality of processors, and a priority associated with each image processing task, wherein an autonomous vehicle control system is configured to generate the instructions to control the at least one vehicle system based upon the processed image processing tasks.
    Type: Grant
    Filed: December 6, 2017
    Date of Patent: February 25, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman Millett
  • Publication number: 20200027241
    Abstract: In various embodiments, methods, systems, and vehicles are provided for calibrating vehicle cameras. In certain embodiments, a vehicle includes a camera, a memory, and a processor. The camera is disposed onboard the vehicle, and is configured to generate a camera image in which an object is detected onboard the vehicle. The memory is configured to store map data relating to the detected object. The processor is disposed onboard the vehicle, and is configured to perform an initial projection using the map data and initial values of calibration parameters for the camera; and update values of the calibration parameters using a comparison of the initial projection with the camera image.
    Type: Application
    Filed: July 19, 2018
    Publication date: January 23, 2020
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Wei Tong, Shuqing Zeng
  • Patent number: 10534368
    Abstract: A crowdsourced virtual sensor generator is provided which could be generated by a vehicle or provided to the vehicle as, for example, a service. The crowdsourced virtual sensor generator may include, but is not limited to, a communication system configured to receive contributing vehicle sensor data from one or more contributing vehicles, a location of the one or more contributing vehicles and a target vehicle, and a processor, the processor configured to filter the received contributing vehicle sensor data based upon the location of the one or more contributing vehicles and the location of the target vehicle, aggregate the filtered contributing vehicle sensor data into at least one of a data-specific dataset and an application-specific data set, and generate a virtual sensor for the target vehicle, the virtual sensor processing the filtered and aggregated contributing vehicle sensor data to generate output data relative to the location of the target vehicle.
    Type: Grant
    Filed: June 26, 2017
    Date of Patent: January 14, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Ramesh Sethu, Arun Adiththan, Shuqing Zeng, Upali P. Mudalige