Patents by Inventor Shige Wang

Shige Wang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11074081
    Abstract: A method in a multiprocessor system for processing multiple perception streams is disclosed. The method comprises: reading data from a plurality of perception streams according to a reading schedule determined by a predetermined policy, each perception stream comprising perception data from a different perception sensor; assigning a unique identification tag to each perception stream; writing each perception stream with its unique identification tag to a server input queue based on the predetermined policy; and processing the tagged perception streams using a server. The processing includes: retrieving tagged perception streams from the server input queue; applying a processing algorithm to process the retrieved tagged perception streams; and outputting the processed perception streams to a server output queue.
    Type: Grant
    Filed: August 2, 2019
    Date of Patent: July 27, 2021
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Ming Yang, Wei Tong, Unmesh Dutta Bordoloi
  • Publication number: 20210218596
    Abstract: A controller area network fault detection and recovery system and method may include a fault detection module, a fault remediation module, a checkpoint manager, and a recovery manager configured to select one or more of the fault remediation mechanisms based upon detected CAN faults. Remediation of detected CAN faults is controlled at a CAN driver software level in accordance with selected fault remediation mechanisms in a predetermined ordered hierarchy.
    Type: Application
    Filed: January 14, 2020
    Publication date: July 15, 2021
    Inventors: Shige Wang, Chang Liu
  • Patent number: 11001272
    Abstract: A method for controlling a vehicle includes: receiving, by a controller, route data, wherein the route data is continuously updated while the vehicle is moving, and the vehicle includes a plurality of vehicle operating modes; receiving, by the controller, feature data, wherein the feature data is information about a plurality of features needed for each of the plurality of vehicle operating modes; determining, by the controller, a plurality of ranges for each of the plurality of vehicle operating modes, wherein each of the plurality of ranges is a function of the route data and the feature data for each of the plurality of vehicle operating modes; and commanding, by the controller, a user interface to display a list of range-mode combinations, wherein the list of range-mode combinations includes the plurality of ranges for each of the plurality of vehicle operating modes.
    Type: Grant
    Filed: February 5, 2019
    Date of Patent: May 11, 2021
    Assignee: GM Global Technology Operations LLC
    Inventors: Unmesh Dutta Bordoloi, Shige Wang, Xinyu Du
  • Publication number: 20210129842
    Abstract: Presented are embedded control systems with logic for computation and data sharing, methods for making/using such systems, and vehicles with distributed sensors and embedded processing hardware for provisioning automated driving functionality. A method for operating embedded controllers connected with distributed sensors includes receiving a first data stream from a first sensor via a first embedded controller, and storing the first data stream with a first timestamp and data lifespan via a shared data buffer in a memory device. A second data stream is received from a second sensor via a second embedded controller. A timing impact of the second data stream is calculated based on the corresponding timestamp and data lifespan. Upon determining that the timing impact does not violate a timing constraint, the first data stream is purged from memory and the second data stream is stored with a second timestamp and data lifespan in the memory device.
    Type: Application
    Filed: November 1, 2019
    Publication date: May 6, 2021
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Wei Tong, Stephen N. McKinnie, Shuqing Zeng
  • Patent number: 10915159
    Abstract: A method of controlling a vehicle includes determining a current operating situation of the vehicle, and identifying a subset of a plurality of sensors of the vehicle needed to provide data to enable a vehicle control function for the current operating situation of the vehicle. A remainder of the plurality of sensors is disengaged to reduce electric energy usage by the vehicle while the vehicle is operating in the current operating situation of the vehicle. A sampling rate for the selected subset of sensors may be reduced to further reduce energy usage of the vehicle. Additionally, an energy reduction processing strategy may be implemented to reduce a processor frequency or a voltage of a computing device used to provide the vehicle control function to further reduce energy usage of the vehicle.
    Type: Grant
    Filed: April 3, 2018
    Date of Patent: February 9, 2021
    Assignee: GM Global Technology Operations LLC
    Inventors: Yiran Hu, Steven E. Muldoon, Wei Tong, Shige Wang
  • Publication number: 20210034371
    Abstract: A method in a multiprocessor system for processing multiple perception streams is disclosed. The method comprises: reading data from a plurality of perception streams according to a reading schedule determined by a predetermined policy, each perception stream comprising perception data from a different perception sensor; assigning a unique identification tag to each perception stream; writing each perception stream with its unique identification tag to a server input queue based on the predetermined policy; and processing the tagged perception streams using a server. The processing includes: retrieving tagged perception streams from the server input queue; applying a processing algorithm to process the retrieved tagged perception streams; and outputting the processed perception streams to a server output queue.
    Type: Application
    Filed: August 2, 2019
    Publication date: February 4, 2021
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Ming Yang, Wei Tong, Unmesh Dutta Bordoloi
  • Patent number: 10909390
    Abstract: Examples of techniques for using fixed-point quantization in deep neural networks are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises quantizing a fixed-point value based on an activation input and a synapse weight. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.
    Type: Grant
    Filed: October 25, 2018
    Date of Patent: February 2, 2021
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shuqing Zeng, Wei Tong, Shige Wang, Roman L. Millett
  • Patent number: 10824943
    Abstract: Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).
    Type: Grant
    Filed: August 21, 2018
    Date of Patent: November 3, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Wei Tong, Shuqing Zeng, Upali P. Mudalige, Shige Wang
  • Patent number: 10754689
    Abstract: A stream manager for managing the distribution of instructions to a plurality of processing devices includes a dispatcher module configured to: receive multiple instruction streams, wherein each instruction stream includes a plurality of requested computations for processing perception data from a perception data source; partition each instruction stream into a plurality of partitions based on type of device to perform a requested computation from the instruction stream; assign a release time and deadline to each partition, and dispatch partition computations to a plurality of scheduling queues to distribute processing of the partition computations amongst the plurality of processing devices. The plurality of scheduling queues include: a plurality of CPU schedulers, wherein each CPU scheduler is assigned to a specific CPU and a specific scheduling queue; and a plurality of accelerator schedulers, wherein each accelerator scheduler is assigned to a specific scheduling queue and a specific type of accelerator.
    Type: Grant
    Filed: February 13, 2019
    Date of Patent: August 25, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Unmesh Dutta Bordoloi, Stephen G. Lusko, Stephen N. McKinnie
  • Publication number: 20200257560
    Abstract: A stream manager for managing the distribution of instructions to a plurality of processing devices includes a dispatcher module configured to: receive multiple instruction streams, wherein each instruction stream includes a plurality of requested computations for processing perception data from a perception data source; partition each instruction stream into a plurality of partitions based on type of device to perform a requested computation from the instruction stream; assign a release time and deadline to each partition, and dispatch partition computations to a plurality of scheduling queues to distribute processing of the partition computations amongst the plurality of processing devices. The plurality of scheduling queues include: a plurality of CPU schedulers, wherein each CPU scheduler is assigned to a specific CPU and a specific scheduling queue; and a plurality of accelerator schedulers, wherein each accelerator scheduler is assigned to a specific scheduling queue and a specific type of accelerator.
    Type: Application
    Filed: February 13, 2019
    Publication date: August 13, 2020
    Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Unmesh Dutta Bordoloi, Stephen G. Lusko, Stephen N. McKinnie
  • Publication number: 20200247428
    Abstract: A method for controlling a vehicle includes: receiving, by a controller, route data, wherein the route data is continuously updated while the vehicle is moving, and the vehicle includes a plurality of vehicle operating modes; receiving, by the controller, feature data, wherein the feature data is information about a plurality of features needed for each of the plurality of vehicle operating modes; determining, by the controller, a plurality of ranges for each of the plurality of vehicle operating modes, wherein each of the plurality of ranges is a function of the route data and the feature data for each of the plurality of vehicle operating modes; and commanding, by the controller, a user interface to display a list of range-mode combinations, wherein the list of range-mode combinations includes the plurality of ranges for each of the plurality of vehicle operating modes.
    Type: Application
    Filed: February 5, 2019
    Publication date: August 6, 2020
    Applicant: GM Global Technology Operations LLC
    Inventors: Unmesh Dutta Bordoloi, Shige Wang, Xinyu Du
  • Patent number: 10686988
    Abstract: Examples of techniques for dynamically selecting a batch size used in vehicle camera image processing are disclosed. In one example implementation, a method includes generating, by a processing device, a batch table and a mode table. The method further includes determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images. The method further includes selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle. The method further includes processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.
    Type: Grant
    Filed: April 11, 2018
    Date of Patent: June 16, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Unmesh Dutta Bordoloi, Shige Wang, Stephen G. Lusko, Jinsong Wang
  • Publication number: 20200167887
    Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.
    Type: Application
    Filed: November 26, 2018
    Publication date: May 28, 2020
    Applicant: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Publication number: 20200160125
    Abstract: A signal processing system includes a central processing unit (CPU) in communication with an accelerator, and an instruction scheduler in communication with the accelerator. A first memory device including a first instruction set is configured to operate the accelerator, a second instruction set is configured to operate the CPU, and a second memory device is configured to receive a datafile. The accelerator includes a plurality of processing engines (PEs) and an instruction scheduler, the instruction set includes a plurality of operators, and the instruction scheduler is configured to implement the operators in the accelerator employing the PEs. The CPU employs the operators implemented in the accelerator to analyze the datafile to extract a feature therefrom.
    Type: Application
    Filed: November 16, 2018
    Publication date: May 21, 2020
    Applicant: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Patent number: 10657617
    Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.
    Type: Grant
    Filed: November 26, 2018
    Date of Patent: May 19, 2020
    Assignee: GM Global Technology Operations LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman L. Millett
  • Publication number: 20200134459
    Abstract: In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.
    Type: Application
    Filed: October 25, 2018
    Publication date: April 30, 2020
    Inventors: Shuqing Zeng, Wei Tong, Shige Wang, Roman L. Millett
  • Publication number: 20200134324
    Abstract: Examples of techniques for using fixed-point quantization in deep neural networks are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises quantizing a fixed-point value based on an activation input and a synapse weight. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.
    Type: Application
    Filed: October 25, 2018
    Publication date: April 30, 2020
    Inventors: Shuqing Zeng, Wei Tong, Shige Wang, Roman L. Millett
  • Publication number: 20200065661
    Abstract: Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).
    Type: Application
    Filed: August 21, 2018
    Publication date: February 27, 2020
    Inventors: Wei Tong, Shuqing Zeng, Upali P. Mudalige, Shige Wang
  • Patent number: 10572748
    Abstract: An adaptive parallel imaging processing system in a vehicle is provided. The system may include, but is not limited to, a plurality of processors and a resource management system including, but not limited to, an execution monitor, the execution monitor configured to calculate an average utilization of each of the plurality of processors over a moving window, and a service scheduler controlling a request queue for each of the plurality of processors, the service scheduler scheduling image processing tasks in the respective request queue for the each of the plurality of processors based upon the average utilization of each of the plurality of processors, the capabilities of each of the plurality of processors, and a priority associated with each image processing task, wherein an autonomous vehicle control system is configured to generate the instructions to control the at least one vehicle system based upon the processed image processing tasks.
    Type: Grant
    Filed: December 6, 2017
    Date of Patent: February 25, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Wei Tong, Shuqing Zeng, Roman Millett
  • Patent number: 10558539
    Abstract: Systems and methods are provided for testing a first computer device of a vehicle. A method includes selecting an operational component of the first computer device and selecting a test operation that is configured to utilize an entire capacity of the operational component. The method further includes instructing the first computer device to perform the test operation and to generate a first result. The method further yet includes retrieving a second result of the test operation and comparing the first result of the test operation from the first computer device with the second result. The method further yet includes indicating that the first computer device is faulty based at least in part on a difference between the first result and the second result.
    Type: Grant
    Filed: September 28, 2017
    Date of Patent: February 11, 2020
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventors: Shige Wang, Gurmitsingh M. Banvait, Sidharth Nakra, Shane M. Boehner, Joseph G. D Ambrosio