Patents by Inventor Minghai Qin

Minghai Qin has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210297678
    Abstract: A video processing unit can include a non-object-based region-of-interest detection neural network, a threshold selection module and a region-of-interest map generator. The non-object-based region-of-interest detection neural network can be configured to receive a video frame and generate a plurality of candidate non-object-based region-of-interest blocks. The threshold selection module can be configured to receive the plurality of candidate non-object-based region-of-interest blocks and identify a plurality of selected region-of-interest blocks based on a predetermined threshold. The region-of-interest map generator can be configured to receive the selected non-object-based region-of-interest blocks and generate a region-of-interest map.
    Type: Application
    Filed: March 23, 2020
    Publication date: September 23, 2021
    Inventors: Minghai QIN, Sicheng LI, Guanlin WU, Tae Meon BAE, Yen-kuang CHEN
  • Publication number: 20210266496
    Abstract: A system for using decoder information in video super resolution processing. A compressed video buffering module is used for receiving a compressed video stream and a decoder module is used for decoding the compressed video stream into an uncompressed stream and extracting motion vector information from the uncompressed stream. A video super resolution deep neural network processor module is used for processing the uncompressed stream in conjunction with the motion vector information to produce a video super resolution stream. An output buffer module is used for buffering the video super resolution stream for subsequent output.
    Type: Application
    Filed: February 21, 2020
    Publication date: August 26, 2021
    Inventors: Minghai QIN, Tae Meon BAE, Guanlin WU, Yen-kuang CHEN
  • Publication number: 20210263992
    Abstract: Vector-vector multiplication or matrix-matrix multiplication computation on computing systems can include computing a first portion of a vector-vector multiplication product based on a most-significant-bit set of a first vector and a most-significant-bit set of a second vector, and determining if the first portion of the vector-vector multiplication product is less than a threshold. If the first partial vector-vector multiplication product is not less than the threshold, a remaining portion of the vector-vector multiplication product can be computed, and a rectified linear vector-vector multiplication product can be determined for the sum of the first portion of the vector-vector multiplication product and the remaining portion of the vector-vector multiplication product.
    Type: Application
    Filed: February 21, 2020
    Publication date: August 26, 2021
    Inventors: Minghai QIN, Zhibin XIAO, Chunsheng LIU
  • Publication number: 20210266570
    Abstract: Video coding techniques including differential bit rate or quality coding of one or more regions of interest and one or more non-regions of interest based on information including one or more of coordinates of the one or more regions of interest, a target complexity, residual encoder bit data, a requested quality, a difference between the current video data frame and a reconstructed video data frame, a target quality, a requested bit rate, frame target bit allocation and an as encoded bit rate.
    Type: Application
    Filed: February 21, 2020
    Publication date: August 26, 2021
    Inventors: Guanlin WU, Minghai QIN, Tae Meon BAE, Sicheng LI, Yuanwei FANG, Yen-Kuang CHEN
  • Publication number: 20210258588
    Abstract: A method and apparatus for characteristic-based video processing include: in response to receiving a region of a picture of a video sequence, determining a characteristic in the region, the region being independent of other regions of the picture for video coding; determining a class associated with the region based on the characteristic, the class being selected from a plurality of classes; and encoding the region using a parameter set associated with the class, the parameter set being selected from a plurality of parameter sets for video coding at different quality levels.
    Type: Application
    Filed: February 14, 2020
    Publication date: August 19, 2021
    Inventors: Shaolin XIE, Minghai QIN, Yen-kuang CHEN, Tae Meon BAE, Qinggang ZHOU
  • Publication number: 20210240684
    Abstract: The present disclosure relates to a method and an apparatus for representation of a sparse matrix in a neural network. In some embodiments, an exemplary operation unit includes a buffer for storing a representation of a sparse matrix in a neural network, a sparse engine communicatively coupled with the buffer, and a processing array communicatively coupled with the sparse engine. The sparse engine includes circuitry to: read the representation of the sparse matrix from the buffer, the representation comprising a first level bitmap, a second level bitmap, and an element array; decompress the first level bitmap to determine whether a block of the sparse matrix comprises a non-zero element; and in response to the block comprising a non-zero element, decompress the second level bitmap using the element array to obtain the block of the sparse matrix. The processing array includes circuitry to execute the neural network with the sparse matrix.
    Type: Application
    Filed: February 5, 2020
    Publication date: August 5, 2021
    Inventors: Zhibin XIAO, Xiaoxin FAN, Minghai QIN
  • Patent number: 11074318
    Abstract: An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.
    Type: Grant
    Filed: June 25, 2019
    Date of Patent: July 27, 2021
    Assignee: Western Digital Technologies, Inc.
    Inventors: Wen Ma, Pi-Feng Chiu, Minghai Qin, Won Ho Choi, Martin Lueker-Boden
  • Publication number: 20210201110
    Abstract: The present disclosure provides methods, systems, and non-transitory computer readable media for performing inference with a neural network. The systems include one or more processing units configured to instantiate a neural network comprising a bypass switch that is associated with at least two bypass networks, wherein each of the at least two bypass networks have at least one hidden layer, the bypass switch is configured to select a bypass network of the at least two bypass networks to activate, and any non-selected bypass network of the at least two bypass networks is not activated.
    Type: Application
    Filed: December 31, 2019
    Publication date: July 1, 2021
    Inventor: Minghai QIN
  • Publication number: 20210201538
    Abstract: Methods and systems are provided for implementing static channel filtering operations upon image datasets transformed to frequency domain representations, including decoding images of an image dataset to generate a frequency domain representation of the image dataset; discarding coefficient values of one or more particular frequency channels of each image of the image dataset in a frequency domain representation; and transporting the image dataset in a frequency domain representation to one or more special-purpose processor(s). Methods and systems of the present disclosure may enable a filtered image dataset to be input to a second layer of a learning model, bypassing a first layer, or may enable a learning model to be designed with a reduced-size first layer. This may achieve benefits such as reducing computational overhead and time of machine learning training and inference computations, reducing volume of image data input into the learning model, and reducing convergence time.
    Type: Application
    Filed: December 31, 2019
    Publication date: July 1, 2021
    Inventors: Kai Xu, Fei Sun, Minghai Qin, Yen-kuang Chen
  • Publication number: 20210203992
    Abstract: Methods and systems are provided for implementing preprocessing operations and augmentation operations upon image datasets transformed to frequency domain representations, including decoding images of an image dataset to generate a frequency domain representation of the image dataset; performing a resizing operation based on resizing factors on the image dataset in a frequency domain representation; performing a reshaping operation based on reshaping factors on the image dataset in a frequency domain representation; and performing a cropping operation on the image dataset in a frequency domain representation. The methods and systems may further include performing an augmentation operation on the image dataset in a frequency domain representation. Methods and systems of the present disclosure may free learning models from computational overhead caused by transforming image datasets into frequency domain representations.
    Type: Application
    Filed: December 31, 2019
    Publication date: July 1, 2021
    Inventors: Kai Xu, Fei Sun, Minghai Qin, Yen-kuang Chen
  • Publication number: 20210150260
    Abstract: Image data is accessed. The image data includes frequency domain components. A subset of the frequency domain components is selected based on the relative importance of the frequency domain components. Only the subset of the frequency domain components is provided to an accelerator that executes a neural network to perform an artificial intelligence task using the subset of frequency domain components.
    Type: Application
    Filed: November 14, 2019
    Publication date: May 20, 2021
    Inventors: Yuhao WANG, Minghai QIN, Yen-kuang CHEN
  • Publication number: 20210150768
    Abstract: A system for processing encoded image components for artificial intelligence tasks. The system can include one or more compute units, one or more controllers and memory. The one or more controllers can include one or more micro-op schedulers and one or more channel switches. The one or more compute units can be configured to process components of the transformed domain image data according to one or more micro-operations for an artificial intelligence task. The one or more channel switches can be configured to selectively control the transfer of the components of transformed domain image data to the one or more compute units based on one or more gating flags. The one or more channel switches can also be configured to selectively control generation of the one or more micro-operations by the one or more micro-op schedulers based on the one or more gating flags.
    Type: Application
    Filed: November 14, 2019
    Publication date: May 20, 2021
    Inventors: Kai XU, Minghai QIN, Yuhao WANG, Fei SUN, Yen-kuang CHEN, Yuan XIE
  • Publication number: 20210152832
    Abstract: Discrete cosine transformation (DCT) information can be estimated from adjacent blocks of the same frame. DCT information can be estimated from different frames. Motion vectors can be used to track the position of objects in some frames of the video. For example, a stream of encoded frames is received; the encoded frames are entropy decoded and dequantized to produce DCT information for blocks of the frames; and DCT information for a block in a frame is determined using the DCT information produced from the entropy decoding and dequantizing for a different block.
    Type: Application
    Filed: November 14, 2019
    Publication date: May 20, 2021
    Inventors: Minghai QIN, Yen-kuang CHEN, Kai XU, Yuhao WANG, Fei SUN, Yuan XIE
  • Publication number: 20210150265
    Abstract: A system for determining the importance of encoded image components for artificial intelligence tasks includes an image capture or storage unit, a processor and a communication interface. The processor can receive components of transformed domain image data from the one or more image capture or storage units across the communication interface. The processor can be configured to determine the relative importance of the components of the transformed domain image data for an artificial intelligence task.
    Type: Application
    Filed: November 14, 2019
    Publication date: May 20, 2021
    Inventors: Kai XU, Minghai QIN, Yuhao WANG, Fei SUN, Yen-kuang CHEN, Yuan XIE
  • Publication number: 20210142210
    Abstract: Methods and systems are provided for implementing training of learning models, including obtaining a pre-trained weight set for a learning model on a sample dataset and on a first loss function; selecting at least two tasks having heterogeneous features to be computed by a reference model; obtaining a reference dataset for the at least two tasks; designating a second loss function for feature embedding between the heterogeneous features of the at least two tasks; training the learning model on the first loss function and training the reference model on the second loss function, in turn; and updating the weight set based on a feature embedding learned by the learning model and a feature embedding learned by the reference model, in turn. Methods and systems of the present disclosure may alleviate computational overhead incurred by executing the learning model and loading different weight sets at a central network of the model.
    Type: Application
    Filed: November 11, 2019
    Publication date: May 13, 2021
    Inventors: Chao Cheng, Xiaoxin Fan, Minghai Qin, Yuan Xie
  • Patent number: 10999592
    Abstract: The systems and methods are configured to efficiently and effectively determine or find an estimated optimal encoding parameter set. In one embodiment, a video encoding parameter set estimation method comprises: performing an offline encoding parameter set characteristic prediction process that determines an estimate of a candidate encoding parameter set characteristic; and performing an encoding parameter set search process that identifies a predicted or estimated optimal video encoding parameter set. The encoding parameter set search process can include applying a constraint to the candidate encoding parameter set characteristic; and determining if candidate encoding parameter set meets an objective, wherein the determining is performed if the constraint is satisfied. The candidate encoding parameter set characteristic can be an estimated encoding time of the candidate encoding parameter set. The objective can be the best video quality out of a plurality of candidate encoding parameter sets.
    Type: Grant
    Filed: April 23, 2020
    Date of Patent: May 4, 2021
    Assignee: Alibaba Group Holding Limited
    Inventors: Tae Meon Bae, Minghai Qin, Yen-kuang Chen, Guanlin Wu, Sicheng Li
  • Publication number: 20210117500
    Abstract: Systems and methods for reducing the impact of defects within a crossbar memory array when performing multiplication operations in which multiple control lines are concurrently selected are described. A group of memory cells within the crossbar memory array may be controlled by a local word line that is controlled by a local word line gating unit that may be configured to prevent the local word line from being biased to a selected word line voltage during an operation; the local word line may instead be set to a disabling voltage during the operation such that the memory cell currents through the group of memory cells are eliminated. If a defect has caused a short within one of the memory cells of the group of memory cells, then the local word line gating unit may be programmed to hold the local word line at the disabling voltage during multiplication operations.
    Type: Application
    Filed: June 26, 2020
    Publication date: April 22, 2021
    Applicant: SanDisk Technologies LLC
    Inventors: Minghai Qin, Pi-Feng Chiu, Wen Ma, Won Ho Choi
  • Publication number: 20210117499
    Abstract: Systems and methods for reducing the impact of defects within a crossbar memory array when performing multiplication operations in which multiple control lines are concurrently selected are described. A group of memory cells within the crossbar memory array may be controlled by a local word line that is controlled by a local word line gating unit that may be configured to prevent the local word line from being biased to a selected word line voltage during an operation; the local word line may instead be set to a disabling voltage during the operation such that the memory cell currents through the group of memory cells are eliminated. If a defect has caused a short within one of the memory cells of the group of memory cells, then the local word line gating unit may be programmed to hold the local word line at the disabling voltage during multiplication operations.
    Type: Application
    Filed: October 17, 2019
    Publication date: April 22, 2021
    Applicant: SanDisk Technologies LLC
    Inventors: Minghai Qin, Pi-Feng Chiu, Wen Ma, Won Ho Choi
  • Publication number: 20210110234
    Abstract: The present disclosure provides methods and systems for executing a neural network. The method includes: receiving a plurality of input vectors of input data; generating, among the plurality of input vectors, an estimation value associated with a subset of an input vector based on a weight vector of the activation function; determining whether the estimation value associated with the subset of the input vector satisfies a threshold condition; and determining an output of the activation function based on the estimation value.
    Type: Application
    Filed: October 11, 2019
    Publication date: April 15, 2021
    Inventors: Minghai QIN, Chunsheng LIU
  • Publication number: 20200374534
    Abstract: AI-assisted programmable hardware video codec is disclosed. According to certain embodiments, a video processing apparatus includes a programmable hardware encoder configured to execute an encoding process on a plurality of input video frames. The video processing apparatus further includes a controller coupled with the programmable hardware encoder. The controller is configured to execute a set of instructions to cause the video processing apparatus to: determine first information of the plurality of input video frames, and adjust the encoding process based on the first information.
    Type: Application
    Filed: May 6, 2020
    Publication date: November 26, 2020
    Inventors: Yen-kuang CHEN, Lingjie XU, Minghai QIN, Ping CHEN, Xinyang YU, Qinggang ZHOU