Patents by Inventor Hujun Bao

Hujun Bao has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11941514
    Abstract: The present disclosure discloses a method for execution of a computational graph in a neural network model and an apparatus thereof, including: creating task execution bodies on a native machine according to a physical computational graph compiled and generated by a deep learning framework, and designing a solution for allocating a plurality of idle memory blocks to each task execution body, so that the entire computational graph participates in deep learning training tasks of different batches of data in a pipelining and parallelizing manner.
    Type: Grant
    Filed: March 29, 2022
    Date of Patent: March 26, 2024
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Hujun Bao, Guang Chen, Lingfang Zeng, Hongcai Cheng, Yong Li, Jian Zhu, Huanbo Zheng
  • Patent number: 11941532
    Abstract: Disclosed is a method for adapting a deep learning framework to a hardware device based on a unified backend engine, which comprises the following steps: S1, adding the unified backend engine to the deep learning framework; S2, adding the unified backend engine to the hardware device; S3, converting a computational graph, wherein the computational graph compiled and generated by the deep learning framework is converted into an intermediate representation of the unified backend engine; S4, compiling the intermediate representation, wherein the unified backend engine compiles the intermediate representation on the hardware device to generate an executable object; S5, running the executable object, wherein the deep learning framework runs the executable object on the hardware device; S6: managing memory of the unified backend engine.
    Type: Grant
    Filed: April 22, 2022
    Date of Patent: March 26, 2024
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Wei Hua, Hujun Bao, Fei Yang
  • Publication number: 20240046570
    Abstract: A drivable implicit three-dimensional human body representation method, which is used for performing dynamic reconstruction by means of optimizing a three-dimensional representation of a drivable model from an input multi-view video.
    Type: Application
    Filed: October 17, 2023
    Publication date: February 8, 2024
    Inventors: Xiaowei ZHOU, Hujun BAO, Sida PENG, Junting DONG
  • Patent number: 11861505
    Abstract: The disclosure discloses a method of executing dynamic graph for neural network computation and the apparatus thereof. The method of executing dynamic graph includes the following steps: S1: constructing and distributing an operator and a tensor; S2: deducing an operator executing process by an operator interpreter; S3: constructing an instruction of a virtual machine at runtime by the operator interpreter; S4: sending the instruction to the virtual machine at runtime by the operator interpreter; S5: scheduling the instruction by the virtual machine; and S6: releasing an executed instruction by the virtual machine. According to the method of executing dynamic graph for neural network computation and the apparatus thereof provided by the disclosure, runtime is abstracted to be the virtual machine, and the virtual machine acquires a sub-graph of each step constructed by a user in real time through the interpreter and schedules, the virtual machines issues, and executes each sub-graph.
    Type: Grant
    Filed: June 6, 2022
    Date of Patent: January 2, 2024
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Hujun Bao, Guang Chen
  • Patent number: 11854172
    Abstract: The present invention discloses a color contrast enhanced rendering method, device and system suitable for an optical see-through head-mounted display. The method includes: (1) acquiring a background environment in real time to obtain a background video and performing Gaussian blur and visual field correction on the video; (2) converting an original rendering color and a processed video color from an RGB color space to a CIELAB color space scaled to a unit sphere range; (3) finding an optimal rendering color based on the original rendering color and the processed video color in the scaled CIELAB space according to a set color difference constraint, a chromaticity saturation constraint, a brightness constraint and a just noticeable difference constraint; and (4) after converting the optimal rendering color back to the RGB space, performing real-time rendering by using the optimal rendering color of the RGB space.
    Type: Grant
    Filed: January 6, 2021
    Date of Patent: December 26, 2023
    Assignee: ZHEJIANG UNIVERSITY
    Inventors: Rui Wang, Hujun Bao, Yunjin Zhang
  • Publication number: 20230410560
    Abstract: Disclosed are a method and apparatus for constructing a three-dimensional data set of a pedestrian re-identification based on a neural radiation field. The method includes the following steps: S1: capturing images of pedestrians to be entered by a group of cameras at different viewing angles; S2: generating a three-dimensional spatial position point set by sampling through camera rays in the scenario, and converting observation directions of the cameras corresponding to the three-dimensional spatial position point set into three-dimensional Cartesian unit vectors; and S3: inputting, into a multi-layer sensor, the three-dimensional spatial position point set and the observation directions converted into the three-dimensional Cartesian unit vectors, to output corresponding densities and colors. The method and apparatus of the present disclosure gives a brand-new method for constructing a pedestrian re-identification data set, and provides a new idea of data set construction.
    Type: Application
    Filed: September 21, 2022
    Publication date: December 21, 2023
    Inventors: Hongsheng WANG, Guang CHEN, Hujun BAO
  • Patent number: 11823053
    Abstract: The disclosure discloses a method of neural network model computation-oriented intermediate representation and apparatus thereof. The method includes the following steps: S1, parsing an input model file so as to acquire topological structure information of a neural network; S2, constructing a logical computation graph; S21, inferring physical layout information of each operator in the logical computation graph; S22, inferring meta attributes of each operator in the logical computation graph; S23, inferring description information of input and output logical tensors of each operator in the logical computation graph; S3, constructing a physical computation graph; S31, generating a physical computation graph, etc.
    Type: Grant
    Filed: April 6, 2022
    Date of Patent: November 21, 2023
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Wei Hua, Weiqiang Jia, Hujun Bao
  • Publication number: 20230353458
    Abstract: The present disclosure provides a neural network computing-oriented modeling method and apparatus for distributed data routing. The method includes the following steps: S1, designing the distributed attribute of a physical tensor: abstracting a mapping relationship between a logic tensor and the physical tensor into three distributed attributes including a broadcast attribute, a scatter attribute and a local reduction attribute; S2, deducing the distributed attribute of an output tensor: specifying the distributed attribute of an input tensor, and then deducing the legal distributed attribute of the output tensor according to the known distributed attribute of the input tensor; and S3, judging, according to the distributed attribute situation, whether an intermediate communication primitive needs to be inserted to obtain the distributed attribute of a local physical tensor.
    Type: Application
    Filed: June 23, 2022
    Publication date: November 2, 2023
    Inventors: Hongsheng WANG, Shuibing HE, Hujun BAO, Guang CHEN
  • Publication number: 20230351145
    Abstract: The present disclosure provides a pipelining and parallelizing graph execution method for neural network model computation and apparatus, and provides a pipelining and parallelizing graph execution method for neural network model computation and apparatus in a deep learning training system. The method includes the graph execution flow in a neural network model computation process and a process of cooperative work of all functional modules. The pipelining and parallelizing graph execution method for neural network model computation includes creating a graph executive on a native machine according to a physical computation graph compiled and generated by a deep learning framework.
    Type: Application
    Filed: June 13, 2022
    Publication date: November 2, 2023
    Inventors: Hongsheng WANG, Bowen TAN, Hujun BAO, Guang CHEN
  • Publication number: 20230351212
    Abstract: The disclosure provides a semi-supervised method and apparatus for public opinion text analysis. The semi-supervised method includes: first acquiring a public opinion data set, and preprocessing the data set; performing a data augmentation algorithm on preprocessed samples to generate data augmented samples; generating category labels for the unlabeled samples in the data set in an unsupervised extraction and clustering manner; calculating similarities of word vector latent semantic spaces and performing linear interpolation operation to generate, according to an operation result, similarity interpolation samples; constructing a final training sample set; adopting a semi-supervised method, inputting the final training sample set into a pre-trained language model to train the model to obtain a classification model; and predicting the test set by using the classification model to obtain a classification result.
    Type: Application
    Filed: June 10, 2022
    Publication date: November 2, 2023
    Inventors: Hongsheng WANG, Qing LIAO, Hujun BAO, Guang CHEN
  • Patent number: 11805025
    Abstract: The present disclosure provides a neural network computing-oriented modeling method and apparatus for distributed data routing. The method includes the following steps: S1, designing the distributed attribute of a physical tensor: abstracting a mapping relationship between a logic tensor and the physical tensor into three distributed attributes including a broadcast attribute, a scatter attribute and a local reduction attribute; S2, deducing the distributed attribute of an output tensor: specifying the distributed attribute of an input tensor, and then deducing the legal distributed attribute of the output tensor according to the known distributed attribute of the input tensor; and S3, judging, according to the distributed attribute situation, whether an intermediate communication primitive needs to be inserted to obtain the distributed attribute of a local physical tensor.
    Type: Grant
    Filed: June 23, 2022
    Date of Patent: October 31, 2023
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Shuibing He, Hujun Bao, Guang Chen
  • Publication number: 20230334334
    Abstract: The disclosure discloses a method of executing dynamic graph for neural network computation and the apparatus thereof. The method of executing dynamic graph includes the following steps: S1: constructing and distributing an operator and a tensor; S2: deducing an operator executing process by an operator interpreter; S3: constructing an instruction of a virtual machine at runtime by the operator interpreter; S4: sending the instruction to the virtual machine at runtime by the operator interpreter; S5: scheduling the instruction by the virtual machine; and S6: releasing an executed instruction by the virtual machine. According to the method of executing dynamic graph for neural network computation and the apparatus thereof provided by the disclosure, runtime is abstracted to be the virtual machine, and the virtual machine acquires a sub-graph of each step constructed by a user in real time through the interpreter and schedules, the virtual machines issues, and executes each sub-graph.
    Type: Application
    Filed: June 6, 2022
    Publication date: October 19, 2023
    Inventors: Hongsheng WANG, Hujun BAO, Guang CHEN
  • Publication number: 20230316651
    Abstract: Disclosed in the present invention is a three-dimensional reconstruction and angle of view synthesis method for a moving human body, which performs reconstruction of a moving human body by optimizing three-dimensional representations of the moving human body from an inputted multi-angle of view video.
    Type: Application
    Filed: June 9, 2023
    Publication date: October 5, 2023
    Inventors: Xiaowei ZHOU, Hujun BAO, Sida PENG
  • Publication number: 20230290099
    Abstract: A method for reconstructing three-dimensional includes the following operations. At least two frames of first key images for current reconstruction are acquired. A first space surrounding visual cones of the at least two frames of the first key images is determined. The first key images are obtained by photographing a to-be-reconstructed target. A first feature map of the first space is determined based on image information in the several frames of the first key images. The first feature map includes first feature information of voxels in the first space. A first reconstruction result of the current reconstruction is obtained based on the first feature map. A second reconstruction result obtained by previous reconstruction is updated based on the first reconstruction result of the current reconstruction.
    Type: Application
    Filed: May 17, 2023
    Publication date: September 14, 2023
    Applicant: Zhejiang SenseTime Technology Development Co., Ltd.
    Inventors: Hujun BAO, Xiaowei ZHOU, Jiaming SUN, Yiming XIE
  • Publication number: 20230274129
    Abstract: The present disclosure discloses a method for execution of a computational graph in a neural network model and an apparatus thereof, including: creating task execution bodies on a native machine according to a physical computational graph compiled and generated by a deep learning framework, and designing a solution for allocating a plurality of idle memory blocks to each task execution body, so that the entire computational graph participates in deep learning training tasks of different batches of data in a pipelining and parallelizing manner.
    Type: Application
    Filed: March 29, 2022
    Publication date: August 31, 2023
    Inventors: Hongsheng WANG, Hujun BAO, Guang CHEN, Lingfang ZENG, Hongcai CHENG, Yong LI, Jian ZHU, Huanbo ZHENG
  • Publication number: 20230267621
    Abstract: Disclosed is a human motion capture method based on unsynchorized videos, which can effectively recover the 3D motion of a target person through multiple unsynchorized videos of the person. In order to utilize multiple unsynchorized videos, the present disclosure provides a video synchronization and motion reconstruction method. The present disclosure is implemented in the following steps: synchronizing multiple videos based on a 3D human pose; performing motion reconstruction based on synchronized videos, modeling the motion difference across different viewpoints by using the low-rank constraint to realize high-precision human motion capture from the plurality of unsynchorized videos. According to the present disclosure, more accurate motion capture is carried out by using multiple unsynchorized videos.
    Type: Application
    Filed: December 21, 2022
    Publication date: August 24, 2023
    Inventors: Hujun BAO, Xiaowei ZHOU, Junting DONG, Qing SHUAI
  • Publication number: 20230259774
    Abstract: The disclosure discloses a method of neural network model computation-oriented intermediate representation and apparatus thereof. The method includes the following steps: S1, parsing an input model file so as to acquire topological structure information of a neural network; S2, constructing a logical computation graph; S21, inferring physical layout information of each operator in the logical computation graph; S22, inferring meta attributes of each operator in the logical computation graph; S23, inferring description information of input and output logical tensors of each operator in the logical computation graph; S3, constructing a physical computation graph; S31, generating a physical computation graph, etc.
    Type: Application
    Filed: April 6, 2022
    Publication date: August 17, 2023
    Inventors: Hongsheng WANG, Wei HUA, Weiqiang JIA, Hujun BAO
  • Patent number: 11714995
    Abstract: Disclosed is a method for distributed type training adaptation and apparatus in a deep learning framework and an AI accelerator card. The method includes the following steps: S1: the deep learning framework supports single-card configuration in a newly added AI accelerator card, and sub-steps thereof are as follows: S11: the deep learning framework supports new hardware; S12: the deep learning framework supports a device thread of the new hardware; S13: the deep learning framework supports a memory operation of the new hardware; and S14: the deep learning framework supports an operator kernel function of the new hardware; S2: the deep learning framework supports multi-card configuration in the newly added AI accelerator card; S3: the deep learning framework supports tensor segmentation and multi-card distribution; and S4: the deep learning framework supports multi-card collective communication in the newly added AI accelerator card.
    Type: Grant
    Filed: May 9, 2022
    Date of Patent: August 1, 2023
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Hujun Bao, Wei Hua, Weiqiang Jia
  • Publication number: 20230177312
    Abstract: Disclosed is a method for distributed type training adaptation and apparatus in a deep learning framework and an AI accelerator card. The method includes the following steps: S1: the deep learning framework supports single-card configuration in a newly added AI accelerator card, and sub-steps thereof are as follows: S11: the deep learning framework supports new hardware; S12: the deep learning framework supports a device thread of the new hardware; S13: the deep learning framework supports a memory operation of the new hardware; and S14: the deep learning framework supports an operator kernel function of the new hardware; S2: the deep learning framework supports multi-card configuration in the newly added AI accelerator card; S3: the deep learning framework supports tensor segmentation and multi-card distribution; and S4: the deep learning framework supports multi-card collective communication in the newly added AI accelerator card.
    Type: Application
    Filed: May 9, 2022
    Publication date: June 8, 2023
    Inventors: Hongsheng WANG, Hujun BAO, Wei HUA, Weiqiang JIA
  • Patent number: 11615247
    Abstract: Disclosed are a labeling method and apparatus for named entity recognition of a legal instrument. The method includes steps: step S1: acquiring a legal text, and transforming the legal text into an index table; step S2: outputting a sentence feature encoding result; step S3: performing training and prediction; step S4: obtaining a set; step S5: obtaining a multi-head score transfer matrix; step S6: obtaining a score transfer matrix corresponding to the legal text; step S7: determining a recognized nested entity; and S8: constructing an entity labeling template by using the recognized nested entity. According to the present disclosure, a user tries to complete recognition of nested entity labeling by changing an input of the BERT model, and a multi-head selection matrix labeling thought of the present disclosure is used to relieve the difficulty in recognizing a long text and a nested entity in an NER task to a larger extent.
    Type: Grant
    Filed: June 2, 2022
    Date of Patent: March 28, 2023
    Assignee: ZHEJIANG LAB
    Inventors: Hongsheng Wang, Hujun Bao, Guang Chen, Chao Ma, Qing Liao