Patents by Inventor Hongxu Yin
Hongxu Yin has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240127067Abstract: Systems and methods are disclosed for improving natural robustness of sparse neural networks. Pruning a dense neural network may improve inference speed and reduces the memory footprint and energy consumption of the resulting sparse neural network while maintaining a desired level of accuracy. In real-world scenarios in which sparse neural networks deployed in autonomous vehicles perform tasks such as object detection and classification for acquired inputs (images), the neural networks need to be robust to new environments, weather conditions, camera effects, etc. Applying sharpness-aware minimization (SAM) optimization during training of the sparse neural network improves performance for out of distribution (OOD) images compared with using conventional stochastic gradient descent (SGD) optimization. SAM optimizes a neural network to find a flat minimum: a region that both has a small loss value, but that also lies within a region of low loss.Type: ApplicationFiled: August 31, 2023Publication date: April 18, 2024Inventors: Annamarie Bair, Hongxu Yin, Pavlo Molchanov, Maying Shen, Jose Manuel Alvarez Lopez
-
Publication number: 20240119291Abstract: Machine learning is a process that learns a neural network model from a given dataset, where the model can then be used to make a prediction about new data. In order to reduce the size, computation, and latency of a neural network model, a compression technique can be employed which includes model sparsification. To avoid the negative consequences of pruning a fully pretrained neural network model and on the other hand of training a sparse model in the first place without any recovery option, the present disclosure provides a dynamic neural network model sparsification process which allows for recovery of previously pruned parts to improve the quality of the sparse neural network model.Type: ApplicationFiled: May 30, 2023Publication date: April 11, 2024Inventors: Jose M. Alvarez Lopez, Pavlo Molchanov, Hongxu Yin, Maying Shen, Lei Mao, Xinglong Sun
-
TECHNIQUES FOR HETEROGENEOUS CONTINUAL LEARNING WITH MACHINE LEARNING MODEL ARCHITECTURE PROGRESSION
Publication number: 20240119361Abstract: One embodiment of a method for training a first machine learning model having a different architecture than a second machine learning model includes receiving a first data set, performing one or more operations to generate a second data set based on the first data set and the second machine learning model, wherein the second data set includes at least one feature associated with one or more tasks that the second machine learning model was previously trained to perform, and performing one or more operations to train the first machine learning model based on the second data set and the second machine learning model.Type: ApplicationFiled: July 6, 2023Publication date: April 11, 2024Inventors: Hongxu YIN, Wonmin BYEON, Jan KAUTZ, Divyam MADAAN, Pavlo MOLCHANOV -
Publication number: 20240096115Abstract: Landmark detection refers to the detection of landmarks within an image or a video, and is used in many computer vision tasks such emotion recognition, face identity verification, hand tracking, gesture recognition, and eye gaze tracking. Current landmark detection methods rely on a cascaded computation through cascaded networks or an ensemble of multiple models, which starts with an initial guess of the landmarks and iteratively produces corrected landmarks which match the input more finely. However, the iterations required by current methods typically increase the training memory cost linearly, and do not have an obvious stopping criteria. Moreover, these methods tend to exhibit jitter in landmark detection results for video. The present disclosure improves current landmark detection methods by providing landmark detection using an iterative neural network.Type: ApplicationFiled: September 7, 2023Publication date: March 21, 2024Inventors: Pavlo Molchanov, Jan Kautz, Arash Vahdat, Hongxu Yin, Paul Micaelli
-
Publication number: 20230394781Abstract: Vision transformers are deep learning models that employ a self-attention mechanism to obtain feature representations for an input image. To date, the configuration of vision transformers has limited the self-attention computation to a local window of the input image, such that short-range dependencies are modeled in the output. The present disclosure provides a vision transformer that captures global context, and that is therefore able to model long-range dependencies in its output.Type: ApplicationFiled: December 16, 2022Publication date: December 7, 2023Applicant: NVIDIA CorporationInventors: Ali Hatamizadeh, Hongxu Yin, Jan Kautz, Pavlo Molchanov
-
Publication number: 20230186077Abstract: One embodiment of the present invention sets forth a technique for executing a transformer neural network. The technique includes computing a first set of halting scores for a first set of tokens that has been input into a first layer of the transformer neural network. The technique also includes determining that a first halting score included in the first set of halting scores exceeds a threshold value. The technique further includes in response to the first halting score exceeding the threshold value, causing a first token that is included in the first set of tokens and is associated with the first halting score not to be processed by one or more layers within the transformer neural network that are subsequent to the first layer.Type: ApplicationFiled: June 15, 2022Publication date: June 15, 2023Inventors: Hongxu YIN, Jan KAUTZ, Jose Manuel ALVAREZ LOPEZ, Arun MALLYA, Pavlo MOLCHANOV, Arash VAHDAT
-
Publication number: 20230080247Abstract: A vision transformer is a deep learning model used to perform vision processing tasks such as image recognition. Vision transformers are currently designed with a plurality of same-size blocks that perform the vision processing tasks. However, some portions of these blocks are unnecessary and not only slow down the vision transformer but use more memory than required. In response, parameters of these blocks are analyzed to determine a score for each parameter, and if the score falls below a threshold, the parameter is removed from the associated block. This reduces a size of the resulting vision transformer, which reduces unnecessary memory usage and increases performance.Type: ApplicationFiled: December 14, 2021Publication date: March 16, 2023Inventors: Hongxu Yin, Huanrui Yang, Pavlo Molchanov, Jan Kautz
-
Publication number: 20230077258Abstract: Apparatuses, systems, and techniques are presented to simplify neural networks. In at least one embodiment, one or more portions of one or more neural networks are cause to be removed based, at least in part, on one or more performance metrics of the one or more neural networks.Type: ApplicationFiled: August 10, 2021Publication date: March 9, 2023Inventors: Maying Shen, Pavlo Molchanov, Hongxu Yin, Lei Mao, Jianna Liu, Jose Manuel Alvarez Lopez
-
Patent number: 11521068Abstract: According to various embodiments, a method for generating one or more optimal neural network architectures is disclosed. The method includes providing an initial seed neural network architecture and utilizing sequential phases to synthesize the neural network until a desired neural network architecture is reached. The phases include a gradient-based growth phase and a magnitude-based pruning phase.Type: GrantFiled: October 25, 2018Date of Patent: December 6, 2022Assignee: THE TRUSTEES OF PRINCETON UNIVERSITYInventors: Xiaoliang Dai, Hongxu Yin, Niraj K. Jha
-
Publication number: 20220292360Abstract: Apparatuses, systems, and techniques to remove one or more nodes of a neural network. In at least one embodiment, one or more nodes of a neural network are removed, based on, for example, whether the one or more nodes are likely to affect performance of the neural network.Type: ApplicationFiled: March 15, 2021Publication date: September 15, 2022Inventors: Maying Shen, Pavlo Molchanov, Hongxu Yin, Jose Manuel Alvarez Lopez
-
Publication number: 20220284283Abstract: Apparatuses, systems, and techniques to invert a neural network. In at least one embodiment, one or more neural network layers are inverted and, in at least one embodiment, loaded in reverse order.Type: ApplicationFiled: March 8, 2021Publication date: September 8, 2022Inventors: Hongxu Yin, Pavlo Molchanov, Jose Manuel Alvarez Lopez, Xin Dong
-
Publication number: 20220284232Abstract: Apparatuses, systems, and techniques to identify one or more images used to train one or more neural networks. In at least one embodiment, one or more images used to train one or more neural networks are identified, based on, for example, one or more labels of one or more objects within the one or more images.Type: ApplicationFiled: March 1, 2021Publication date: September 8, 2022Inventors: Hongxu Yin, Arun Mallya, Arash Vahdat, Jose Manuel Alvarez Lopez, Jan Kautz, Pavlo Molchanov
-
Publication number: 20220240864Abstract: According to various embodiments, a machine-learning based system for diabetes analysis is disclosed. The system includes one or more processors configured to interact with a plurality of wearable medical sensors (WMSs). The processors are configured to receive physiological data from the WMSs and demographic data from a user interface. The processors are further configured to train at least one neural network based on a grow-and-prune paradigm to generate at least one diabetes inference model. The neural network grows at least one of connections and neurons based on gradient information and prunes away at least one of connections and neurons based on magnitude information. The processors are also configured to output a diabetes-based decision by inputting the received physiological data and demographic data into the generated diabetes inference model.Type: ApplicationFiled: June 16, 2020Publication date: August 4, 2022Applicant: The Trustees of Princeton UniversityInventors: Hongxu Yin, Bilal Mukadam, Xiaoliang Dai, Niraj K. Jha
-
Publication number: 20220222534Abstract: According to various embodiments, a method for generating a compact and accurate neural network for a dataset that has initial data and is updated with new data is disclosed. The method includes performing a first training on the initial neural network architecture to create a first trained neural network architecture. The method additionally includes performing a second training on the first trained neural network architecture when the dataset is updated with new data to create a second trained neural network architecture. The second training includes growing one or more connections for the new data based on a gradient of each connection, growing one or more connections for the new data and the initial data based on a gradient of each connection, and iteratively pruning one or more connections based on a magnitude of each connection until a desired neural network architecture is achieved.Type: ApplicationFiled: March 20, 2020Publication date: July 14, 2022Applicant: The Trustees of Princeton UniversityInventors: Xiaoliang DAI, Hongxu YIN, Niraj K. JHA
-
Publication number: 20210182683Abstract: According to various embodiments, a method for generating one or more optimal neural network architectures is disclosed. The method includes providing an initial seed neural network architecture and utilizing sequential phases to synthesize the neural network until a desired neural network architecture is reached. The phases include a gradient-based growth phase and a magnitude-based pruning phase.Type: ApplicationFiled: October 25, 2018Publication date: June 17, 2021Applicant: The Trustees of Princeton UniversityInventors: Xiaoliang DAI, Hongxu YIN, Niraj K. JHA
-
Publication number: 20210133540Abstract: According to various embodiments, a method for generating an optimal hidden-layer long short-term memory (H-LSTM) architecture is disclosed. The H-LSTM architecture includes a memory cell and a plurality of deep neural network (DNN) control gates enhanced with hidden layers. The method includes providing an initial seed H-LSTM architecture, training the initial seed H-LSTM architecture by growing one or more connections based on gradient information and iteratively pruning one or more connections based on magnitude information, and terminating the iterative pruning when training cannot achieve a predefined accuracy threshold.Type: ApplicationFiled: March 14, 2019Publication date: May 6, 2021Applicant: The Trustees of Princeton UniversityInventors: Xiaoliang DAI, Hongxu YIN, Niraj K. JHA
-
Publication number: 20200382286Abstract: According to various embodiments, an Internet of Things (IoT) sensor architecture is disclosed. The architecture includes one or more IoT sensor components configured to capture data and one or more processors configured to analyze the captured data. The processors include a data compression module configured to convert received data into compressed data, a machine learning module configured to extract features from the received data and classify the extracted features, and an encryption/hashing module configured to encrypt and ensure integrity of resulting data from the machine learning module or the received data.Type: ApplicationFiled: January 10, 2019Publication date: December 3, 2020Inventors: Ayten Ozge Akamandor, Hongxu Yin, Niraj Jha
-
Publication number: 20200320395Abstract: A method for training a machine learning model, including acquiring an initial machine learning model, updating features of the initial machine learning model, updating dimension of the initial machine learning model based on the updated features of the initial machine learning model and one or more latency hysteresis points obtained based on a hardware profile of an accelerator configured to perform machine learning operations, and generating a final machine learning model based on the updated dimensions.Type: ApplicationFiled: April 3, 2019Publication date: October 8, 2020Inventors: Hongxu YIN, Weifeng ZHANG, Guoyang CHEN
-
Publication number: 20190374160Abstract: According to various embodiments, a hierarchical health decision support system (HDSS) configured to receive data from one or more wearable medical sensors (WMSs) is disclosed. The system includes a clinical decision support system, which includes a diagnosis engine configured to generate diagnostic suggestions based on the data received from the WMSs. The HDSS is configured with a plurality of tiers to sequentially model general healthcare from daily health monitoring, initial clinical checkup, detailed clinical examination, and postdiagnostic treatment.Type: ApplicationFiled: December 29, 2017Publication date: December 12, 2019Applicant: The Trustees of Princeton UniversityInventors: Hongxu Yin, Niraj K. Jha