Network Structures Patents (Class 382/158)
  • Patent number: 11967150
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel processing of video frames using neural networks. One of the methods includes receiving a video sequence comprising a respective video frame at each of a plurality of time steps; and processing the video sequence using a video processing neural network to generate a video processing output for the video sequence, wherein the video processing neural network includes a sequence of network components, wherein the network components comprise a plurality of layer blocks each comprising one or more neural network layers, wherein each component is active for a respective subset of the plurality of time steps, and wherein each layer block is configured to, at each time step at which the layer block is active, receive an input generated at a previous time step and to process the input to generate a block output.
    Type: Grant
    Filed: February 13, 2023
    Date of Patent: April 23, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Simon Osindero, Joao Carreira, Viorica Patraucean, Andrew Zisserman
  • Patent number: 11954591
    Abstract: This application provides a method of generating a description for a picture set performed at a computer device, and a storage medium. The method includes: acquiring a picture set to be processed; performing picture feature extraction on each picture in the picture set to acquire a picture feature corresponding to the picture, and forming a picture feature sequence corresponding to the picture set by using the picture features corresponding to the pictures; performing scene feature extraction on a picture feature sub-sequence corresponding to each scene in the picture feature sequence to acquire a scene feature corresponding to the scene, and forming a scene feature sequence corresponding to the picture set by using the scene features corresponding to the scenes; and generating textual description information of the picture set according to the picture feature sequence and the scene feature sequence that correspond to the picture set.
    Type: Grant
    Filed: August 11, 2020
    Date of Patent: April 9, 2024
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Bairui Wang, Lin Ma, Wei Liu
  • Patent number: 11915474
    Abstract: Techniques and apparatus for analyzing visual content using a visual transformer are described. An example technique includes generating a first set of tokens based on a visual content item. Each token in the first set of tokens is associated with a regional feature from a different region of a plurality of regions of the visual content item. A second set of tokens is generated based on the visual content item. Each token in the second set of tokens is associated with a local feature from one of the plurality of regions of the visual content item. At least one feature map is generated for the visual content item, based on analyzing the first set of tokens and the second set of tokens separately using a hierarchical vision transformer. At least one vision task is performed based on the at least one feature map.
    Type: Grant
    Filed: May 31, 2022
    Date of Patent: February 27, 2024
    Assignee: International Business Machines Corporation
    Inventors: Richard Chen, Rameswar Panda, Quanfu Fan
  • Patent number: 11854211
    Abstract: Training a multi-object tracking model includes: generating a plurality of training images based at least on scene generation information, each training image comprising a plurality of objects to be tracked; generating, for each training image, original simulated data based at least on the scene generation information, the original simulated data comprising tag data for a first object; locating, within the original simulated data, tag data for the first object, based on at least an anomaly alert (e.g., occlusion alert, proximity alert, motion alert) associated with the first object in the first training image; based at least on locating the tag data for the first object, modifying at least a portion of the tag data for the first object from the original simulated data, thereby generating preprocessed training data from the original simulated data; and training a multi-object tracking model with the preprocessed training data to produce a trained multi-object tracker.
    Type: Grant
    Filed: January 26, 2022
    Date of Patent: December 26, 2023
    Assignee: Microsoft Technology Licensing, LLC.
    Inventors: Ishani Chakraborty, Jonathan C. Hanzelka, Lu Yuan, Pedro Urbina Escos, Thomas M. Soemo
  • Patent number: 11810341
    Abstract: A computer-implemented method of identifying filters for use in determining explainability of a trained neural network. The method comprises obtaining a dataset comprising the input image and an annotation of an input image, the annotation indicating at least one part of the input image which is relevant for inferring classification of the input image, determining an explanation filter set by iteratively: selecting a filter of the plurality of filters; adding the filter to the explanation filter set; computing an explanation heatmap for the input image by resizing and combining an output of each filter in the explanation filter set to obtain the explanation heatmap, the explanation heatmap having a spatial resolution of the input image; and computing a similarity metric by comparing the explanation heatmap to the annotation of the input image; until the similarity metric is greater than or equal to a similarity threshold; and outputting the explanation filter set.
    Type: Grant
    Filed: January 8, 2021
    Date of Patent: November 7, 2023
    Assignee: ROBERT BOSCH GMBH
    Inventor: Andres Mauricio Munoz Delgado
  • Patent number: 11790523
    Abstract: A device receives an input image of a portion of a patient's body, and applies the input image to a feature extraction model, the feature extraction model comprising a trained machine learning model that is configured to generate an output that comprises, for each respective location of a plurality of locations in the input image, an indication that the input image contains an object of interest that is indicative of a presence of a disease state at the respective location. The device applies the output of the feature extraction model to a diagnostic model, the diagnostic model comprising a trained machine learning model that is configured to output a diagnosis of a disease condition in the patient based on the output of the feature extraction model. The device outputs the determined diagnosis of a disease condition in the patient obtained from the diagnostic model.
    Type: Grant
    Filed: October 30, 2018
    Date of Patent: October 17, 2023
    Assignee: Digital Diagnostics Inc.
    Inventors: Meindert Niemeijer, Ryan Amelon, Warren Clarida, Michael D. Abramoff
  • Patent number: 11768912
    Abstract: A computer-implemented method according to one embodiment includes receiving historical two-dimensional (2D) multivariate time series data; transforming the historical 2D multivariate time series data into a three-dimensional (3D) temporal tensor; training one or more deep volumetric 3D convolutional neural networks (CNNs), utilizing the 3D temporal tensor; and predicting future values for additional multivariate time series data, utilizing the one or more trained deep volumetric 3D CNNs.
    Type: Grant
    Filed: July 12, 2019
    Date of Patent: September 26, 2023
    Assignee: International Business Machines Corporation
    Inventors: Mu Qiao, Yuya Jeremy Ong, Divyesh Jadav
  • Patent number: 11756284
    Abstract: An apparatus of labeling for object detection according to an embodiment of the present disclosure includes an image selector that determines a plurality of labeling target images from among a plurality of unlabeled images, and determines a labeling order of the plurality of labeling target images, a feedback obtainer that obtains label inspection information on the plurality of labeling target images from a user, and a model trainer that learns the label inspection information input from the user by using the labeling target images, obtains a pseudo label for supervised learning based on a learning result using the label inspection information, and re-determines the labeling order of the labeling target images based on the pseudo label.
    Type: Grant
    Filed: February 17, 2021
    Date of Patent: September 12, 2023
    Assignee: SAMSUNG SDS CO., LTD.
    Inventors: Seung Ho Shin, Ji Hoon Kim, Se Won Woo, Kyoung Jin Oh, Seong Won Park
  • Patent number: 11741734
    Abstract: Aspects of the disclosure provide for mechanisms for identification of blocks of associated words in documents using neural networks. A method of the disclosure includes obtaining a plurality of words of a document, the document having a first block of associated words, determining a plurality of vectors representative of the plurality of words, processing the plurality of vectors using a first neural network to obtain a plurality of recalculated vectors having values based on the plurality of vectors, determining a plurality of association values corresponding to a connections between at least two words of the document, and identifying, using the plurality of recalculated vectors and the plurality of association values, the first block of associated symbol sequences.
    Type: Grant
    Filed: January 13, 2022
    Date of Patent: August 29, 2023
    Assignee: ABBYY Development Inc.
    Inventor: Stanislav Semenov
  • Patent number: 11741723
    Abstract: A system and method for performing intersection scenario retrieval that includes receiving a video stream of a surrounding environment of an ego vehicle. The system and method also include analyzing the video stream to trim the video stream into video clips of an intersection scene associated with the travel of the ego vehicle. The system and method additionally include annotating the ego vehicle, dynamic objects, and their motion paths that are included within the intersection scene with action units that describe an intersection scenario. The system and method further include retrieving at least one intersection scenario based on a query of an electronic dataset that stores a combination of action units to operably control a presentation of at least one intersection scenario video clip that includes the at least one intersection scenario.
    Type: Grant
    Filed: June 29, 2020
    Date of Patent: August 29, 2023
    Assignee: HONDA MOTOR CO., LTD.
    Inventors: Yi-Ting Chen, Nakul Agarwal, Behzad Dariush, Ahmed Taha
  • Patent number: 11735174
    Abstract: A method of training a natural language neural network comprises obtaining at least one constituency span; obtaining a training video input; applying a multi-modal transform to the video input, thereby generating a transformed video input; comparing the at least one constituency span and the transformed video input using a compound Probabilistic Context-Free Grammar (PCFG) model to match the at least one constituency span with corresponding portions of the transformed video input; and using results from the comparison to learn a constituency parser.
    Type: Grant
    Filed: October 12, 2022
    Date of Patent: August 22, 2023
    Assignee: TENCENT AMERICA LLC
    Inventor: Linfeng Song
  • Patent number: 11734347
    Abstract: A video retrieval method and apparatus, a device and a storage medium are provided. The method comprises the following steps: acquiring a comparison video clip from a video library according to the duration of a to-be-tested video (S110); determining the similarity between the to-be-tested video and the comparison video clip by a target spatio-temporal neural network, a spatio-temporal convolutional layer of the target spatio-temporal neural network being configured to be capable of performing two-dimensional convolution and temporal dimension information processing, respectively (S120); and traversing the video library, and outputting a retrieval result according to the similarity (S130).
    Type: Grant
    Filed: July 7, 2021
    Date of Patent: August 22, 2023
    Assignee: LYNXI TECHNOLOGIES CO., LTD.
    Inventors: Zhenzhi Wu, Yaolong Zhu
  • Patent number: 11694082
    Abstract: An all-optical Diffractive Deep Neural Network (D2NN) architecture learns to implement various functions or tasks after deep learning-based design of the passive diffractive or reflective substrate layers that work collectively to perform the desired function or task. This architecture was successfully confirmed experimentally by creating 3D-printed D2NNs that learned to implement handwritten classifications and lens function at the terahertz spectrum. This all-optical deep learning framework can perform, at the speed of light, various complex functions and tasks that computer-based neural networks can implement, and will find applications in all-optical image analysis, feature detection and object classification, also enabling new camera designs and optical components that can learn to perform unique tasks using D2NNs. In alternative embodiments, the all-optical D2NN is used as a front-end in conjunction with a trained, digital neural network back-end.
    Type: Grant
    Filed: June 17, 2022
    Date of Patent: July 4, 2023
    Assignee: THE REGENTS OF THE UNIVERSITY OF CALIFORNIA
    Inventors: Aydogan Ozcan, Yair Rivenson, Xing Lin, Deniz Mengu, Yi Luo
  • Patent number: 11580736
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel processing of video frames using neural networks. One of the methods includes receiving a video sequence comprising a respective video frame at each of a plurality of time steps; and processing the video sequence using a video processing neural network to generate a video processing output for the video sequence, wherein the video processing neural network includes a sequence of network components, wherein the network components comprise a plurality of layer blocks each comprising one or more neural network layers, wherein each component is active for a respective subset of the plurality of time steps, and wherein each layer block is configured to, at each time step at which the layer block is active, receive an input generated at a previous time step and to process the input to generate a block output.
    Type: Grant
    Filed: January 7, 2019
    Date of Patent: February 14, 2023
    Assignee: DeepMind Technologies Limited
    Inventors: Simon Osindero, Joao Carreira, Viorica Patraucean, Andrew Zisserman
  • Patent number: 11557026
    Abstract: A technique for detecting a glitch in an image is provided. The technique includes providing an image to a plurality of individual classifiers to generate a plurality of individual classifier outputs and providing the plurality of individual classifier outputs to an ensemble classifier to generate a glitch classification.
    Type: Grant
    Filed: September 23, 2020
    Date of Patent: January 17, 2023
    Assignees: Advanced Micro Devices, Inc., ATI Technologies ULC
    Inventors: Nicholas Malaya, Max Kiehn, Stanislav Ivashkevich
  • Patent number: 11521605
    Abstract: A method of training a natural language neural network comprises obtaining at least one constituency span; obtaining a training video input; applying a multi-modal transform to the video input, thereby generating a transformed video input; comparing the at least one constituency span and the transformed video input using a compound Probabilistic Context-Free Grammar (PCFG) model to match the at least one constituency span with corresponding portions of the transformed video input; and using results from the comparison to learn a constituency parser.
    Type: Grant
    Filed: February 22, 2021
    Date of Patent: December 6, 2022
    Assignee: TENCENT AMERICA LLC
    Inventor: Linfeng Song
  • Patent number: 11496635
    Abstract: An information processing system acquires, using a reading device, a read image from an original on which a handwritten character is written; acquires, based on the read image, a partial image that is a partial region of the read image and a binarized image that expresses the partial image by two tones; performs learning of a learning model based on learning data that uses the partial image as a correct answer image and the binarized image as an input image; acquires print data including a font character; generates conversion image data including a gradation character obtained by inputting the font character to the learning model; and causes an image forming device to form an image based on the generated conversion image data.
    Type: Grant
    Filed: April 12, 2021
    Date of Patent: November 8, 2022
    Assignee: Canon Kabushiki Kaisha
    Inventor: Keisui Okuma
  • Patent number: 11475658
    Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.
    Type: Grant
    Filed: February 18, 2021
    Date of Patent: October 18, 2022
    Assignee: ANCESTRY.COM OPERATIONS INC.
    Inventors: Mohammad K. Ebrahimpour, Yen-Yun Yu, Jiayun Li, Jack Reese, Azadeh Moghtaderi
  • Patent number: 11475660
    Abstract: One embodiment facilitates recognizing parts of a vehicle. A convolution module is configured to generate a convolution feature map of a vehicle image. A region proposal module is configured to determine, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part. A classification module is configured to determine a class and a bounding box of a vehicle part corresponding to a proposed region based on a feature of the proposed region. A conditional random field module is configured to optimize classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions. A reporting module is configured to generate a result which indicates a list including an insurance claim item and corresponding damages based on the optimized classes and bounding boxes of the vehicle parts.
    Type: Grant
    Filed: August 29, 2019
    Date of Patent: October 18, 2022
    Assignee: Advanced New Technologies Co., LTD.
    Inventor: Qingpei Guo
  • Patent number: 11429103
    Abstract: Systems, methods, devices, and techniques for planning travel of an autonomous robot. A system identifies one or more obstacles that are located in proximity of at least a portion of a planned route for the autonomous robot. For each obstacle, the system: (i) determines a semantic class of the obstacle, including selecting the semantic class from a library that defines a set of multiple possible semantic classes for obstacles, and (ii) selects a planning policy for the obstacle that corresponds to the semantic class of the obstacle. The system can generate a trajectory along the at least the portion of the planned route using the selected planning policies. The robot can then initiate travel according to the trajectory.
    Type: Grant
    Filed: March 27, 2020
    Date of Patent: August 30, 2022
    Assignee: X Development LLC
    Inventors: David Millard, Mikael Persson
  • Patent number: 11417007
    Abstract: A method of controlling an electronic apparatus includes acquiring an image and depth information of the acquired image; inputting the acquired image into a neural network model trained to acquire information on objects included in the acquired image; acquiring an intermediate feature value output by an intermediate layer of the neural network model; identifying a feature area for at least one object among the objects included in the acquired image based on the intermediate feature value; and acquiring distance information between the electronic apparatus and the at least one object based on the feature area for the at least one object and the depth information.
    Type: Grant
    Filed: November 6, 2020
    Date of Patent: August 16, 2022
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Seongmin Kang, Heungwoo Han
  • Patent number: 11403511
    Abstract: In some implementations at an electronic device, training a dual EDNN includes defining a data structure of attributes corresponding to defined parts of a task, processing a first instance of an input using a first EDNN to produce a first output while encoding a first set of the attributes in a first latent space, and processing a second instance of the input using a second EDNN to produce a second output while encoding attribute differences from attribute averages in a second latent space. The device then determines a second set of the attributes based on the attribute differences and the attribute averages. The device then adjusts parameters of the first and second EDNNs based on comparing the first instance of the input to the first output, the second instance of the input to the second output, and the first set of attributes to the second set of attributes.
    Type: Grant
    Filed: July 18, 2019
    Date of Patent: August 2, 2022
    Assignee: Apple Inc.
    Inventors: Peter Meier, Tanmay Batra
  • Patent number: 11386340
    Abstract: The inventive concepts herein relate to performing block retrieval on a block to be processed of a urine sediment image. The method comprises: using a plurality of decision trees to perform block retrieval on the block to be processed, wherein each of the plurality of decision trees comprises a judgment node and a leaf node, and the judgment node judges the block to be processed to make it reach the leaf node by using a block retrieval feature in a block retrieval feature set to form a block retrieval result at the leaf node, and at least two decision trees in the plurality of decision trees are different in structures thereof and/or judgments performed by the judgment nodes thereof by using the block retrieval feature; and integrating the block retrieval results of the plurality of decision trees so as to form a final block retrieval result.
    Type: Grant
    Filed: August 7, 2020
    Date of Patent: July 12, 2022
    Assignee: SIEMENS HEALTHCARE DIAGNOSTIC INC.
    Inventors: Tian Shen, Juan Xu, XiaoFan Zhang
  • Patent number: 11308402
    Abstract: Artificial intelligence (AI) techniques that map disallowed states and enable access to those states under certain conditions through a search algorithm are disclosed. In other words, scenario boundaries may be crossed by jumping from one scenario that is less desirable or even has no solution to another scenario that is more desirable.
    Type: Grant
    Filed: July 2, 2020
    Date of Patent: April 19, 2022
    Assignee: THE AEROSPACE CORPORATION
    Inventors: Terence Yeoh, Nehal Desai
  • Patent number: 11270084
    Abstract: A method for generating a human-like response to a voice or text command includes receiving an input sequence of words and processing the input sequence of words to generate a trigger word that is indicative of a desired nature of the human-like response. The method further includes encoding a neural network using the trigger word and generating the human-like response using an output of the neural network. The method enables implementation of voice command functionality in various types of devices with only a small amount of training data.
    Type: Grant
    Filed: October 12, 2018
    Date of Patent: March 8, 2022
    Assignee: Johnson Controls Tyco IP Holdings LLP
    Inventors: Viswanath Ramamurti, Young M. Lee
  • Patent number: 11232299
    Abstract: Aspects of the disclosure provide for mechanisms for identification of blocks of associated words in documents using neural networks. A method of the disclosure includes obtaining a plurality of words of a document, the document having a first block of associated words, determining a plurality of vectors representative of the plurality of words, processing the plurality of vectors using a first neural network to obtain a plurality of recalculated vectors having values based on the plurality of vectors, determining a plurality of association values corresponding to a connections between at least two words of the document, and identifying, using the plurality of recalculated vectors and the plurality of association values, the first block of associated symbol sequences.
    Type: Grant
    Filed: December 18, 2019
    Date of Patent: January 25, 2022
    Assignee: ABBYY Production LLC
    Inventor: Stanislav Semenov
  • Patent number: 11200056
    Abstract: A parallel union control device includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to cause each of the plurality of arithmetic units included in an parallel computer including a vector register to: successively compare input elements of a pair of input sets to undergo union processing, the pair being stored in an input operand register in the vector register; select one of the input elements as an output element of an output set, based on a comparison result; and store the output element into an output operand register in the vector register; shift a pointer pointing to the input element; load the input sets into the input operand register from a memory; store the output sets into the memory from the output operand register; and determine whether union processing performed in parallel is ended.
    Type: Grant
    Filed: February 5, 2019
    Date of Patent: December 14, 2021
    Assignee: NEC CORPORATION
    Inventors: Harumichi Yokoyama, Takuya Araki, Haoran Li
  • Patent number: 11164087
    Abstract: There is provided a system including a non-transitory memory storing an executable code and a hardware processor executing the executable code to receive an input sentence including a first predicate and at least a first argument depending from the first predicate, identify the first predicate, identify the first argument based on the first predicate, apply a dependency multiplication to determine a semantic role of the first argument based on the first predicate, and assign the first argument to an argument cluster including one or more similar arguments based on the semantic role of the first argument.
    Type: Grant
    Filed: May 20, 2016
    Date of Patent: November 2, 2021
    Assignee: Disney Enterprises, Inc.
    Inventors: Boyang Li, Yi Luan
  • Patent number: 11132605
    Abstract: Cardinal sine function used as an activation function for a hierarchical classifier. Application of a sine function, or a cardinal sine function, for hierarchical classification of a subject within subject matter domains and sub-domains. Hierarchical classification or multi-level classification is improved through use of the cardinal sine function or even standard sine function. Some embodiments of the present invention focus on the usage of cardinal sine function as activation function and how to apply this cardinal sine function for hierarchical classification of a subject. Some embodiments include a technique by which hierarchical classification or multi-level classification can benefit from application of a cardinal sine function.
    Type: Grant
    Filed: November 20, 2017
    Date of Patent: September 28, 2021
    Assignee: International Business Machines Corporation
    Inventor: Abhishek Dasgupta
  • Patent number: 11113836
    Abstract: Embodiments of object detection method, device, apparatus and a computer-readable storage medium are provided. The method can include: obtaining an enclosing frame of a target object in an input image; according to the enclosing frame, determining a reference frame from a predetermined candidate frame set comprising a plurality of candidate frames; generating a size-related feature according to a size of the reference frame and a size of the enclosing frame; and detecting an object in the input image by applying the size-related feature in a machine learning model. In an embodiment of the present application, the object detection is performed by using a feature related to an object size, that is, the prediction criterion related to the object size is added to an original feature in a machine learning model, thereby further improving the accuracy of the object detection.
    Type: Grant
    Filed: July 15, 2019
    Date of Patent: September 7, 2021
    Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
    Inventors: Xuehui Wang, Ming Li, Tian Xia
  • Patent number: 11113840
    Abstract: A method configured to implemented on at least one image processing device for detecting objects in images includes obtaining an image including an object. The method also includes generating one or more feature vectors related to the image based on a first convolutional neural network, wherein the one or more feature vectors includes a plurality of parameters. The method further includes determining the position of the object based on at least one of the plurality of parameters. The method still further includes determining a category associated with the object based on at least one the plurality of parameters.
    Type: Grant
    Filed: June 28, 2019
    Date of Patent: September 7, 2021
    Assignee: ZHEJIANG DAHUA TECHNOLOGY CO., LTD.
    Inventors: Xin Ye, Songlin Yang
  • Patent number: 10977842
    Abstract: A method for processing a multi-directional X-ray computed tomography (CT) image using a neural network and an apparatus therefor are provided. The method includes receiving a predetermined number of multi-directional X-ray CT data and reconstructing an image for the multi-directional X-ray CT data using a neural network learned in each of an image domain and a sinogram domain.
    Type: Grant
    Filed: June 4, 2019
    Date of Patent: April 13, 2021
    Assignee: Korea Advanced Institute of Science and Technology
    Inventors: JongChul Ye, Yoseob Han
  • Patent number: 10964061
    Abstract: A deep neural network (DNN) system learns a map representation for estimating a camera position and orientation (pose). The DNN is trained to learn a map representation corresponding to the environment, defining positions and attributes of structures, trees, walls, vehicles, etc. The DNN system learns a map representation that is versatile and performs well for many different environments (indoor, outdoor, natural, synthetic, etc.). The DNN system receives images of an environment captured by a camera (observations) and outputs an estimated camera pose within the environment. The estimated camera pose is used to perform camera localization, i.e., recover the three-dimensional (3D) position and orientation of a moving camera, which is a fundamental task in computer vision with a wide variety of applications in robot navigation, car localization for autonomous driving, device localization for mobile navigation, and augmented/virtual reality.
    Type: Grant
    Filed: May 12, 2020
    Date of Patent: March 30, 2021
    Assignee: NVIDIA Corporation
    Inventors: Jinwei Gu, Samarth Manoj Brahmbhatt, Kihwan Kim, Jan Kautz
  • Patent number: 10948966
    Abstract: The disclosed computer-implemented method may include (i) identifying an artificial neural network that processes each input to the artificial neural network in a fixed number of operations, (ii) performing an analysis on the artificial neural network to determine an execution metric that represents the fixed number of operations performed by the artificial neural network to process each input, (iii) determining a quality-of-service metric for an executing system that executes the artificial neural network, and (iv) optimizing power consumption of the executing system by configuring, based on the execution metric and the quality-of-service metric, a processing throughput of at least one physical processor of the executing system, thereby causing the executing system to execute the artificial neural network at a rate that satisfies the quality-of-service metric while limiting the power consumption of the executing system. Various other methods, systems, and computer-readable media are also disclosed.
    Type: Grant
    Filed: March 7, 2018
    Date of Patent: March 16, 2021
    Assignee: Facebook, Inc.
    Inventors: Nadav Rotem, Abdulkadir Utku Diril, Mikhail Smelyanskiy, Jong Soo Park
  • Patent number: 10943105
    Abstract: A system and method for invoice field detection and parsing includes the steps of extracting character bounding blocks using optical character recognition (OCR) or digital character extraction (DCE), enhancing the image quality, analyzing the document layout based on imaging techniques, detecting the invoice field based on the machine learning techniques, and parsing the invoice field value based on the content information.
    Type: Grant
    Filed: September 24, 2019
    Date of Patent: March 9, 2021
    Assignee: The Neat Company, Inc.
    Inventors: Shuo Chen, Venkataraman Pranatharthiharan
  • Patent number: 10879946
    Abstract: Methods and systems for processing a noisy time series input to detect a signal, generate a de-noising mask, and/or output a de-noised time series output are provided. The input is transformed into one or more datagrams, such as real and imaginary time-frequency grams. The datagrams are stacked and provided as first and second channel inputs to a neural network. A neural network is trained to detect signals within the input. Alternatively or in addition, the network is trained to generate a de-noise mask, and/or to output a de-noised time series output. Implementation of the method and systems can include the use of multiple deep neural networks (DNNs), such as convolutional neural networks (CNN's), that are provided with inputs in the form of RF spectrograms. Embodiments of the present disclosure can be applied to various RF devices, such as communication devices, including but not limited to multiple inputs multiple output (MIMO) devices and 5G communication system devices, and RADAR devices.
    Type: Grant
    Filed: October 30, 2019
    Date of Patent: December 29, 2020
    Assignee: Ball Aerospace & Technologies Corp.
    Inventor: James M. Shima
  • Patent number: 10843835
    Abstract: Medicine packaging apparatuses and methods for accurately determining a remaining sheet amount of a medicine packaging sheet are described. The apparatus includes: a roll support section to which a core tube of a medicine packaging sheet roll is attached; a sensor disposed in the roll support section for outputting a count value according to a rotation amount; a wireless reader-writer unit for writing information to a core tube IC tag and reading said information; an information generation section for generating information to be written to the core tube IC tag; a remaining sheet amount estimation section for estimating a current amount of remaining sheet based on the information and dimensional information of the core tube; and a controller which selectively performs an operation if a reference time-point count value is not yet written to the core tube IC tag and another operation if the count value is already written thereto.
    Type: Grant
    Filed: January 30, 2019
    Date of Patent: November 24, 2020
    Assignee: YUYAMA MFG. CO., LTD.
    Inventors: Katsunori Yoshina, Tomohiro Sugimoto, Noriyoshi Fujii
  • Patent number: 10832124
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating descriptions of input images. One of the methods includes obtaining an input image; processing the input image using a first neural network to generate an alternative representation for the input image; and processing the alternative representation for the input image using a second neural network to generate a sequence of a plurality of words in a target natural language that describes the input image.
    Type: Grant
    Filed: August 12, 2019
    Date of Patent: November 10, 2020
    Assignee: Google LLC
    Inventors: Samy Bengio, Oriol Vinyals, Alexander Toshkov Toshev, Dumitru Erhan
  • Patent number: 10796221
    Abstract: Systems and techniques for facilitating a deep learning architecture for automated image feature extraction are presented. In one example, a system includes a machine learning component. The machine learning component generates learned imaging output regarding imaging data based on a convolutional neural network that receives the imaging data. The machine learning component also performs a plurality of sequential and/or parallel downsampling and upsampling of the imaging data associated with convolutional layers of the convolutional neural network.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: October 6, 2020
    Assignee: General Electric Company
    Inventors: Min Zhang, Gopal Biligeri Avinash
  • Patent number: 10796152
    Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.
    Type: Grant
    Filed: September 17, 2019
    Date of Patent: October 6, 2020
    Assignee: ANCESTRY.COM OPERATIONS INC.
    Inventors: Mohammad K. Ebrahimpour, Yen-Yun Yu, Jiayun Li, Jack Reese, Azadeh Moghtaderi
  • Patent number: 10776691
    Abstract: Methods, systems and apparatuses, including computer programs encoded on computer storage media, are provided for learning or optimizing an indirect encoding of a mapping from digitally-encoded input arrays to digitally-encoded output arrays, with numerous technical advantages in terms of efficiency and effectiveness.
    Type: Grant
    Filed: June 23, 2016
    Date of Patent: September 15, 2020
    Assignee: Uber Technologies, Inc.
    Inventors: Zoubin Ghahramani, Gary Marcus
  • Patent number: 10713533
    Abstract: Provided are neural network-based image processing method and apparatus, and a computer-readable storage medium. The image processing method includes: inputting an image into an optimized neural network; extracting, by the optimized neural network, image features of the image; and outputting the image features, wherein the optimized neural network is obtained by performing a first optimization process on at least one sub-layer in a pre-trained initial neural network, each sub-layer of the at least one sub-layer includes a convolutional layer, and the first optimization process comprises: for each sub-layer of the at least one sub-layer, determining one or more channels to be removed from a filter of the convolutional layer and removing said one or more channels, and optimizing parameters of remaining channels in the filter of the convolutional layer, so that error of output features of each optimized sub-layer is minimized.
    Type: Grant
    Filed: July 12, 2018
    Date of Patent: July 14, 2020
    Assignee: MEGVII (BEIJING) TECHNOLOGY CO., LTD.
    Inventors: Xiangyu Zhang, Yihui He
  • Patent number: 10692244
    Abstract: A deep neural network (DNN) system learns a map representation for estimating a camera position and orientation (pose). The DNN is trained to learn a map representation corresponding to the environment, defining positions and attributes of structures, trees, walls, vehicles, etc. The DNN system learns a map representation that is versatile and performs well for many different environments (indoor, outdoor, natural, synthetic, etc.). The DNN system receives images of an environment captured by a camera (observations) and outputs an estimated camera pose within the environment. The estimated camera pose is used to perform camera localization, i.e., recover the three-dimensional (3D) position and orientation of a moving camera, which is a fundamental task in computer vision with a wide variety of applications in robot navigation, car localization for autonomous driving, device localization for mobile navigation, and augmented/virtual reality.
    Type: Grant
    Filed: September 20, 2018
    Date of Patent: June 23, 2020
    Assignee: NVIDIA Corporation
    Inventors: Jinwei Gu, Samarth Manoj Brahmbhatt, Kihwan Kim, Jan Kautz
  • Patent number: 10679119
    Abstract: The present disclosure provides for generating a spiking neural network. Generating a spiking neural network can include determining that a first input fan-in from a plurality of input neurons to each of a plurality of output neurons is greater than a threshold, generating a plurality of intermediate neurons based on a determination that the first input fan-in is greater than the threshold, and coupling the plurality of intermediate neurons to the plurality of input neurons and the plurality of output neurons, wherein each of the plurality of intermediate neurons has a second input fan-in that is less than the first input fan-in and each of the plurality of output neurons has a third input fan-in that is less than the first input fan-in.
    Type: Grant
    Filed: March 24, 2017
    Date of Patent: June 9, 2020
    Assignee: INTEL CORPORATION
    Inventors: Arnab Paul, Narayan Srinivasa
  • Patent number: 10681059
    Abstract: The present invention provides a target centric monitoring of a network enabling a likelihood score for the existence of an attack to be calculated. The score is calculated by monitoring a plurality of network nodes for a range of symptoms. Detected symptoms are then profiled using a classical Bayesian-based framework such that a node score is calculated for every node. The node scores are compared against reference activity so as to identify deviations from reference activity. The reference activity may comprise peer analysis comparing the node scores against the nodes scores or per nodes and discord analysis comparing the node score of a particular node against historical behaviour. Based on the deviations, the method can enable the calculation of a likelihood of suspicious activity for each node.
    Type: Grant
    Filed: May 25, 2016
    Date of Patent: June 9, 2020
    Assignee: CyberOwl Limited
    Inventors: Siraj Ahmed Shaikh, Harsha Kumara Kalutarage
  • Patent number: 10628683
    Abstract: Methods, systems, and techniques for sharing layers between convolutional neural networks (CNNs). A data processing system may include a first and a second CNN that share a first group of layers. The first CNN may include the first group of layers in series with a second group of layers and be configured such that data for the first CNN is input to the first group of layers. The second CNN may include the first group of layers in series with a third group of layers and be configured such that data for the second CNN is also input to the first group of layers.
    Type: Grant
    Filed: December 5, 2017
    Date of Patent: April 21, 2020
    Assignee: Avigilon Corporation
    Inventors: Moussa Doumbouya, Lu He, Mahesh Saptharishi
  • Patent number: 10606269
    Abstract: Systems, methods, devices, and techniques for planning travel of an autonomous robot. A system identifies one or more obstacles that are located in proximity of at least a portion of a planned route for the autonomous robot. For each obstacle, the system: (i) determines a semantic class of the obstacle, including selecting the semantic class from a library that defines a set of multiple possible semantic classes for obstacles, and (ii) selects a planning policy for the obstacle that corresponds to the semantic class of the obstacle. The system can generate a trajectory along the at least the portion of the planned route using the selected planning policies. The robot can then initiate travel according to the trajectory.
    Type: Grant
    Filed: December 19, 2017
    Date of Patent: March 31, 2020
    Assignee: X Development LLC
    Inventors: David Millard, Mikael Persson
  • Patent number: 10586312
    Abstract: A method for video compression through image processing and object detection, based on images or a digital video stream of images, to enhance and isolate frequency domain signals representing content to be identified, and decrease or ignore frequency domain noise with respect to the content. A digital image or sequence of digital images defined in a spatial domain are obtained. One or more pairs of sparse zones are selected, each pair generating a selected feature, each zone defined by two sequences of spatial data. The selected features are transformed into frequency domain data. The transfer function, shape and direction of the frequency domain data are varied for each zone, thus generating a normalized complex vector for each feature. The normalized complex vectors are then combined to define a model of the content to be identified.
    Type: Grant
    Filed: February 19, 2018
    Date of Patent: March 10, 2020
    Assignee: COGISEN S.R.L.
    Inventor: Christiaan Erik Rijnders
  • Patent number: 10575699
    Abstract: A system for enabling spot cleaning includes a mobile computing device and a mobile cleaning robot. The mobile computing device includes at least one camera configured to capture images of an environment, and at least one data processor configured to (a) establish, based at least in part on first information provided by the at least one image sensor, a coordinate system in the environment, (b) determine, based at least in part on second information provided by the at least one camera, a first set of coordinates of a region at a first location, (c) determine, based at least in part on third information provided by the at least one camera, a second set of coordinates of a mobile cleaning robot at a second location, (d) send the first set of coordinates and second set of coordinates, or coordinates of the first location relative to the second location, to the mobile cleaning robot, and (e) send an instruction to the mobile cleaning robot to request the mobile cleaning robot to travel to the first location.
    Type: Grant
    Filed: January 5, 2018
    Date of Patent: March 3, 2020
    Assignee: iRobot Corporation
    Inventors: Angela Bassa, Husain Al-Mohssen
  • Patent number: 10540537
    Abstract: A method for content detection based on images or a digital video stream of images, to enhance and isolate frequency domain signals representing content to be identified, and decrease or ignore frequency domain noise with respect to the content. A digital image or sequence of digital images defined in a spatial domain are obtained. One or more pairs of sparse zones are selected, each pair generating a feature, each zone defined by two sequences of spatial data. The selected features are transformed into frequency domain data. The transfer function, shape and direction of the frequency domain data are varied for each zone, thus generating a normalized complex vector for each feature. The normalized complex vectors are then combined to define a model of the content to be identified.
    Type: Grant
    Filed: February 19, 2018
    Date of Patent: January 21, 2020
    Assignee: COGISEN S.R.L.
    Inventor: Christiaan Erik Rijnders