Patents Issued in June 21, 2022
  • Patent number: 11367163
    Abstract: Artistic styles extracted from source images may be applied to target images to generate stylized images and/or video sequences. The extracted artistic styles may be stored as a plurality of layers in one or more neural networks, which neural networks may be further optimized, e.g., via the fusion of various elements of the networks' architectures. The artistic style may be applied to the target images and/or video sequences using various optimization methods, such as the use of a first version of the neural network by a first processing device at a first resolution to generate one or more sets of parameters (e.g., scaling and/or biasing parameters), which parameters may then be mapped for use by a second version of the neural network by a second processing device at a second resolution. Analogous multi-processing device and/or multi-network solutions may also be applied to other complex image processing tasks for increased efficiency.
    Type: Grant
    Filed: February 19, 2020
    Date of Patent: June 21, 2022
    Assignee: Apple Inc.
    Inventors: Francesco Rossi, Marco Zuliani, Bartlomiej W. Rymkowski, Albert Antony, Brian P. Keene, Xiaojin Shi
  • Patent number: 11367164
    Abstract: An eyewear camera as a system is introduced that includes many subsystems. These subsystems include: scene imaging, control methods, tracking a user's eye, methods and techniques to increase the resolution of an image captured by a scene camera, methods to create a viewfinder, and methods to capture an image of a user's eye while simultaneously projecting an image into the user's eye. Such an eyewear will allow a user to capture a scene effortlessly, select an object within the scene, get extra information about the object via a digital personal assistant, or even modify a subset of the scene. Specific applications will be discussed that include visual aid for people with low vision and upgrading existing security cameras via proposed single camera super resolution techniques.
    Type: Grant
    Filed: November 18, 2019
    Date of Patent: June 21, 2022
    Inventor: Masoud Vaziri
  • Patent number: 11367165
    Abstract: In one embodiment, a method includes receiving a first frame associated with a first time and one or more second frames of a video having a resolution lower than a target resolution, wherein each second frame is associated with a second time prior to the first time, generating a first feature map for the first frame and one or more second feature maps for the one or more second frames, up-sampling the first feature map and the one or more second feature maps to the target resolution, warping each of the up-sampled second feature maps according to a motion estimation between the associated second time and the first time, and generating a reconstructed frame having the target resolution corresponding to the first frame by using a machine-learning model to process the up-sampled first feature map and the one or more up-sampled and warped second feature maps.
    Type: Grant
    Filed: September 30, 2020
    Date of Patent: June 21, 2022
    Assignee: Facebook Technologies, LLC.
    Inventors: Lei Xiao, Salah Eddine Nouri, Douglas Robert Lanman, Anton S Kaplanyan, Alexander Jobe Fix, Matthew Steven Chapman
  • Patent number: 11367166
    Abstract: In one implementation, a method includes obtaining an image. The method includes correcting color aberration of the image comprising, for a particular pixel of the image by determining one or more chromatic characteristic values for the particular pixel, determining a likelihood that the particular pixel exhibits color aberration based on the one or more chromatic characteristic values for the particular pixel, generating a filtered version of the particular pixel by filtering the particular pixel using pixels in a neighborhood surrounding the particular pixel, and generating a color-corrected version of the particular pixel based on the likelihood that the particular pixel exhibits color aberration and the filtered version of the particular pixel.
    Type: Grant
    Filed: June 19, 2020
    Date of Patent: June 21, 2022
    Assignee: APPLE INC.
    Inventors: Zahra Sadeghipoor Kermani, Sheng Lin, Emmanuel Piuze Phaneuf, Ryan Joseph Dunn, Joachim Michael Deguara, Farhan A. Baqai
  • Patent number: 11367167
    Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.
    Type: Grant
    Filed: August 21, 2020
    Date of Patent: June 21, 2022
    Assignee: Apple Inc.
    Inventors: Jim C. Chou, Yun Gong
  • Patent number: 11367168
    Abstract: In an image processing apparatus, an image acquirer is configured to acquire a captured image from a camera mounted to a vehicle. A histogram generator is configured to generate a histogram representing a relationship between a luminance and a pixel count of the captured image. The pixel count is a number of pixels in the captured image. An image corrector is configured to generate a corrective tone-curve for correcting the relationship between the luminance and the pixel count of the captured image based on the histogram of the captured image, and correct the captured image using the corrective tone-curve to generate a corrected image. The corrective tone-curve is a tone-curve representing a relationship between input and output values of luminance.
    Type: Grant
    Filed: October 29, 2020
    Date of Patent: June 21, 2022
    Assignee: DENSO CORPORATION
    Inventor: Yuzuru Koshisaka
  • Patent number: 11367169
    Abstract: Disclosed herein is a method for processing a picture of an automatic driving vehicle. The method for processing a picture of an automatic driving vehicle according to an embodiment of the present disclosure includes obtaining a first image from a camera while a vehicle on which the camera is driving, generating a prediction signal for predicting a sign of occurrence of White-Out from the first image, when a quantity of light which is a threshold brightness or higher from the first image, storing the first image during a first period when the prediction signal is generated, and predicting and correcting a second image based on the first image stored during the first period and a correction model, when the second image in which the White-Out occurs in the first image is detected on a specific time during the first period, and the correction model is a result of learning the second image by processing high dynamic range (HDR) image.
    Type: Grant
    Filed: April 26, 2019
    Date of Patent: June 21, 2022
    Assignee: LG ELECTRONICS INC.
    Inventor: Jonghoon Chae
  • Patent number: 11367170
    Abstract: A component mounter images suction states of components picked up by a suction nozzle, processes images of the suction states of the components to recognize the suction state of the component, and stores the images in a storage device with linking production information related to the components. An inspection machine images a mounted state of each component on a board, processes an image of mounted state of each component to recognize the mounted state of each component, and inspects whether a mounting error occurred for each component based on the recognition result. The inspection machine determines that a mounting error occurred for any of the components the image of the mounted state of the component for which a mounting error was determined to have occurred and a searched image of the suction state of the component are displayed on the display monitor in a comparative manner.
    Type: Grant
    Filed: December 1, 2016
    Date of Patent: June 21, 2022
    Assignee: FUJI CORPORATION
    Inventors: Kenji Sugiyama, Shuichiro Kito, Hiroshi Oike
  • Patent number: 11367171
    Abstract: The present invention belongs to the technical field of constructional engineering safety detection, and provides a method for detecting hollowing of an exterior wall finish layer. The method for detecting a defect of a facade finish layer of a building proposed by the present invention can realize remote detection in the on-site detection operation and avoid high-altitude hazardous operation of personnel, and is simple and convenient in operation. In data analysis and processing, the method not only can overcome the subjectivity and the instability caused by manual color identification of a temperature image, but also can realize batch processing of image analysis through software or conduct real-time detection and analysis.
    Type: Grant
    Filed: March 13, 2018
    Date of Patent: June 21, 2022
    Assignee: DALIAN UNIVERSITY OF TECHNOLOGY
    Inventors: Zongshan Wang, Zhiqiang Zhai, Lin Duanmu, Zhengjian Shao, Siyang Yue
  • Patent number: 11367172
    Abstract: There is described a method of labeling wood products moved across a handling area of a production line. The method generally has, using a camera, generating an image representing a wood product moving across the handling area at a first moment in time; using a controller, determining coordinates of the wood product at the first moment in time based on the image; and anticipating coordinates of the wood product at a second moment in time assuming an incremental movement of the wood product at a given speed from the determined coordinates of the wood product at the first moment in time; and using a light projector, projecting, at the second moment in time, a wood product label at the anticipated coordinates of the wood product at the second moment in time.
    Type: Grant
    Filed: April 21, 2020
    Date of Patent: June 21, 2022
    Assignee: TIMBER TECHNOLOGY INC.
    Inventors: Marc Voyer, Marc-Antoine Paquet
  • Patent number: 11367173
    Abstract: According to examples, an apparatus may include a processor and a non-transitory computer readable medium that the processor may execute to determine whether a token is to be printed printed onto the medium, following printing onto the medium in a printing system, cause a scan bar in a media feed path of the printing system to capture an image of the printed medium. The processor may also determine, from the captured image of the printed medium, whether the token was properly printed onto the printed medium and, based on a determination that the token was not properly printed onto the printed medium, output an indication and/or an instruction corresponding to the token being improperly printed.
    Type: Grant
    Filed: December 6, 2018
    Date of Patent: June 21, 2022
    Assignee: Hewlett-Packard Development Company, L.P.
    Inventors: Darrel D. Cherry, Kristofer D. Provencio, Mark Q. Shaw
  • Patent number: 11367174
    Abstract: A transfer system includes a first imaging and detection system that detects an object by using a captured first image, an industrial machine that operates by using information detected by the first imaging and detection system, a second imaging and detection system disposed upstream of the first imaging and detection system to detect an object by using a captured second image, and an operation terminal that outputs, to the second imaging and detection system, an instruction instructing that detection be performed, and to change imaging and detection system data used in the second imaging and detection system. The first imaging and detection system includes a receiving unit to receive the imaging and detection system data from the second imaging and detection system. The second imaging and detection system includes a transmitting unit to transmit the imaging and detection system data to the first imaging and detection system.
    Type: Grant
    Filed: June 23, 2020
    Date of Patent: June 21, 2022
    Assignee: FANUC CORPORATION
    Inventor: Masaru Oda
  • Patent number: 11367175
    Abstract: Provided is an in-vehicle stereo camera which enables the continuation of automated driving if failures occur during imaging. A pair of captured images 301, 302, which is captured by a pair of imaging units so that both contain an overlapping region, are acquired (S201), it is determined whether an abnormal region is present in at least one of the pair of captured images, and if an abnormal region is present, the degree of impact from the abnormal region on an object recognition process of an object recognition unit is diagnosed (S202), and the processing content of the object recognition process is updated according to the degree of impact (S203).
    Type: Grant
    Filed: March 11, 2019
    Date of Patent: June 21, 2022
    Assignee: HITACHI ASTEMO, LTD.
    Inventors: Shouji Muramatsu, Takeshi Shima, Makoto Kudo
  • Patent number: 11367176
    Abstract: A commodity management device includes a camera interface circuit connectable to a camera to capture an image of one of commodities arranged in a row along a first direction from a back plate to a front of the shelf. The device includes a sensor interface circuit connectable to a sensor measuring a distance to the commodities in the row. A processor is configured to detect a change in the distance measured by the sensor, identify the commodities based on the image, then acquire a thickness of each commodity in the row, and calculate the number of the commodities removed from the shelf based on the distance change and the thickness of the commodities.
    Type: Grant
    Filed: January 14, 2021
    Date of Patent: June 21, 2022
    Assignee: TOSHIBA TEC KABUSHIKI KAISHA
    Inventors: Takayuki Sawada, Tetsuya Nobuoka
  • Patent number: 11367177
    Abstract: A checking device for a label, with a processor for the detection of the label and a layout unit for the creation of a layout of the label from layout data, wherein the checking device comprises a data converter device configured in such a way that it generates test data from layout data in memory in order to control the processor. A data converter device for use in a checking device, wherein the data converter device is configured in such a way that it generates test data from layout data in memory for checking the label.
    Type: Grant
    Filed: June 18, 2013
    Date of Patent: June 21, 2022
    Assignee: Wipotec Wiege-und Positioniersysteme GmbH
    Inventors: Hans Michael Czerny, Karl Georg Burri
  • Patent number: 11367178
    Abstract: A method for constructing an image includes: defining a foreground area associated with an object in an original image; identifying a plurality of contour points defining a contour of the object; for each of the contour points, obtaining a reference contour point set that includes at least one reference contour point on each of two sides of the contour point; obtaining a plurality of characteristic lines, each associated with the reference contour point set and defined by an end point obtained from the contour points; and aligning the end points on one side to form a straight edge and making the characteristic lines adjoin each other side by side, so as to construct a reconstructed image.
    Type: Grant
    Filed: December 17, 2020
    Date of Patent: June 21, 2022
    Assignee: V5 TECHNOLOGIES CO., LTD.
    Inventors: Sheng-Chih Hsu, Chien-Ting Chen
  • Patent number: 11367179
    Abstract: Systems and techniques for determining degree of motion using machine learning to improve medical image quality are presented. In one example, a system generates, based on a convolutional neural network, motion probability data indicative of a probability distribution of a degree of motion for medical imaging data generated by a medical imaging device. The system also determines motion score data for the medical imaging data based on the motion probability data.
    Type: Grant
    Filed: September 30, 2019
    Date of Patent: June 21, 2022
    Assignees: GE Precision Healthcare LLC, Partners Healthcare System, Inc., The General Hospital Corporation, The Bringham and Women's Hospital, Inc.
    Inventors: Jason Polzin, Bernardo Bizzo, Bradley Wright, John Kirsch, Pamela Schaefer
  • Patent number: 11367180
    Abstract: Systems and methods for classification and mutation prediction from histopathology images using deep learning. In particular, the examples described herein utilize lung cancer as an example, in particular adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) as two subtypes of lung cancer to identify and distinguish between.
    Type: Grant
    Filed: December 11, 2019
    Date of Patent: June 21, 2022
    Assignee: New York University
    Inventors: Nicolas Coudray, Paolo Santiago Ocampo, Andre L. Moreira, Narges Razavian, Aristotelis Tsirigos
  • Patent number: 11367181
    Abstract: Systems and methods for ossification center detection (OCD) and bone age assessment (BAA) may be provided. The method may include obtaining a bone age image of a subject. The method may include generating a normalized bone age image by preprocessing the bone age image. The method may include determining, based on the normalized bone age image, positions of a plurality of ossification centers using an ossification center localization (OCL) model. The method may include estimating, based on the normalized bone age image and information related to the positions of the plurality of ossification centers, a bone age of the subject using a bone age assessment (BAA) model.
    Type: Grant
    Filed: December 28, 2019
    Date of Patent: June 21, 2022
    Assignee: SHANGHAI UNITED IMAGING INTELLIGENCE CO., LTD.
    Inventors: Minqing Zhang, Qin Liu, Dijia Wu, Yiqiang Zhan, Xiang Sean Zhou
  • Patent number: 11367182
    Abstract: A medical image processing device includes: a memory; and a processor including hardware.
    Type: Grant
    Filed: December 30, 2019
    Date of Patent: June 21, 2022
    Assignee: SONY OLYMPUS MEDICAL SOLUTIONS INC.
    Inventor: Taihei Michihata
  • Patent number: 11367183
    Abstract: An automatic field-of-view (FOV) prescription system is provided. The system includes an FOV prescription computing device that includes at least one processor electrically coupled to at least one memory device. The at least one processor is programmed to receive localizer images that depict an anatomy, and generate masks associated with the localizer images, wherein the masks represent part of the localizer images that depict the anatomy. The at least one processor is also programmed to calculate bounding boxes surrounding the anatomy based on the masks, generate an FOV based on the bounding boxes, and output the FOV.
    Type: Grant
    Filed: February 28, 2020
    Date of Patent: June 21, 2022
    Assignee: GE PRECISION HEALTHCARE LLC
    Inventors: Deepthi S., Jignesh Dholakia, Dheeraj Kulkarni, Rakesh Shrikant Shevde
  • Patent number: 11367184
    Abstract: Pathologists are adopting digital pathology for diagnosis, using whole slide images (WSIs). Explainable AI (xAI) is a new approach to AI that can reveal underlying reasons for its results. As such, xAI can promote safety, reliability, and accountability of machine learning for critical tasks such as pathology diagnosis. HistoMapr provides intelligent xAI guides for pathologists to improve the efficiency and accuracy of pathological diagnoses. HistoMapr can previews entire pathology cases' WSIs, identifies key diagnostic regions of interest (ROIs), determines one or more conditions associated with each ROI, provisionally labels each ROI with the identified conditions, and can triages them. The ROIs are presented to the pathologist in an interactive, explainable fashion for rapid interpretation. The pathologist can be in control and can access xAI analysis via a “why?” interface. HistoMapr can track the pathologist's decisions and assemble a pathology report using suggested, standardized terminology.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: June 21, 2022
    Assignee: SPINTELLX, INC.
    Inventors: Akif Burak Tosun, Srinivas Chakra Chennubhotla, Jeffrey Louis Fine
  • Patent number: 11367185
    Abstract: A fully image-based framework for CT image, or other medical image, quality evaluation and virtual clinical trial using deep-learning techniques is provided. This framework includes deep learning-based noise insertion, lesion insertion, and model observer, which enable efficient, objective, and quantitative image quality evaluation and virtual clinical trial directly performed on patient images.
    Type: Grant
    Filed: July 13, 2020
    Date of Patent: June 21, 2022
    Assignee: Mayo Foundation for Medical Education and Research
    Inventors: Lifeng Yu, Hao Gong, Shuai Leng, Cynthia H. McCollough
  • Patent number: 11367186
    Abstract: The disclosure relates to stent detection and shadow detection in the context of intravascular data sets obtained using a probe such as, for example, and optical coherence tomography probe or an intravascular ultrasound probe.
    Type: Grant
    Filed: July 14, 2020
    Date of Patent: June 21, 2022
    Assignee: LightLab Imaging, Inc.
    Inventors: Sonal Ambwani, Christopher E. Griffin
  • Patent number: 11367187
    Abstract: A method is for detecting respective potential presence of respective different antinuclear antibody fluorescence pattern types on a biological cell substrate including human epithelioma cells, including: acquiring a first image which represents staining of the cell substrate by a first fluorescent dye, and acquiring a second image which represents staining of the cell substrate by a second fluorescent dye, detecting, on the basis of the first image, respective image segments which in each case represent at least one mitotic cell, selecting, on the basis of the detected image segments, subimages of the first image and subimages corresponding thereto of the second image and detecting, on the basis of the selected subimages of the first image and of the selected subimages of the second image, respective actual presence of respective cellular fluorescence pattern types by means of a convolutional neural network.
    Type: Grant
    Filed: July 2, 2020
    Date of Patent: June 21, 2022
    Assignee: EUROIMMUN Medizinische Labordiagnostika AG
    Inventors: Jens Krauth, Christopher Krause, Joern Voigt, Melanie Hahn, Christian Marzahl, Stefan Gerlach
  • Patent number: 11367188
    Abstract: A GAN is trained to process input images and produce a synthetic dental image. The GAN further takes masks as inputs with each image, the masks labeling pixels of the image corresponding to dental features (anatomy and/or treatments). The GAN includes an encoder-decoder with normalization between stages of the decoder according to the masks. A synthetic image and an unpaired dental image is evaluated by a first discriminator of the GAN to obtain a realism estimate. The synthetic image and an unpaired dental image may be processed using a pretrained dental encoder to obtain a perceptual loss. The GAN is trained with the realism estimate and perceptual loss. Utilization may include modifying a mask for an input image to include or exclude a shape of a feature such that the synthetic image includes or excludes a dental feature.
    Type: Grant
    Filed: September 25, 2020
    Date of Patent: June 21, 2022
    Assignee: Retrace Labs
    Inventors: Vasant Kearney, Hamid Hekmatian, Stephen Chan, Ali Sadat
  • Patent number: 11367189
    Abstract: A hierarchical deep-learning object detection framework provides a method for identifying objects of interest in high-resolution, high pixel count images, wherein the objects of interest comprise a relatively a small pixel count when compared to the overall image. The method uses first deep-learning model to analyze the high pixel count images, in whole or as a patchwork, at a lower resolution to identify objects, and a second deep-learning model to analyze the objects at a higher resolution to classify the objects.
    Type: Grant
    Filed: October 16, 2020
    Date of Patent: June 21, 2022
    Assignee: CARNEGIE MELLON UNIVERSITY
    Inventors: Daniel Clymer, Jonathan Cagan, Philip LeDuc
  • Patent number: 11367190
    Abstract: The disclosure herein relates to systems, methods, and devices for medical image analysis, diagnosis, risk stratification, decision making and/or disease tracking. In some embodiments, the systems, devices, and methods described herein are configured to analyze non-invasive medical images of a subject to automatically and/or dynamically identify one or more features, such as plaque and vessels, and/or derive one or more quantified plaque parameters, such as radiodensity, radiodensity composition, volume, radiodensity heterogeneity, geometry, location, and/or the like. In some embodiments, the systems, devices, and methods described herein are further configured to generate one or more assessments of plaque-based diseases from raw medical images using one or more of the identified features and/or quantified parameters.
    Type: Grant
    Filed: July 5, 2021
    Date of Patent: June 21, 2022
    Assignee: CLEERLY, INC.
    Inventors: James K. Min, James P. Earls, Hugo Miguel Rodrigues Marques, Ben Hootnick
  • Patent number: 11367191
    Abstract: Disclosed is a system and a method for adapting a report of nodules in computed tomography (CT) scan image. A CT scan image may be resampled into a plurality of slices. A plurality of region of interests may be identified on each slice using an image processing technique. Subsequently, a plurality of nodules may be detected in each region of interest using the deep learning. Further, a plurality of characteristics associated with each nodule may be identified. The plurality of nodules may be classified into AI-confirmed nodules and AI-probable nodules based on a malignancy score. Further, feedback associated with the AI-confirmed nodules and the AI-probable may be received form a radiologist. Furthermore, data may be adapted based on the feedback. Finally, a report comprising adapted data may be generated.
    Type: Grant
    Filed: December 3, 2021
    Date of Patent: June 21, 2022
    Inventors: Prashant Warier, Ankit Modi, Preetham Putha, Prakash Vanapalli, Pradeep Kumar Thummala, Vijay Senapathi, Kunjesh Kumar
  • Patent number: 11367192
    Abstract: In a method of generating a virtual 3D model of a dental site, scan data comprising an intraoral image is received during an intraoral scan of a dental site. A representation of a foreign object is identified in the intraoral image based on an analysis of the scan data. The intraoral image is modified by removing the representation of the foreign object from the intraoral image. Additional scan data comprising a plurality of additional intraoral images of the dental site is received during the intraoral scan. A virtual 3D model of the dental site is then generated using the modified intraoral image and the plurality of additional intraoral images.
    Type: Grant
    Filed: March 4, 2020
    Date of Patent: June 21, 2022
    Assignee: Align Technology, Inc.
    Inventors: Avi Kopelman, Michael Sabina
  • Patent number: 11367193
    Abstract: A method for medical imaging may include obtaining a plurality of successive images of a region of interest (ROI) including at least a portion of an object's heart. The plurality of successive images may be based on imaging data acquired from the ROI by a scanner without electrocardiography (ECG) gating. The plurality of successive images may be related to one or more cardiac cycles of the object's heart. The method may also include automatically determining, in the plurality of successive images, target images that correspond to at least one of the one or more cardiac cycles of the object's heart.
    Type: Grant
    Filed: December 31, 2019
    Date of Patent: June 21, 2022
    Assignee: UIH AMERICA, INC.
    Inventors: Jingyuan Lyu, Yu Ding, Qi Liu, Jian Xu
  • Patent number: 11367194
    Abstract: Systems, devices, media, and methods are presented for segmenting an image of a video stream with a client device, binarizing an area of interest within one or more image, identifying an initial pupil location and an initial iris radius, and determining a final pupil location and a final iris radius. Some embodiments enable the client device to perform one or more operations within a user interface based on the image segmentation.
    Type: Grant
    Filed: February 19, 2020
    Date of Patent: June 21, 2022
    Assignee: Snap Inc.
    Inventor: Fedir Poliakov
  • Patent number: 11367195
    Abstract: An image segmentation method, an image segmentation apparatus, an image segmentation device are provided, the image segmentation method including: extracting, from a current frame of a video image, a skeleton two-dimensional estimation and a skeleton three-dimensional estimation of a human three-dimensional skeleton; obtaining a target three-dimensional skeleton based on the skeleton two-dimensional estimation and the skeleton three-dimensional estimation; implementing image segmentation based on the target three-dimensional skeleton; wherein the human three-dimensional skeleton has multiple nodes, and the video image is a two-dimensional image, when calculating the target three-dimensional skeleton in the current frame, by comprehensively considering the skeleton two-dimensional estimation and skeleton three-dimensional skeleton estimation of the human three-dimensional skeleton in the current frame, the accuracy and robustness of the obtained target three-dimensional skeleton can be improved, thereby improv
    Type: Grant
    Filed: June 24, 2020
    Date of Patent: June 21, 2022
    Assignees: Beijing BOE Optoelectronics Technology Co., Ltd., BOE Technology Group Co., Ltd.
    Inventors: Yaoyu Lv, Lili Chen, Hao Zhang, Fuqiang Ma, Jiankang Sun, Shuo Zhang, Gang Li, Yachong Xue, Xiangjun Peng, Chenxi Zhao
  • Patent number: 11367196
    Abstract: This application relates to an image processing method and apparatus, a storage medium, and a computer device. The method includes obtaining acquired image frames; identifying, in each obtained image frame, a target area and a reference area that are obtained through image semantic segmentation; detecting, when a location relationship between the target area and the reference area in an obtained first image frame meets an action start condition and a location relationship between the target area and the reference area in an obtained second image frame meets an action end condition, an action to trigger adding an additional element, the second image frame being acquired after the first image frame; obtaining the additional element when the triggering action is detected; and adding the additional element to image frames acquired after the second image frame.
    Type: Grant
    Filed: August 19, 2020
    Date of Patent: June 21, 2022
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jun Cheng, Ying Zhu, Haoyuan Li, Feng Li, Xiaoxiang Zuo
  • Patent number: 11367197
    Abstract: Systems and methods of the present disclosure can facilitate determining a three-dimensional surface representation of an object. In some embodiments, the system includes a computer, a calibration module, which is configured to determine a camera geometry of a set of cameras, and an imaging module, which is configured to capture spatial images using the cameras. The computer is configured to determine epipolar lines in the spatial images, transform the spatial images with a collineation transformation, determine second derivative spatial images with a second derivative filter, construct epipolar plane edge images based on zero crossings of second derivative epipolar planes image based on the epipolar lines, select edges and compute depth estimates, sequence the edges based on contours in a spatial edge image, filter the depth estimates, and create a three-dimensional surface representation based on the filtered depth estimates and the original spatial images.
    Type: Grant
    Filed: January 6, 2020
    Date of Patent: June 21, 2022
    Inventor: Henry Harlyn Baker
  • Patent number: 11367198
    Abstract: Systems, methods and apparatuses for tracking at least a portion of a body by fitting data points received from a depth sensor and/or other sensors and/or “markers” as described herein to a body model. For example, in some embodiments, certain of such data points are identified as “super points,” and apportioned greater weight as compared to other points. Such super points can be obtained from objects attached to the body, including, but not limited to, active markers that provide a detectable signal, or a passive object, including, without limitation, headgear or a mask (for example for VR (virtual reality)), or a smart watch. Such super points may also be obtained from specific data points that are matched to the model, such as data points that are matched to vertices that correspond to joints in the model.
    Type: Grant
    Filed: July 28, 2019
    Date of Patent: June 21, 2022
    Assignee: MindMaze Holding SA
    Inventors: Tej Tadi, Nicolas Fremaux, Jose Rubio, Jonas Ostlund, Max Jeanneret
  • Patent number: 11367199
    Abstract: Systems and methods provide editing operations in a smart editing system that may generate a focal point within a mask of an object for each frame of a video segment and perform editing effects on the frames of the video segment to quickly provide users with natural video editing effects. An eye-gaze network may produce a hotspot map of predicted focal points in a video frame. These predicted focal points may then be used by a gaze-to-mask network to determine objects in the image and generate an object mask for each of the detected objects. This process may then be repeated to effectively track the trajectory of objects and object focal points in videos. Based on the determined trajectory of an object in a video clip and editing parameters, the editing engine may produce editing effects relative to an object for the video clip.
    Type: Grant
    Filed: June 12, 2020
    Date of Patent: June 21, 2022
    Assignee: Adobe Inc.
    Inventors: Lu Zhang, Jianming Zhang, Zhe Lin, Radomir Mech
  • Patent number: 11367200
    Abstract: A method for determining the image position of a marker point (3) in an image of an image sequence including the method steps of: setting (S2) a marker point in a first image (1) of the image sequence, determining (S4) a transformation at least between corresponding portions of the first image (1) and a second image (4) of the image sequence, transforming (S5) at least the portion of the first image (1) or the portion of the second image (4) on the basis of the transformation determined, localizing (S6) the marker point (3) in the transformed portion of the image (4?), and mapping (S7) the localized marker point into the second image (4) on the basis of the determined transformation.
    Type: Grant
    Filed: June 17, 2020
    Date of Patent: June 21, 2022
    Assignee: Scholly Fiberoptic GmbH
    Inventors: Michael Kronenthaler, Frank Hassenpflug
  • Patent number: 11367201
    Abstract: Systems and methods for tracking the location of a non-destructive inspection (NDI) scanner using scan data converted into images of a target object. Scan images are formed by aggregating successive scan strips acquired using one or two one-dimensional sensor arrays. An image processor constructs and then compares successive partially overlapping scan images that include common features corresponding to respective structural features of the target object. The image processor is further configured to compute a change in location of the NDI scanner relative to a previous location based on the respective positions of those common features in the partially overlapping scan images. This relative physical distance is then added to the previous (old) absolute location estimate to obtain the current (new) absolute location of the NDI scanner.
    Type: Grant
    Filed: September 24, 2019
    Date of Patent: June 21, 2022
    Assignee: The Boeing Company
    Inventors: James J. Troy, Gary E. Georgeson, Joseph L. Hafenrichter
  • Patent number: 11367202
    Abstract: A method of making-up or breaking-out tubular string components can include threading tubulars with each other while a camera obtains images of the tubulars, outputting image data from the camera to an image processor that detects optical flow vector fields from the image data, the optical flow vector fields representing displacements of the respective tubulars during the threading, and controlling the threading in response to a difference between the displacements. Another method can include positioning a camera so that the camera simultaneously observes at least two tubulars, threading the tubulars with each other, outputting image data from the camera to an image processor, the image processor detecting optical flow vector fields from the image data, the optical flow vector fields representing displacements of the respective tubulars during the threading, and controlling the threading in response to the image processor detecting the optical flow vector fields.
    Type: Grant
    Filed: May 14, 2020
    Date of Patent: June 21, 2022
    Assignee: Weatherford Technology Holdings, LLC
    Inventors: Benjamin Sachtleben, David Geissler, Christina Hebebrand, Rainer Ruehmann
  • Patent number: 11367203
    Abstract: A cross-sensor object-attribute analysis method for detecting at least one object in a space by using cooperation of a plurality of image sensing devices, the method including: the image sensing devices sending raw data or attribute vector data of sensed multiple images to a main information processing device, where the raw data and attribute vector data all correspond to a time record; and the main information processing device generating one or more of the attribute vectors according to the raw data of each of the images and using each of the one or more of the attribute vectors to correspond to one of the at least one object, or the main information processing device directly using the attribute vector data to correspond to the at least one object.
    Type: Grant
    Filed: December 24, 2020
    Date of Patent: June 21, 2022
    Assignee: QNAP SYSTEMS, INC.
    Inventors: Kung-Han Chen, Peng-Yan Sie, Jian-Kai Wang, Yun-Tao Chen
  • Patent number: 11367204
    Abstract: A multi-sensor spatial data auto-synchronization system and method is provided. The method may include collecting laser point cloud data through a laser radar and pre-processing the laser point cloud data; collecting image point cloud data through a monocular camera and collecting intrinsic parameters of the camera by using a calibration board; performing plane fitting on the pre-processed laser point cloud data to determine coordinates of fitted laser point cloud data; performing image feature extraction on the image point cloud data; calculating pose transformation matrices for the laser point cloud data coordinates and an image feature data result by using a PNP algorithm; and optimizing a rotation vector and a translation vector in the pose transformation matrix. The present invention achieves automatic calculation of a spatial synchronization relationship between two sensors, and greatly improves the data synchronization precision of the laser radar and the monocular camera.
    Type: Grant
    Filed: December 16, 2021
    Date of Patent: June 21, 2022
    Assignee: ECOTRON LLC
    Inventor: Ming Liao
  • Patent number: 11367205
    Abstract: Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.
    Type: Grant
    Filed: December 19, 2019
    Date of Patent: June 21, 2022
    Assignee: Snap Inc.
    Inventors: Shenlong Wang, Linjie Luo, Ning Zhang, Jia Li
  • Patent number: 11367206
    Abstract: In order to provide monocular depth prediction, a trained neural network may be used. To train the neural network, edge detection on a digital image may be performed to determine at least one edge of the digital image, and then a first point and a second point of the digital image may be sampled, based on the at least one edge. A relative depth between the first point and the second point may be predicted, and the neural network may be trained to perform monocular depth prediction using a loss function that compares the predicted relative depth with a ground truth relative depth between the first point and the second point.
    Type: Grant
    Filed: February 13, 2020
    Date of Patent: June 21, 2022
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Oliver Wang, Mai Long, Ke Xian, Jianming Zhang
  • Patent number: 11367207
    Abstract: A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on the visual information in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.
    Type: Grant
    Filed: September 25, 2020
    Date of Patent: June 21, 2022
    Assignee: Blue River Technology Inc.
    Inventors: Chia-Chun Fu, Christopher Grant Padwick, James Patrick Ostrowski
  • Patent number: 11367208
    Abstract: Operations may comprise obtaining a plurality of light detection and ranging (LIDAR) scans of a region. The operations may also comprise identifying a plurality of LIDAR poses that correspond to the plurality of LIDAR scans. In addition, the operations may comprise identifying, as a plurality of keyframes, a plurality of images of the region that are captured during capturing of the plurality of LIDAR scans. The operations may also comprise determining, based on the plurality of LIDAR poses, a plurality of camera poses that correspond to the keyframes. Further, the operations may comprise identifying a plurality of two-dimensional (2D) keypoints in the keyframes. The operations also may comprise generating one or more three-dimensional (3D) keypoints based on the plurality of 2D keypoints and the respective camera poses of the plurality of keyframes.
    Type: Grant
    Filed: June 25, 2020
    Date of Patent: June 21, 2022
    Assignee: NVIDIA CORPORATION
    Inventors: Ronghua Zhang, Derik Schroeter, Mengxi Wu, Di Zeng
  • Patent number: 11367209
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for visually detecting a halocline. In some implementations, a method includes moving a camera through different depths of water within a fish enclosure, capturing, at the different depths, images of fish, determining that changes in focus in the images correspond to changes in depth that the images were captured, and based on determining that the changes in focus in the images correspond to the changes in depths that the images were captured, detecting a halocline at a particular depth.
    Type: Grant
    Filed: October 23, 2020
    Date of Patent: June 21, 2022
    Assignee: X Development LLC
    Inventor: Barnaby John James
  • Patent number: 11367210
    Abstract: Disclosed herein is a system to smoothly change the focus of a camera between multiple targets. The system can obtain an indication of a target, an indication of a manner of focus transition between a first target and a second target, and camera settings. The system can determine a point associated with the second target, where the point has a property that focusing the camera on the point places the second target in focus, and the point is closer to the current focus point of the camera than a substantial portion of other points having the property. The system can obtain a nonlinear function indicating a second manner of focus transition between the first target and the second target. The system can change the focus of the camera between the first target and the second target by changing the focus of the camera from the current focus point to the determined point based on the nonlinear function.
    Type: Grant
    Filed: November 12, 2021
    Date of Patent: June 21, 2022
    Assignee: Unity Technologies SF
    Inventors: Eric S. Hayes, Robert S. Hurlburt, Corey B. Frew
  • Patent number: 11367211
    Abstract: A target detection system includes an imager that sequentially captures frames, in different emission bands. These frames are registered with respect to one another based on information from an Inertial Navigation System (INS) and compensation is made for any motion of the imager between frames. Identification of a candidate target object that is detected, or otherwise identified in one band, initiates a search for a correlated object in the other band within a radius, or distance, in a common image-space reference frame. When a pair of objects is identified, a measure of the intensities of the signals in the two bands is compared to an expected range of values to discriminate a target of interest from an object that is not of interest.
    Type: Grant
    Filed: July 29, 2019
    Date of Patent: June 21, 2022
    Assignee: Raytheon Company
    Inventor: Anthony J. Ragucci
  • Patent number: 11367212
    Abstract: Based on an image from a stationary sensor, a marker displayed on one or more digital displays on a vehicle is detected. A first location and a first orientation of the marker in a coordinate system is determined by analyzing on pixels in the image. Based on a stationary sensor location and orientation from a map, a second location and a second orientation of the vehicle in the coordinate system is determined.
    Type: Grant
    Filed: November 21, 2019
    Date of Patent: June 21, 2022
    Assignee: FORD GLOBAL TECHNOLOGIES, LLC
    Inventors: Punarjay Chakravarty, Ramchandra Ganesh Karandikar