Patents by Inventor Sheng Nan Zhu
Sheng Nan Zhu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11869257Abstract: A method for detecting and labeling a target object in a 2D image includes receiving a plurality of 2D images from a visual sensor, manually marking points of the target object on each of the 2D images, generating from the 2D images a 3D world coordinate system of the environment surrounding the target object, mapping each of the marked points on the 2D images to the 3D world coordinate system using a simultaneous localization and mapping (SLAM) engine, automatically generating a 3D bounding box covering all the marked points mapped to the 3D world coordinate system, mapping the 3D bounding box to each of the 2D images, generating a label for the target object on each of the 2D images using a machine learning object detection model, and training the machine learning object detection model based on the generated label for the target object.Type: GrantFiled: March 19, 2021Date of Patent: January 9, 2024Assignee: International Business Machines CorporationInventors: Guoqiang Hu, Sheng Nan Zhu, Yuan Yuan Ding, Hong Bing Zhang, Dan Zhang, Tian Tian Chai
-
Patent number: 11615634Abstract: A system, method, and computer program product provides a way to separate connected or adhered adjacent characters of a digital image for license plate recognition. As a threshold processing, the method performs a recognition of character adhesion by obtaining character parameters using an image processor. The parameters include a horizontal max crossing and a ratio of width and height. A first rule-based module is used responsive to the character parameters to distinguish the adhered characters (character adhesions) that are easy to judge, leaving the uncertain part to a character adhesion classifier model for discrimination. Character adhesion data is obtained by data augmentation including the adding of a random distance between two single characters to create class like adhered characters. Then the character adhesion classifier model of single character and character adhesion data is trained. Any uncertain part can be distinguished by the trained character adhesion classifier model.Type: GrantFiled: July 1, 2021Date of Patent: March 28, 2023Assignee: International Business Machines CorporationInventors: Sheng Nan Zhu, Guoqiang Hu, Yuan Yuan Ding, Peng Ji, Fan Li, Jian Xu
-
Patent number: 11587233Abstract: The present invention relates to a method, system and computer program product for defect enhancement. According to the method, a plurality of proposed regions from a plurality of images taken for a display panel is obtained. Each of proposed region of the plurality of proposed regions include a suspected defect on the display panel. At least two proposed regions from the plurality of proposed regions that deserve to be superimposed based on a set of conditions is determined. The at least two proposed regions for acquiring an enhanced defect are superimposed.Type: GrantFiled: October 17, 2019Date of Patent: February 21, 2023Assignee: International Business Machines CorporationInventors: JingChang Huang, Sheng Nan Zhu, Fan Li, Peng Ji, Jian Xu, Wei Zhao, Jinfeng Li
-
Publication number: 20230004747Abstract: A system, method, and computer program product provides a way to separate connected or adhered adjacent characters of a digital image for license plate recognition. As a threshold processing, the method performs a recognition of character adhesion by obtaining character parameters using an image processor. The parameters include a horizontal max crossing and a ratio of width and height. A first rule-based module is used responsive to the character parameters to distinguish the adhered characters (character adhesions) that are easy to judge, leaving the uncertain part to a character adhesion classifier model for discrimination. Character adhesion data is obtained by data augmentation including the adding of a random distance between two single characters to create class like adhered characters. Then the character adhesion classifier model of single character and character adhesion data is trained. Any uncertain part can be distinguished by the trained character adhesion classifier model.Type: ApplicationFiled: July 1, 2021Publication date: January 5, 2023Inventors: Sheng Nan Zhu, GUOQIANG HU, Yuan Yuan Ding, Peng Ji, Fan Li, Jian Xu
-
Publication number: 20220300738Abstract: A method for detecting and labeling a target object in a 2D image includes receiving a plurality of 2D images from a visual sensor, manually marking points of the target object on each of the 2D images, generating from the 2D images a 3D world coordinate system of the environment surrounding the target object, mapping each of the marked points on the 2D images to the 3D world coordinate system using a simultaneous localization and mapping (SLAM) engine, automatically generating a 3D bounding box covering all the marked points mapped to the 3D world coordinate system, mapping the 3D bounding box to each of the 2D images, generating a label for the target object on each of the 2D images using a machine learning object detection model, and training the machine learning object detection model based on the generated label for the target object.Type: ApplicationFiled: March 19, 2021Publication date: September 22, 2022Inventors: Guoqiang Hu, Sheng Nan Zhu, Yuan Yuan Ding, Hong Bing Zhang, Dan Zhang, Tian Tian Chai
-
Patent number: 11354793Abstract: In an approach for object detection with missing annotations under visual inspection, a processor receives an image. A processor classifies the image being a not-good image using a pre-trained classifier. A not-good image means one or more defect objects being in the image. A processor, in response to classifying the image being the not-good image, detects the one or more defect objects in the not-good image. A processor masks the one or more defect objects in the not-good image. A processor inputs the masked image to train a detector.Type: GrantFiled: December 16, 2019Date of Patent: June 7, 2022Assignee: International Business Machines CorporationInventors: Jian Xu, Guo Qiang Hu, Fan Li, Sheng Nan Zhu, Jinfeng Li, Jun Zhu
-
Patent number: 11341370Abstract: The present disclosure relates to training a machine learning model to classify images. An example method generally includes receiving a training data set including images in a first category and images in a second category. A convolutional neural network (CNN) is trained using the training data set, and a feature map is generated from layers of the CNN based on features of images in the training data set. A first area in the feature map including images in the first category and a second area in the feature map where images in the first category overlap with images in the second category are identified. The first category is split into a first subcategory corresponding to the first area and a second subcategory corresponding to the second area. The CNN is retrained based on the images in the first subcategory, images in the second subcategory, and images in the second category.Type: GrantFiled: November 22, 2019Date of Patent: May 24, 2022Assignee: International Business Machines CorporationInventors: Peng Ji, Guo Qiang Hu, Yuan Yuan Ding, Jun Zhu, Jing Chang Huang, Sheng Nan Zhu
-
Patent number: 11295439Abstract: A method, a device and a computer program product for image processing are proposed. In the method, a first training image and region information are obtained. The region information indicates a region of a defect in the first training image. A second training image with the defect at least partially removed is generated using an image generator based on the first training image and the region information. The image generator is trained to recover the first training image by replacing pixels included in the region indicated by the region information. The image generator is updated based on the second training image. In this way, the image including the defect can be accurately and efficiently recovered.Type: GrantFiled: October 16, 2019Date of Patent: April 5, 2022Assignee: International Business Machines CorporationInventors: Fan Li, Guo Qiang Hu, Jun Zhu, Sheng Nan Zhu, JingChang Huang, Yuan Yuan Ding, Peng Ji
-
Publication number: 20220092756Abstract: A plurality of different images of a same region of interest in an object are input into a set of neural networks, wherein each image of the region has been captured under a different value of a variable condition. A classification for each image is generated by the set of neural networks, wherein each classification includes a confidence score in a prediction of whether a feature is present in the region. The image classifications are ensembled to generate a final classification for the region. By applying a loss function, a loss is computed based on comparing the final classification to a ground truth of whether the feature is present in the region. The parameters of the set of neural networks are adjusted based on the computed loss.Type: ApplicationFiled: September 21, 2020Publication date: March 24, 2022Inventors: JingChang Huang, GUO QIANG HU, Peng Ji, Yuan Yuan Ding, Sheng Nan Zhu, Jinfeng Li
-
Publication number: 20210183038Abstract: In an approach for object detection with missing annotations under visual inspection, a processor receives an image. A processor classifies the image being a not-good image using a pre-trained classifier. A not-good image means one or more defect objects being in the image. A processor, in response to classifying the image being the not-good image, detects the one or more defect objects in the not-good image. A processor masks the one or more defect objects in the not-good image. A processor inputs the masked image to train a detector.Type: ApplicationFiled: December 16, 2019Publication date: June 17, 2021Inventors: Jian Xu, Guo Qiang Hu, Fan Li, Sheng Nan Zhu, Jinfeng Li, Jun Zhu
-
Patent number: 11030738Abstract: A method, a device and a computer program product for image processing are proposed. In the method, whether a first image indicates a defect associated with a target object is determined. In response to determining that the first image indicates the defect, a second image absent from the defect is obtained based on the first image. The defect is identified by comparing the first image with the second image. In this way, the defect associated with the target object in the image can be accurately and efficiently identified or segmented.Type: GrantFiled: July 5, 2019Date of Patent: June 8, 2021Assignee: International Business Machines CorporationInventors: Fan Li, Guo Qiang Hu, Sheng Nan Zhu, Jun Zhu, Jing Chang Huang, Peng Ji, Yuan Yuan Ding
-
Publication number: 20210158094Abstract: The present disclosure relates to training a machine learning model to classify images. An example method generally includes receiving a training data set including images in a first category and images in a second category. A convolutional neural network (CNN) is trained using the training data set, and a feature map is generated from layers of the CNN based on features of images in the training data set. A first area in the feature map including images in the first category and a second area in the feature map where images in the first category overlap with images in the second category are identified. The first category is split into a first subcategory corresponding to the first area and a second subcategory corresponding to the second area. The CNN is retrained based on the images in the first subcategory, images in the second subcategory, and images in the second category.Type: ApplicationFiled: November 22, 2019Publication date: May 27, 2021Inventors: Peng JI, Guo Qiang Hu, Yuan Yuan Ding, Jun Zhu, Jing Chang Huang, Sheng Nan Zhu
-
Publication number: 20210118113Abstract: A method, a device and a computer program product for image processing are proposed. In the method, a first training image and region information are obtained. The region information indicates a region of a defect in the first training image. A second training image with the defect at least partially removed is generated using an image generator based on the first training image and the region information. The image generator is trained to recover the first training image by replacing pixels included in the region indicated by the region information. The image generator is updated based on the second training image. In this way, the image including the defect can be accurately and efficiently recovered.Type: ApplicationFiled: October 16, 2019Publication date: April 22, 2021Inventors: Fan Li, Guo Qiang Hu, Jun Zhu, Sheng Nan Zhu, JingChang Huang, Yuan Yuan Ding, Peng Ji
-
Publication number: 20210118141Abstract: The present invention relates to a method, system and computer program product for defect enhancement. According to the method, a plurality of proposed regions from a plurality of images taken for a display panel is obtained. Each of proposed region of the plurality of proposed regions include a suspected defect on the display panel. At least two proposed regions from the plurality of proposed regions that deserve to be superimposed based on a set of conditions is determined. The at least two proposed regions for acquiring an enhanced defect are superimposed.Type: ApplicationFiled: October 17, 2019Publication date: April 22, 2021Inventors: JingChang Huang, Sheng Nan Zhu, Fan Li, Peng Ji, Jian Xu, Wei Zhao, Jinfeng Li
-
Publication number: 20210073686Abstract: Techniques for generating machine learning architectures are provided. A data set is received for training one or more machine learning (ML) models, where the data set comprises labeled exemplars for a plurality of classes. The data set is partitioned into a training set and a testing set. A first ML model is trained using the training set, and a quality of the first ML model with respect to each class of the plurality of classes is evaluated using the testing set. Upon determining that the quality of the first ML model is below a predefined threshold with respect to a first class and a second class of the plurality of classes, a subset of the training set is identified, where each exemplar in the subset corresponds to either the first class or the second class. A second ML model is trained using the subset of the training set.Type: ApplicationFiled: September 6, 2019Publication date: March 11, 2021Inventors: Yuan Yuan Ding, GUO QIANG HU, Jun Zhu, Jing Chang Huang, Sheng Nan Zhu, Fan LI, Peng Ji
-
Patent number: 10896341Abstract: A computer implemented method for surface defect inspection that includes recording an optical image of a surface including a defect; converting the optical image including the defect into a heat map; extracting a region of interest including the defect from the heat map; and comparing the region of interest including the defect from the heat map to a binary classification model using a sliding window based voting mechanism to determine if the defect is greater than or less than a threshold failure value.Type: GrantFiled: November 15, 2018Date of Patent: January 19, 2021Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATIONInventors: Sheng Nan Zhu, Guo Qiang Hu, Jun Zhu, Jing Chang Huang, Peng Ji
-
Publication number: 20210004945Abstract: A method, a device and a computer program product for image processing are proposed. In the method, whether a first image indicates a defect associated with a target object is determined. In response to determining that the first image indicates the defect, a second image absent from the defect is obtained based on the first image. The defect is identified by comparing the first image with the second image. In this way, the defect associated with the target object in the image can be accurately and efficiently identified or segmented.Type: ApplicationFiled: July 5, 2019Publication date: January 7, 2021Inventors: FAN LI, GUO QIANG HU, Sheng Nan Zhu, JUN ZHU, Jing Chang Huang, Peng Ji, Yuan Yuan Ding
-
Publication number: 20200160083Abstract: A computer implemented method for surface defect inspection that includes recording an optical image of a surface including a defect; converting the optical image including the defect into a heat map; extracting a region of interest including the defect from the heat map; and comparing the region of interest including the defect from the heat map to a binary classification model using a sliding window based voting mechanism to determine if the defect is greater than or less than a threshold failure value.Type: ApplicationFiled: November 15, 2018Publication date: May 21, 2020Inventors: Sheng Nan Zhu, Guo Qiang Hu, Jun Zhu, Jing Chang Huang, Peng Ji