Patents by Inventor Zhichun Xiao

Zhichun Xiao has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230334088
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to automatically annotating images. In some embodiments, a system for automatically annotating images comprises a database, wherein the database is configured to store images and annotations for the images and a control circuit, wherein the control circuit is communicatively coupled to the database, and wherein the control circuit is configured to retrieve, from the database, an image, generate, based on the image, a collection of augmented images, generate segmentation maps for each image in the collection of augmented images, wherein each of the segmentation maps include segments, select, based on a threshold, ones of the segments above a threshold, merge the ones of the segments above the threshold to create a segmented image, and generate, for each segment of the segmented image, classifications, wherein an annotation for the image includes the segmented images and the classifications.
    Type: Application
    Filed: June 20, 2023
    Publication date: October 19, 2023
    Inventors: Haining Liu, Feiyun Zhu, Ryan B. Reagan, Zhichun Xiao
  • Publication number: 20230274248
    Abstract: Some embodiments provide image evaluation systems and methods, comprising: a plurality of camera systems distributed about a retail facility and each of the plurality of camera systems; and an image processing system configured to receive multiple images over time, process each image comprising: determine, from pixel data, a gradient amplitude and directional component; determine a histogram curve from the gradient amplitudes as a function of the directional component of the pixel data, and identify a key direction relative to a maximum accumulation of the gradient amplitudes; for the gradient amplitudes, of the pixel data, having a corresponding directional component that is within a direction threshold of the key direction, identify a number of local maxima corresponding to the key direction; and determine a quantity of items of the product corresponding to a quantity of the number of local maxima.
    Type: Application
    Filed: January 31, 2022
    Publication date: August 31, 2023
    Inventors: Zhichun Xiao, Chunmei Wang
  • Patent number: 11720623
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to automatically annotating images. In some embodiments, a system for automatically annotating images comprises a database, wherein the database is configured to store images and annotations for the images and a control circuit, wherein the control circuit is communicatively coupled to the database, and wherein the control circuit is configured to retrieve, from the database, an image, generate, based on the image, a collection of augmented images, generate segmentation maps for each image in the collection of augmented images, wherein each of the segmentation maps include segments, select, based on a threshold, ones of the segments above a threshold, merge the ones of the segments above the threshold to create a segmented image, and generate, for each segment of the segmented image, classifications, wherein an annotation for the image includes the segmented images and the classifications.
    Type: Grant
    Filed: November 13, 2020
    Date of Patent: August 8, 2023
    Assignee: Walmart Apollo, LLC
    Inventors: Haining Liu, Feiyun Zhu, Ryan B. Reagan, Zhichun Xiao
  • Publication number: 20230245535
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to self-checkout verification at a retail facility. In some embodiments, there is provided a system for self-checkout verification at a retail facility including a first optical imaging unit; and a control circuit. The control circuit configured to: receive purchase receipt data; receive one or more images of the items in the container; and execute a machine learning model trained to: perform item detection, item classification, and item verification of each item shown in the one or more images; and output electronic data corresponding to an electronic receipt of the items in the container. The control circuit may automatically detect each unpaid item in the container based on a comparison of the purchase receipt data with the electronic data; and provide an alert signal in response to automatically detecting an unpaid item.
    Type: Application
    Filed: January 27, 2023
    Publication date: August 3, 2023
    Inventors: Zhichun Xiao, Lingfeng Zhang, Yutao Tang, Mingquan Yuan, Tianyi Mao, Han Zhang, Chunmei Wang, Feiyun Zhu, Zhaoliang Duan, Yanmei Jin, Jing Wang, Wei Wang, Ranran Cao, Zhiwei Huang
  • Publication number: 20230222825
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting text in images. In some embodiments, a system for detecting text in images comprises a database configured to store images and a control circuit configured to retrieve an image, generate, based on the image, a collection of augmented images, detect characters in each of the augmented images, generate bounding boxes for the characters in each of augmented images, recognize the characters in each of the augmented images, select, based on the recognition of the characters in each of the augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the augmented images, detect, for the image, a color associated with the characters, and store, in the database, the image, the candidate characters, and the color associated with the characters.
    Type: Application
    Filed: March 7, 2023
    Publication date: July 13, 2023
    Inventors: Haining Liu, Feiyun Zhu, Jon Hammer, Ryan B. Reagan, Pingjian Yu, Zhichun Xiao, Yuqi Zhang, Yao Liu
  • Patent number: 11620839
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting text in images. In some embodiments, a system for detecting text in images comprises a database configured to store images and a control circuit configured to retrieve an image, generate, based on the image, a collection of augmented images, detect characters in each of the augmented images, generate bounding boxes for the characters in each of augmented images, recognize the characters in each of the augmented images, select, based on the recognition of the characters in each of the augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the augmented images, detect, for the image, a color associated with the characters, and store, in the database, the image, the candidate characters, and the color associated with the characters.
    Type: Grant
    Filed: November 13, 2020
    Date of Patent: April 4, 2023
    Assignee: Walmart Apollo, LLC
    Inventors: Haining Liu, Feiyun Zhu, Jon Hammer, Ryan B. Reagan, Pingjian Yu, Zhichun Xiao, Yuqi Zhang, Yao Liu
  • Publication number: 20220383383
    Abstract: There are provided systems and methods relating to checking and confirming merchandise purchased at shopping facilities. In one form, the system includes: a shopping cart containing merchandise items to be purchased; a sales transaction database; a point-of-sales system that creates transaction records identified by transaction identifiers; and a mobile device used by an employee that includes a sensor to scan a paper or digital receipt to obtain the transaction identifier and a camera to capture images of the items in the shopping cart. The system also includes a control circuit that receives the transaction identifier, accesses the database using the identifier to determine the purchased items; analyzes the images of the merchandise items in the shopping cart and creates a computer vision receipt listing detected items; compares the purchased items with the detected items; and takes an action if there is a discrepancy.
    Type: Application
    Filed: November 12, 2020
    Publication date: December 1, 2022
    Inventors: Zhichun XIAO, Lingfeng ZHANG, Yao LIU, Jon HAMMER, Yutao TANG, Sicong FANG, Haining LIU, Yijing SUN, Mingquan YUAN, Shouyi ZHANG, Pingjian YU, Ryan B. REAGAN, Tianyi MAO, Shangeetha RAVICHANDRAN SUSSEELAA, Zhenyu WANG, Feiyun ZHU
  • Publication number: 20220343668
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting inbound and outbound traffic. In some embodiments, there is provided a system for detecting inbound and outbound traffic at a facility including a video camera and a control circuit configured to detect a human and estimate a location of the human; determine whether the detected human is inbound or outbound the facility based on a relative movement of the detected human within a first region, a second region, and a third region; and transmit data based on the determination.
    Type: Application
    Filed: July 11, 2022
    Publication date: October 27, 2022
    Inventors: Yi Ding, Joseph Duffy, Jiankun Liu, Tianyi Mao, Ryan B. Reagan, Zhichun Xiao, Jinhan Zhang, Shouyi Zhang
  • Patent number: 11417104
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to automatically determining location of an object. In some embodiments, there is provided a system for automatically determining location of an object inside a retail store via one or more cameras including: a plurality of locations inside a retail store; a first camera capturing a first image of a first location; a portable electronic device configured to transmit a signal used to determine an absolute location of a test object; one or more first receiver circuits configured to receive transmitted first signals from the portable electronic device; and a control circuit configured to: associate a first set of store coordinates of a first coordinate system associated with an entire area inside the retail store with a first set of frame coordinates of a second coordinate system associated with the plurality of cameras; and cause a database to store the association.
    Type: Grant
    Filed: November 2, 2020
    Date of Patent: August 16, 2022
    Assignee: Walmart Apollo, LLC
    Inventors: Yanbin Ye, Zhichun Xiao
  • Publication number: 20220036568
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting and tracking humans. In some embodiments, there is provided a system for detecting and tracking humans from one image to another image including: a camera; a control circuit configure to: receive a first image; detect a plurality of key body joints of a first human captured on the first image; determine segmentations of the plurality of key body joints to determine one or more body parts of the first human; determine a color distribution map of aggregate pixels associated with each body part of the first human on the first image; and cause a database to store the color distribution map; and a database comprising one or more color distribution map sets each associated with a detected human in a captured image of the camera.
    Type: Application
    Filed: July 29, 2021
    Publication date: February 3, 2022
    Inventors: Haining Liu, Ryan B. Reagan, Zhichun Xiao
  • Publication number: 20210233110
    Abstract: Systems and methods are for providing targeted advertisements to customers at a retail store include digital display devices and digital cameras that capture physical characteristics data and behavioral data associated with the in-store customers of the retail store. The system also includes a computing device that obtains, from an electronic advertisement database, a request for the purchase of the targeted advertisements by a client wishing to digitally advertise at the store, and the physical characteristics data, behavioral data, and/or historical sales data. The computing device then correlates the request for the purchase of the targeted advertisements by the client to the physical characteristics data, behavioral data, and/or historical sales data in order to determine a schedule for displaying the targeted advertisements to in-store customers via the digital display devices, and to select one or more of the digital display devices for displaying the targeted advertisements to the in-store customers.
    Type: Application
    Filed: January 29, 2021
    Publication date: July 29, 2021
    Inventors: Yanbin Ye, Zhichun Xiao
  • Publication number: 20210150276
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to automatically annotating images. In some embodiments, a system for automatically annotating images comprises a database, wherein the database is configured to store images and annotations for the images and a control circuit, wherein the control circuit is communicatively coupled to the database, and wherein the control circuit is configured to retrieve, from the database, an image, generate, based on the image, a collection of augmented images, generate segmentation maps for each image in the collection of augmented images, wherein each of the segmentation maps include segments, select, based on a threshold, ones of the segments above a threshold, merge the ones of the segments above the threshold to create a segmented image, and generate, for each segment of the segmented image, classifications, wherein an annotation for the image includes the segmented images and the classifications.
    Type: Application
    Filed: November 13, 2020
    Publication date: May 20, 2021
    Inventors: Haining Liu, Feiyun Zhu, Ryan B. Reagan, Zhichun Xiao
  • Publication number: 20210150245
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting text in images. In some embodiments, a system for detecting text in images comprises a database configured to store images and a control circuit configured to retrieve an image, generate, based on the image, a collection of augmented images, detect characters in each of the augmented images, generate bounding boxes for the characters in each of augmented images, recognize the characters in each of the augmented images, select, based on the recognition of the characters in each of the augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the augmented images, detect, for the image, a color associated with the characters, and store, in the database, the image, the candidate characters, and the color associated with the characters.
    Type: Application
    Filed: November 13, 2020
    Publication date: May 20, 2021
    Inventors: Haining Liu, Feiyun Zhu, Jon Hammer, Ryan B. Reagan, Pingjian Yu, Zhichun Xiao, Yuqi Zhang, Yao Liu
  • Publication number: 20210133860
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to automatically determining location of an object. In some embodiments, there is provided a system for automatically determining location of an object inside a retail store via one or more cameras including: a plurality of locations inside a retail store; a first camera capturing a first image of a first location; a portable electronic device configured to transmit a signal used to determine an absolute location of a test object; one or more first receiver circuits configured to receive transmitted first signals from the portable electronic device; and a control circuit configured to: associate a first set of store coordinates of a first coordinate system associated with an entire area inside the retail store with a first set of frame coordinates of a second coordinate system associated with the plurality of cameras; and cause a database to store the association.
    Type: Application
    Filed: November 2, 2020
    Publication date: May 6, 2021
    Inventors: Yanbin Ye, Zhichun Xiao
  • Publication number: 20210019725
    Abstract: A transaction record is created showing a purchase transaction of a customer. A CV profile showing a list of items in the transaction obtained from images is also obtained. The items in the transaction record are compared to items on the list. When there is a discrepancy, an action to take is determined.
    Type: Application
    Filed: July 16, 2020
    Publication date: January 21, 2021
    Inventors: Zhichun Xiao, Lingfeng Zhang, Jon Hammer, Joseph Duffy, Yao Liu, Sicong Fang, Xiang Yao, Pingyuan Wang, Yu Tao, Tianyi Mao, Yutao Tang, Feiyun Zhu, Han Zhang, Chunmei Wang, Pingjian Yu, Muzzammil Afroz, Haining Liu
  • Publication number: 20200380252
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to detecting egress. In some embodiments, there is provided a system for detecting egress at an entrance including a video camera; a computer; a network; and a control circuit configured to receive live video footage; detect a human and estimate a location of the human; track a location and movement of a detected human; determine that the detected human has moved from a first region to a second region and to a third region; and transmit an alert message that indicates that the human has exited the retail facility through the entrance area. In some embodiments, the systems and methods may be configured to comply with privacy requirements which may vary between jurisdictions. For example, before any recording, collection, capturing or processing of user images (e.g., video image, video footage, etc.), a “consent to capture” process may be implemented.
    Type: Application
    Filed: May 29, 2020
    Publication date: December 3, 2020
    Inventors: Yi Ding, Joseph Duffy, Jiankun Liu, Tianyi Mao, Ryan B. Reagan, Zhichun Xiao, Jinhan Zhang, Shouyi Zhang