Patents by Inventor Zhaoliang Duan

Zhaoliang Duan has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250245710
    Abstract: Examples provide a system for exit computer vision basket matching receipts to shopping carts automatically in real-time as a customer is exiting a retail facility. Computer vision analysis of one or more images of a shopping cart produces a set of recognized items from the images. The set of recognized items are compared to the items identified in each candidate receipt in a plurality of candidate receipts. A confidence score is generated for each candidate receipt based on the number of common items found in both the set of recognized items and each receipt. The score is weighted based on factors such as uniqueness of the items and number of instances of each item. The receipt having the highest weighted confidence score is paired with the shopping cart. The paired receipt and image of the shopping cart are output to a user for verification via a user interface.
    Type: Application
    Filed: January 24, 2025
    Publication date: July 31, 2025
    Inventors: Feiyun Zhu, Huanyu Zang, Wei Wang, Lingfeng Zhang, Mingquan Yuan, Zhaoliang Duan, Colin Grant Mitchell, Yanmei Jin, William Craig Robinson
  • Patent number: 12374115
    Abstract: Systems and methods of detecting and recognizing products on product storage structures of a product storage facility include an image capture device that moves about and captures images of the product storage structures at the product storage facility. A computing device processes the obtained images to detect and identify the products on the product storage structure, crops each of the identified individual products from the image to generate a plurality of cropped images and generates an image histogram template, feature vector template and location information template for each of the cropped images. The cropped images are stored in an electronic database and represent a reference model for each of the identified individual products and are stored in association with the generated image histogram template, feature vector template and location information template to facilitate recognition of products subsequently captured on the product storage structure by the image capture device.
    Type: Grant
    Filed: January 24, 2023
    Date of Patent: July 29, 2025
    Assignee: Walmart Apollo, LLC
    Inventors: Zhaoliang Duan, Benjamin R. Ellison, Lingfeng Zhang, Avinash M. Jade, Raghava Balusu, Mingquan Yuan, Zhichun Xiao
  • Patent number: 12367457
    Abstract: Systems and methods of verifying pairings of price tag labels and products at a product storage facility include an image capture device that moves about and captures images of product storage structures at the product storage facility, and a database that stores inventory data associated with the products stocked at the product storage facility. A computing device processes the images to extract one or more characters from the price tag labels and products detected in the images, correlates the extracted characters to the inventory data to identify a product code and a product name that matches the characters extracted from the price tag labels and the products. If the characters extracted from the price tag and from the on-shelf product to which the price tag label is allocated match, the database is updated to reflect that the price tag label is properly allocated to the appropriate on-shelf product.
    Type: Grant
    Filed: November 9, 2022
    Date of Patent: July 22, 2025
    Assignee: Walmart Apollo, LLC
    Inventors: Lingfeng Zhang, Wei Wang, Avinash Madhusudanrao Jade, Raghava Balusu, Mingquan Yuan, Jing Wang, Han Zhang, Zhaoliang Duan, William Craig Robinson, Jr., Benjamin R. Ellison, Paul Lewis Lobo
  • Publication number: 20250232603
    Abstract: Systems and methods of monitoring inventory of a product storage facility include an image capture device configured to move about the product storage areas of the product storage facility and capture images of the product storage areas from various angles. A computing device coupled to the image capture device obtains the images of the product storage areas captured by the image capture device and processes the obtained images of the product storage areas to detect individual products captured in the obtained images. Based on detection of the individual products captured in the images, the computing device analyzes each of the obtained images to extract meta data from the packaging the individual products to detect one more keywords and determine the locations of the detected keywords on the packaging, and then utilize this information to predict an identity of the products associated with the packaging.
    Type: Application
    Filed: April 2, 2025
    Publication date: July 17, 2025
    Inventors: Lingfeng Zhang, Han Zhang, Abhinav Pachauri, Amit Jhunjhunwala, Ashlin Ghosh, Avinash M. Jade, Raghava Balusu, Srinivas Muktevi, Mingquan Yuan, Zhaoliang Duan, Zhiwei Huang, Tianyi Mao
  • Publication number: 20250218199
    Abstract: Examples provide a system and method for identifying unpaid items in real-time using computer vision object detection and recognition models. Images of a cart are selected based on proximity of the cart to one or more anchor points. The object detection and recognition models analyze the image data and identify a set of items in the selected cart. An e-receipt including a set of paid items is selected from a plurality of active electronic receipts based on matching the set of identified items to the set of paid items. Any unmatched items are added to a set of predicted unpaid items. When a receipt corresponding to the selected e-receipt is scanned, a notification identifying the set of predicted unpaid items is provided to a user device for display, enabling a user to identify any unpaid items in a basket of items quickly and accurately in real-time at store exit.
    Type: Application
    Filed: December 20, 2024
    Publication date: July 3, 2025
    Inventors: Feiyun Zhu, Wei Wang, Lingfeng Zhang, Colin Grant Mitchell, Yanmei Jin, Huanyu Zang, Xin Ma, Michael Alvin Schubert, JR., William Craig Robinson, Zhaoliang Duan, Mingquan Yuan, Nilesh Pandey, Han Zhang, Ketan Shah, Paul Lewis Lobo
  • Publication number: 20250218029
    Abstract: Examples provide a system for filtering the contents of adjacent item storage compartments from item recognition results obtained using computer vision (CV) for more accurate mapping of item locations within a retail environment. A filter manager selects a vertical member and tracks it throughout a series of images generated by an image capture device. The selected vertical member defines at least a portion of a target compartment in each image. The vertical member can be a display case door or a vertical steel bar defining the side of an item storage compartment. A set of adjacent items located on each side of the target compartment is filtered from the item recognition results. The target compartment location is determined based on a location tag of the target compartment. The items remaining after filtering are mapped to the target compartment location while reducing CV item location false positives.
    Type: Application
    Filed: January 3, 2024
    Publication date: July 3, 2025
    Inventors: Han Zhang, Siddhartha Chakraborty, Avinash Madhusudanrao Jade, Lingfeng Zhang, Zhaoliang Duan, Eric W. Rader, William Craig Robinson, Mingquan Yuan, Abhinav Pachauri, Benjamin Ellison, Ishan Arora, Raghava Balusu, Ashlin Ghosh, Rongdong Chai, Ketan Shah, Paul Lobo
  • Publication number: 20250218000
    Abstract: Examples enable pallet tag tracking and cluster voting for more accurate pallet tag management using images of a selected pallet. A tag manager tracks a pallet through multiple images of the pallet to ensure the same pallet appears in every image. If the pallet tag is absent from all the images, a tag missing confidence score is generated that indicates the degree of confidence that the tag is missing from the pallet and not merely out of view. The score is used to prioritize handling of pallet tag missing exceptions. If the pallet tag is present in the images, optical character recognition (OCR) results for each tag image are aggregated into a tag cluster with a confidence score calculated for each result. A pallet tag identification (ID) number is predicted based on the result having the highest confidence score to ensure the pallet tag ID is complete and accurate.
    Type: Application
    Filed: January 3, 2024
    Publication date: July 3, 2025
    Inventors: Han Zhang, Avinash Madhusudanrao Jade, Lingfeng Zhang, Zhaoliang Duan, Mingquan Yuan, Eric W. Rader, Zhiwei Huang, Benjamin Ellison, William Craig Robinson, Siddhartha Chakraborty, Raghava Balusu, Aadarsh Gupta, Jing Wang, Rongdong Chai, Ashlin Ghosh, Oleksandr Viatchaninov
  • Publication number: 20250200504
    Abstract: Examples provide for pallet classification and pallet tag text recognition. The system includes a pallet text manager that classifies a type of pallet tag based on detected lines of text in the pallet tag using a classification model. Qualified lines of text are selected from the detected lines of text based on the classification type and corresponding format of the text. Each qualified line of text is associated with a pallet attribute, such as a pallet identifier (ID), an item ID, or a date of creation of the pallet tag. Attribute values from the set of qualified lines of text are paired with location data for the current location of the pallet. The attribute values and the paired location data are saved in a pallet attribute table. The pallet attributes are used to identify the location of pallets in a retail facility with improved accuracy and efficiency.
    Type: Application
    Filed: December 13, 2023
    Publication date: June 19, 2025
    Inventors: Yilun Chen, Zhaoliang Duan, Lingfeng Zhang, Mingquan Yuan, William Craig Robinson, Ishan Arora, Benjamin Ellison, Eric W. Rader, Elizabeth Ann Siler, Han Zhang, Abhinav Pachaurri, Siddhartha Chakraborty, Raghava Balusu, Ashlin Ghosh, Avinash Madhusudanrao Jade, Subhash Anand, Aadarsh Gupta, Paul Lobo, Ketan Shah, Zhiwei Huang, Jing Wang, Rongdong Chai
  • Patent number: 12333488
    Abstract: Systems and methods of analyzing on-shelf price tag labels and products at a product storage facility include an image capture device that captures one or more images of one or more product storage structures at a product storage facility. A computing device communicatively coupled to the image capture device analyzes the images of the product storage structures captured by the image capture device and detects individual price tag labels and individual products located on the product storage structure. Based on the detection of the price tag labels and the products, the computing device also defines separate product storage spaces of the product storage structure, determines which price tag labels are allocated to which of the separate product storage spaces, and associates in a database the price tag labels allocated to the product storage spaces with the products stored in those product storage spaces.
    Type: Grant
    Filed: October 21, 2022
    Date of Patent: June 17, 2025
    Assignee: Walmart Apollo, LLC
    Inventors: Jing Wang, Han Zhang, Lingfeng Zhang, Zhaoliang Duan, Mingquan Yuan, Wei Wang, Benjamin R. Ellison, Avinash M. Jade, Raghava Balusu, Zhichun Xiao
  • Patent number: 12288408
    Abstract: Systems and methods of monitoring inventory of a product storage facility include an image capture device configured to move about the product storage areas of the product storage facility and capture images of the product storage areas from various angles. A computing device coupled to the image capture device obtains the images of the product storage areas captured by the image capture device and processes the obtained images of the product storage areas to detect individual products captured in the obtained images. Based on detection of the individual products captured in the images, the computing device analyzes each of the obtained images to extract meta data from the packaging the individual products to detect one more keywords and determine the locations of the detected keywords on the packaging, and then utilize this information to predict an identity of the products associated with the packaging.
    Type: Grant
    Filed: October 11, 2022
    Date of Patent: April 29, 2025
    Assignee: Walmart Apollo, LLC
    Inventors: Lingfeng Zhang, Han Zhang, Abhinav Pachauri, Amit Jhunjhunwala, Ashlin Ghosh, Avinash Madhusudanrao Jade, Raghava Balusu, Srinivas Muktevi, Mingquan Yuan, Zhaoliang Duan, Zhiwei Huang, Tianyi Mao
  • Publication number: 20240273863
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to processing captured images. In some embodiments, there is provided a system for processing captured images of objects at a product storage facility including a trained machine learning model; and a control circuit. The control circuit may group a plurality of product identifiers into one or more clusters based on at least one of visual similarity of corresponding images, textual similarity of corresponding associated descriptions, and associated relationships between product identifiers of the plurality of product identifiers; determine clusters having common elements that are at least within a similarity threshold of each other; merge the clusters with the common elements; and generate a mapping dataset used to retrain the trained machine learning model to identify a plurality of objects. The mapping dataset may include a plurality of associations of associated product identifiers to a single object.
    Type: Application
    Filed: February 13, 2023
    Publication date: August 15, 2024
    Inventors: Ashlin Ghosh, Feiyun Zhu, Avinash M. Jade, Lingfeng Zhang, Amit Jhunjhunwala, Raghava Balusu, William Craig Robinson, JR., Benjamin R. Ellison, Srinivas Muktevi, Zhaoliang Duan
  • Publication number: 20240273463
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to processing captured images. In some embodiments, there is provided a system for processing captured images of objects at a product storage facility including a trained machine learning model; and a control circuit. The control circuit may identify a product identifier associated with an object in a captured image; generate predicted product identifiers associated with the object in the captured image based on text identified from the object in the captured image; aggregate the predicted product identifiers; determine a feature of the objects associated with the aggregated predicted product identifiers; determine one or more confusing product identifiers based on a determination of the aggregated predicted product identifiers being associated with the feature; and update a dataset with at least one of the one or more confusing product identifiers and images associated with the one or more confusing product identifiers.
    Type: Application
    Filed: February 13, 2023
    Publication date: August 15, 2024
    Inventors: Abhinav Pachauri, Raghava Balusu, Avinash M. Jade, Lingfeng Zhang, Srinivas Muktevi, Amit Jhunjhunwala, Zhaoliang Duan
  • Publication number: 20240265565
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to processing captured images. In some embodiments, there is provided a system for processing captured images of objects at a product storage facility including a trained machine learning model stored in a memory; and a control circuit. The control circuit may obtain an image at the product storage facility; cluster objects depicted in the image that have same product identifiers into a corresponding group; determine coordinates of each bounding box of each clustered object in the corresponding group; determine a bounding box representative depth value of pixels inside the bounding box of each clustered object; determine an overall representative depth value of the corresponding group based on bounding box representative depth values of clustered objects; and exclude the clustered objects from identified objects in the image upon a determination that the overall representative depth value is greater than a threshold.
    Type: Application
    Filed: February 6, 2023
    Publication date: August 8, 2024
    Inventors: Han Zhang, Yilun Chen, Lingfeng Zhang, Adam Cantor, Avinash M. Jade, Benjamin R. Ellison, William Craig Robinson, JR., Mingquan Yuan, Zhaoliang Duan, Wei Wang
  • Publication number: 20240265663
    Abstract: Systems and methods of pairing product labels with products located on a product storage structure of a product storage facility include an image capture device that captures one or more images of the product storage structure and a computing device that obtains images of the product storage structure captured by the image capture device, analyzes the obtained images to detect product labels and products located on the product storage structure, and crops the detected individual products and individual price tag labels from the images to generate cropped images. Then the computing device stitches the cropped price tag label and product images, receives one or more characters extracted from the portions of the stitched images corresponding to the cropped images, and associates, based on known positional coordinates of the products and product labels in the stitched images, the received extracted characters with corresponding cropped images of the products and product labels.
    Type: Application
    Filed: February 6, 2023
    Publication date: August 8, 2024
    Inventors: Zhaoliang Duan, Mingquan Yuan, Paul Lewis Lobo, Lingfeng Zhang, Avinash M. Jade, Yutao Tang
  • Publication number: 20240257380
    Abstract: Systems and methods of detecting support members of product storage structures that store products at a product storage facility include an image capture device that captures images of a product storage structure including vertical and horizontal support members. A computing device including a control circuit is configured to: obtain the images of the product storage structure captured by the image capture device, stitch the obtained images together to generate a stitched image that depicts the product storage structure, generate a color distribution map of the stitched image of the product storage structure to detect individual ones of the horizontal and vertical support members of the product storage structure.
    Type: Application
    Filed: January 30, 2023
    Publication date: August 1, 2024
    Inventors: Wei Wang, Lingfeng Zhang, Han Zhang, Avinash M. Jade, Mingquan Yuan, Zhaoliang Duan, Siddhartha Chakraborty, Benjamin R. Ellison, William Craig Robinson, JR., Eric W. Rader
  • Publication number: 20240257043
    Abstract: Systems and methods of updating templates for use in recognizing individual products in images captured at a product storage facility include an image capture device that captures one or more images of product storage structure at a product storage facility, a computing device in communication with the image capture device, and an electronic database that stores keyword model templates and feature model templates associated with images of previously recognized individual products detected at the product storage facility.
    Type: Application
    Filed: January 30, 2023
    Publication date: August 1, 2024
    Inventors: Han Zhang, Abhinav Pachauri, Raghava Balusu, Ashlin Ghosh, Avinash M. Jade, Lingfeng Zhang, Srinivas Muktevi, Amit Jhunjhunwala, Zhaoliang Duan
  • Publication number: 20240249505
    Abstract: In some embodiments, apparatuses and methods are provided herein useful to processing captured images of objects at a product storage facility. In some embodiments, there is provided a system for processing captured images of objects including a trained machine learning model and a control circuit. In some embodiments, the trained machine learning model is configured to process unprocessed captured images. In some embodiments, the control circuit is configured to associate each of the processed images into one of a first group, a second group, or a third group; remove at least one processed image associated with the first group from the processed images in accordance with a first processing rule; and output remaining processed images associated with the first group and processed images associated with the second group to be used to retrain the trained machine learning model.
    Type: Application
    Filed: January 24, 2023
    Publication date: July 25, 2024
    Inventors: Raghava Balusu, Avinash M. Jade, Lingfeng Zhang, William C. Robinson, JR., Benjamin R. Ellison, Srinivas Muktevi, Amit Jhunjhunwala, Zhaoliang Duan, Siddhartha Chakraborty, Ashlin Ghosh, Mingquan Yuan
  • Publication number: 20240249524
    Abstract: Systems and methods of detecting and recognizing products on product storage structures of a product storage facility include an image capture device that moves about and captures images of the product storage structures at the product storage facility. A computing device processes the obtained images to detect and identify the products on the product storage structure, crops each of the identified individual products from the image to generate a plurality of cropped images and generates an image histogram template, feature vector template and location information template for each of the cropped images. The cropped images are stored in an electronic database and represent a reference model for each of the identified individual products and are stored in association with the generated image histogram template, feature vector template and location information template to facilitate recognition of products subsequently captured on the product storage structure by the image capture device.
    Type: Application
    Filed: January 24, 2023
    Publication date: July 25, 2024
    Inventors: Zhaoliang Duan, Benjamin R. Ellison, Lingfeng Zhang, Avinash M. Jade, Raghava Balusu, Mingquan Yuan, Zhichun Xiao
  • Publication number: 20240249239
    Abstract: Systems and methods of creating reference template images for detecting and recognizing products at a product storage facility include an image capture device having a field of view that includes a product storage structure of the product storage facility, and a computing device including a control circuit and being communicatively coupled to the image capture device. The computing device obtains images of the product storage structure captured by the image capture device, analyzes the obtained images to detect individual ones of the products located on the product storage structure. Then, the computing device identifies the individual ones of the products detected in the images and crops each of the individual ones of the identified products from the images to generate cropped images. The computing device then creates a cluster of the cropped images, and selects one of the cropped images as a reference template image of an identified individual product.
    Type: Application
    Filed: January 24, 2023
    Publication date: July 25, 2024
    Inventors: Ashlin Ghosh, Raghava Balusu, Abhinav Pachauri, Avinash M. Jade, Lingfeng Zhang, Amit Jhunjhunwala, William Craig Robinson, JR., Benjamin R. Ellison, Srinivas Muktevi, Zhaoliang Duan
  • Publication number: 20240232795
    Abstract: Systems and methods of analyzing on-shelf price tag labels and products at a product storage facility include an image capture device that captures one or more images of one or more product storage structures at a product storage facility. A computing device communicatively coupled to the image capture device analyzes the images of the product storage structures captured by the image capture device and detects individual price tag labels and individual products located on the product storage structure. Based on the detection of the price tag labels and the products, the computing device also defines separate product storage spaces of the product storage structure, determines which price tag labels are allocated to which of the separate product storage spaces, and associates in a database the price tag labels allocated to the product storage spaces with the products stored in those product storage spaces.
    Type: Application
    Filed: October 21, 2022
    Publication date: July 11, 2024
    Inventors: Jing Wang, Han Zhang, Lingfeng Zhang, Zhaoliang Duan, Mingquan Yuan, Wei Wang, Benjamin R. Ellison, Avinash M. Jade, Raghava Balusu, Zhichun Xiao