Patents by Inventor Yunchao Gong

Yunchao Gong has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11900688
    Abstract: A computer vision processor of a camera generates hyperzooms for persons or vehicles from image frames captured by the camera. The hyperzooms include a first hyperzoom associated with the persons or vehicles. The computer vision processor tracks traffic patterns of the persons or vehicles while obviating network usage by the camera by predicting positions of the persons or vehicles using a Kalman Filter from the first hyperzoom. The persons or vehicles are detected in the second hyperzoom. The positions of the persons or vehicles are updated based on detecting the persons or vehicles in the second hyperzoom. The first hyperzoom is removed from the camera. Tracks of the persons or vehicles are generated based on the updated positions. The second hyperzoom is removed from the camera. Track metadata is generated from the tracks for storing in a key-value database located on a non-transitory computer-readable storage medium of the camera.
    Type: Grant
    Filed: January 17, 2023
    Date of Patent: February 13, 2024
    Assignee: Verkada Inc.
    Inventors: Yi Xu, Mayank Gupta, Xia Yang, Yuanyuan Chen, Zixiao (Shawn) Wang, Qiang (Kevin) Fu, Yunchao Gong, Naresh Nagabushan
  • Publication number: 20230394850
    Abstract: A method and related system operations includes, determining, for each respective frame of a frame sequence, a respective bounding box that surrounds a respective sub-image of the respective frame, determining a respective string and respective confidence values associated with the respective string based on the respective sub-image, updating the tracklet to comprise the respective bounding box based on the respective string and at least one string generated by the object recognition model for a previous frame, and updating a voting table by adding the respective confidence values to the voting table. The method also includes generating an aggregated string based on the voting table by, for a set of positions of the aggregated string, determining a character associated with a maximum confidence value indicated by the voting table and associating the aggregated string with the tracklet in a data structure.
    Type: Application
    Filed: August 22, 2023
    Publication date: December 7, 2023
    Inventors: Mayank GUPTA, Suraj Arun VATHSA, Song CAO, Yi XU, Yuanyuan CHEN, Yunchao GONG
  • Patent number: 11836185
    Abstract: A computing system receives, from a client device, image data describing an image captured by an optical sensor of the client device. The computing system compares the image to a set of reference images that include associated metadata describing a real-world feature depicted by the respective reference image. The computing system determines, based on the comparison, a subset of reference images that are similar to the image, and then determines, based on associated metadata of the subset of reference images, that the image captured by the optical sensor of the client device depicts a first real-world feature. The computing system selects a subset of media overlays related to the first real-world feature based on metadata associated with each media overlay that describes the respective media overlay. The computing system transmits the subset of media overlays to the client device.
    Type: Grant
    Filed: July 21, 2021
    Date of Patent: December 5, 2023
    Assignee: Snap Inc.
    Inventors: Kirk Ouimet, Yunchao Gong
  • Publication number: 20230367808
    Abstract: A computer vision processor of a camera extracts attributes of persons or vehicles from hyperzooms generated from image frames. The hyperzooms represent traffic patterns. The extracting is performed using a feature extractor of an on-camera convolutional neural network (CNN) including an inverted residual structure. The attributes include at least colors of clothing of the persons or colors of the vehicles. Mobile semantic segmentation models of the CNN are generated using the hyperzooms and the attributes. Attribute analytics are generated by executing the mobile semantic segmentation models while obviating network usage by the camera. The attribute analytics are stored in a key-value database located on a memory card of the camera. A query is received from the server instance specifying one or more of the attributes. The attribute analytics are filtered using the one or more of the attributes to obtain a portion of the traffic patterns.
    Type: Application
    Filed: July 5, 2023
    Publication date: November 16, 2023
    Inventors: Yi XU, Mayank GUPTA, Xia YANG, Yuanyuan CHEN, Zixiao (Shawn) WANG, Qiang (Kevin) FU, Yunchao GONG, Naresh NAGABUSHAN
  • Publication number: 20230368332
    Abstract: The technical problem of removing an object depicted in a selected region of an image to create a natural-looking edited image is addressed by providing systems, methods, and computer-readable storage media to perform automatic image inpainting. The method includes replacing the selected region using a color mask. A color mask can be generated using a mean color of pixels from a portion of the image that is distinct from and outside of the selected region.
    Type: Application
    Filed: May 12, 2023
    Publication date: November 16, 2023
    Inventors: Kun Duan, Yunchao Gong, Nan Hu
  • Publication number: 20230343114
    Abstract: A method and related system operations include obtaining a video stream with an image sensor of a camera device, detecting a plurality of target objects by executing a neural network model based on the video stream with a vision processor unit of the camera device. The method also includes generating a plurality of bounding boxes, determining a plurality of character sequences by, for each respective bounding box of the plurality of bounding boxes, performing a set of optical character recognition (OCR) operations to determine a respective character sequence of the plurality of character sequences. The method also includes updating a plurality of tracklets to indicate the plurality of bounding boxes and storing the plurality of tracklets in association with the plurality of character sequences in a memory of the camera device.
    Type: Application
    Filed: February 13, 2023
    Publication date: October 26, 2023
    Inventors: Mayank GUPTA, Suraj Arun VATHSA, Song CAO, Yi XU, Yuanyuan CHEN, Yunchao GONG
  • Publication number: 20230343113
    Abstract: Automatic license plate recognition occurs when a light sensor that continually captures video detects motion as a vehicle is driven through a gate. The light sensor detects the vehicle and license plate in the video stream captured by the light sensor. An algorithm associated with the video stream of the light sensor is trained to detect license plates. The light sensor starts executing the recognition algorithm when it detects motion. Recognition of characters in the license plate is based upon an aggregation of several captured video frames in which a license plate is detected.
    Type: Application
    Filed: December 12, 2022
    Publication date: October 26, 2023
    Inventors: Yi XU, Yunchao GONG, Suraj Arun VATHSA, Mayank GUPTA, Naresh NAGABUSHAN
  • Publication number: 20230327848
    Abstract: A method includes obtaining a plurality of representative vectors associated with face-related data. The method includes determining an encryption key based on a parameter stored in a record, generating an encrypted vector set by, for each respective vector of the plurality of representative vectors, encrypting the respective vector with a homomorphic encryption operation based on the encryption key, where the encrypted vector set includes a first encrypted vector that is linked to a subset of the face-related data associated with the first plurality of face vectors. The method further includes obtaining an encrypted face search vector using the encryption key to perform homomorphic encryption. The method further includes selecting a first encrypted vector based on the encrypted face search vector and retrieving the subset of the face-related data based on the first encrypted vector.
    Type: Application
    Filed: September 30, 2022
    Publication date: October 12, 2023
    Inventors: Kiumars SOLTANI, Yuewei WANG, Kabir CHHABRA, Jose M. GIRON NANNE, Yunchao GONG
  • Publication number: 20230305689
    Abstract: Disclosed are systems, methods, and computer-readable storage media to perform an interactive image recolorization process. The method includes receiving user input including a stroke drawn on an image presented on a client device. The stroke comprises a user-specified color. The method further includes determining a region of interest in the image. The method further includes recolorizing the region of interest on the image based on the user-specified color and causing presentation of a result of the recolorization on the client device.
    Type: Application
    Filed: June 2, 2023
    Publication date: September 28, 2023
    Inventors: Kun Duan, Yunchao Gong, Nan Hu
  • Publication number: 20230298359
    Abstract: A computer vision processor of a camera generates hyperzooms for persons or vehicles from image frames captured by the camera. The hyperzooms include a first hyperzoom associated with the persons or vehicles. The computer vision processor tracks traffic patterns of the persons or vehicles while obviating network usage by the camera by predicting positions of the persons or vehicles using a Kalman Filter from the first hyperzoom. The persons or vehicles are detected in the second hyperzoom. The positions of the persons or vehicles are updated based on detecting the persons or vehicles in the second hyperzoom. The first hyperzoom is removed from the camera. Tracks of the persons or vehicles are generated based on the updated positions. The second hyperzoom is removed from the camera. Track metadata is generated from the tracks for storing in a key-value database located on a non-transitory computer-readable storage medium of the camera.
    Type: Application
    Filed: January 17, 2023
    Publication date: September 21, 2023
    Inventors: Yi XU, Mayank GUPTA, Xia YANG, Yuanyuan CHEN, Zixiao (Shawn) WANG, Qiang (Kevin) FU, Yunchao GONG, Naresh NAGABUSHAN
  • Patent number: 11734343
    Abstract: A computer vision processor of a camera extracts attributes of persons or vehicles from hyperzooms generated from image frames. The hyperzooms represent traffic patterns. The extracting is performed using a feature extractor of an on-camera convolutional neural network (CNN) including an inverted residual structure. The attributes include at least colors of clothing of the persons or colors of the vehicles. Mobile semantic segmentation models of the CNN are generated using the hyperzooms and the attributes. Attribute analytics are generated by executing the mobile semantic segmentation models while obviating network usage by the camera. The attribute analytics are stored in a key-value database located on a memory card of the camera. A query is received from the server instance specifying one or more of the attributes. The attribute analytics are filtered using the one or more of the attributes to obtain a portion of the traffic patterns.
    Type: Grant
    Filed: January 17, 2023
    Date of Patent: August 22, 2023
    Assignee: Verkada Inc.
    Inventors: Yi Xu, Mayank Gupta, Xia Yang, Yuanyuan Chen, Zixiao (Shawn) Wang, Qiang (Kevin) Fu, Yunchao Gong, Naresh Nagabushan
  • Patent number: 11687220
    Abstract: Disclosed are systems, methods, and computer-readable storage media to perform an interactive image recolorization process. The method includes receiving user input including a stroke drawn on an image presented on a client device. The stroke comprises a user-specified color. The method further includes determining a region of interest in the image. The method further includes recolorizing the region of interest on the image based on the user-specified color and causing presentation of a result of the recolorization on the client device.
    Type: Grant
    Filed: August 4, 2021
    Date of Patent: June 27, 2023
    Assignee: Snap Inc.
    Inventors: Kun Duan, Yunchao Gong, Nan Hu
  • Patent number: 11682105
    Abstract: The technical problem of removing an object depicted in a selected region of an image to create a natural-looking edited image is addressed by providing systems, methods, and computer-readable storage media to perform automatic image inpainting. The method includes replacing the selected region using a color mask. A color mask can be generated using a mean color of pixels from a portion of the image that is distinct from and outside of the selected region.
    Type: Grant
    Filed: August 12, 2021
    Date of Patent: June 20, 2023
    Assignee: SNAP INC.
    Inventors: Kun Duan, Yunchao Gong, Nan Hu
  • Patent number: 11676418
    Abstract: A method includes generating a first representative vector based on a first vectors, wherein the first representative vector is associated with the first vectors in a collection of representative vectors, and the first vectors comprises a set of vector values within a latent space. The method further includes generating a second representative vector based on a second vectors, wherein the second representative vector is associated with the second vectors in the collection of representative vectors. The method further includes determining a latent space distance based on the first and second vectors. The method further includes determining whether the latent space distance satisfies a threshold. In response to a determination that the latent space distance satisfies the threshold, the method further includes associating a combined representative vector with the first vectors and the second vectors and removing the first and second representative vectors from the collection of representative vectors.
    Type: Grant
    Filed: November 18, 2022
    Date of Patent: June 13, 2023
    Assignee: Verkada Inc.
    Inventors: Kiumars Soltani, Yuewei Wang, Kabir Chhabra, Jose M. Giron Nanne, Yunchao Gong
  • Patent number: 11594043
    Abstract: A computer vision processor of a camera generates hyperzooms for persons or vehicles from image frames captured by the camera. The hyperzooms include a first hyperzoom associated with the persons or vehicles. The computer vision processor tracks traffic patterns of the persons or vehicles while obviating network usage by the camera by predicting positions of the persons or vehicles using a Kalman Filter from the first hyperzoom. The persons or vehicles are detected in the second hyperzoom. The positions of the persons or vehicles are updated based on detecting the persons or vehicles in the second hyperzoom. The first hyperzoom is removed from the camera. Tracks of the persons or vehicles are generated based on the updated positions. The second hyperzoom is removed from the camera. Track metadata is generated from the tracks for storing in a key-value database located on a non-transitory computer-readable storage medium of the camera.
    Type: Grant
    Filed: March 21, 2022
    Date of Patent: February 28, 2023
    Assignee: Verkada Inc.
    Inventors: Yi Xu, Mayank Gupta, Xia Yang, Yuanyuan Chen, Zixiao (Shawn) Wang, Qiang (Kevin) Fu, Yunchao Gong, Naresh Nagabushan
  • Patent number: 11586667
    Abstract: A computer vision processor of a camera extracts attributes of persons or vehicles from hyperzooms generated from image frames. The hyperzooms represent traffic patterns. The extracting is performed using a feature extractor of an on-camera convolutional neural network (CNN) including an inverted residual structure. The attributes include at least colors of clothing of the persons or colors of the vehicles. Mobile semantic segmentation models of the CNN are generated using the hyperzooms and the attributes. Attribute analytics are generated by executing the mobile semantic segmentation models while obviating network usage by the camera. The attribute analytics are stored in a key-value database located on a memory card of the camera. A query is received from the server instance specifying one or more of the attributes. The attribute analytics are filtered using the one or more of the attributes to obtain a portion of the traffic patterns.
    Type: Grant
    Filed: July 20, 2022
    Date of Patent: February 21, 2023
    Assignee: Verkada Inc.
    Inventors: Yi Xu, Mayank Gupta, Xia Yang, Yuanyuan Chen, Zixiao (Shawn) Wang, Qiang (Kevin) Fu, Yunchao Gong, Naresh Nagabushan
  • Patent number: 11557133
    Abstract: Automatic license plate recognition occurs when a light sensor that continually captures video detects motion as a vehicle is driven through a gate. The light sensor detects the vehicle and license plate in the video stream captured by the light sensor. An algorithm associated with the video stream of the light sensor is trained to detect license plates. The light sensor starts executing the recognition algorithm when it detects motion. Recognition of characters in the license plate is based upon an aggregation of several captured video frames in which a license plate is detected.
    Type: Grant
    Filed: July 22, 2022
    Date of Patent: January 17, 2023
    Assignee: Verkada Inc.
    Inventors: Yi Xu, Yunchao Gong, Suraj Arun Vathsa, Mayank Gupta, Naresh Nagabushan
  • Publication number: 20230011368
    Abstract: Among other things, embodiments of the present disclosure improve the functionality of electronic messaging and imaging software and systems by enabling users to generate customized media overlays that can be shared with other users. For example, media overlays can be generated by the system and displayed in conjunction with media content (e.g., images and/or video) generated by an image-capturing device (e.g., a digital camera). In some embodiments, existing media overlays may be used by users to create derivative media overlays. The system may track usage of media overlays and any derivatives created based thereon, and allow users to control the distribution and use of their overlays in future derivatives. In some embodiments, for example, a user can modify an overlay they created and cause the modification to propagate to all derivative overlays based on the user’s overlay.
    Type: Application
    Filed: July 25, 2022
    Publication date: January 12, 2023
    Inventors: Newar Husam Al Majid, Sheldon Chang, Yunchao Gong, Kirk Ouimet, Haibo Zhao
  • Patent number: 11514714
    Abstract: A method includes generating a first representative vector based on a first vectors, wherein the first representative vector is associated with the first vectors in a collection of representative vectors, and the first vectors comprises a set of vector values within a latent space. The method further includes generating a second representative vector based on a second vectors, wherein the second representative vector is associated with the second vectors in the collection of representative vectors. The method further includes determining a latent space distance based on the first and second vectors. The method further includes determining whether the latent space distance satisfies a threshold. In response to a determination that the latent space distance satisfies the threshold, the method further includes associating a combined representative vector with the first vectors and the second vectors and removing the first and second representative vectors from the collection of representative vectors.
    Type: Grant
    Filed: April 8, 2022
    Date of Patent: November 29, 2022
    Assignee: Verkada Inc.
    Inventors: Kiumars Soltani, Yuewei Wang, Kabir Chhabra, Jose M. Giron Nanne, Yunchao Gong
  • Patent number: 11496288
    Abstract: A method includes obtaining a plurality of representative vectors associated with face-related data. The method includes determining an encryption key based on a parameter stored in a record, generating an encrypted vector set by, for each respective vector of the plurality of representative vectors, encrypting the respective vector with a homomorphic encryption operation based on the encryption key, where the encrypted vector set includes a first encrypted vector that is linked to a subset of the face-related data associated with the first plurality of face vectors. The method further includes obtaining an encrypted face search vector using the encryption key to perform homomorphic encryption. The method further includes selecting a first encrypted vector based on the encrypted face search vector and retrieving the subset of the face-related data based on the first encrypted vector.
    Type: Grant
    Filed: April 8, 2022
    Date of Patent: November 8, 2022
    Assignee: Verkada Inc.
    Inventors: Kiumars Soltani, Yuewei Wang, Kabir Chhabra, Jose M. Giron Nanne, Yunchao Gong