Patents by Inventor Shaomin Xiong

Shaomin Xiong has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230004742
    Abstract: Systems and methods described herein provide for the use and storage of intermediate layer data within a neural network processing system. A first neural network, such as an object detection neural network may receive and process raw video image data to generate output utilized for metadata creation. Secondary neural networks may be configured to accept input data from one or more intermediate layers of the first neural network instead of the raw video image data. In this way, the initial data processed by the intermediate layers of the first neural network can be stored and utilized as a shortcut for processing additional features or attributes within the video image data. This alleviates the need to process video image data multiple times in different neural networks. The intermediate layer data can be stored in a different and often cheaper storage system and recalled faster and with fewer resources for future use.
    Type: Application
    Filed: June 30, 2021
    Publication date: January 5, 2023
    Inventors: Shaomin Xiong, Toshiki Hirano, Ramy Ayad, Damien Kah
  • Publication number: 20220414382
    Abstract: Systems and methods for site-based calibration of object detection values, such as for surveillance video cameras, are described. Video data from a video image sensor may be processed using an object detector to determine object data and a confidence score for a detected object. The object data and confidence score may be post-processed to apply calibration values based on the camera location to one or more parameters used for determining detection events. Event notifications may be sent for detection events. The calibration values may be determined from a calibration period where a verification object detector is used to verify the object detections and failure analysis is applied to determine calibration values for the camera location.
    Type: Application
    Filed: June 25, 2021
    Publication date: December 29, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Damien Kah
  • Publication number: 20220374059
    Abstract: A data storage device including, in one implementation, a non-volatile memory device having a memory block including a number of memory dies, and a controller coupled to the non-volatile memory device. The controller is configured to monitor a temperature of the data storage device and determine whether the monitored temperature exceeds a first temperature threshold. The controller is also configured to perform a default thermal throttling operation based on the monitored temperature exceeding the first temperature threshold, determine whether the monitored temperature exceeds a second temperature threshold, and perform a customized thermal throttling operation based on the monitored temperature exceeding the second temperature threshold.
    Type: Application
    Filed: May 21, 2021
    Publication date: November 24, 2022
    Inventors: Shaomin Xiong, Qian Zhong, Toshiki Hirano
  • Publication number: 20220374635
    Abstract: Systems and methods for processing surveillance video streams using image classification and object detection are described. Video data from a video image sensor may be processed using an image classifier to determine whether an object type is present in a video frame. If the object type is present, the video frame and/or subsequent video frames may be processed using an object detector to provide additional object data, such as position information, for use in other video surveillance processes. In some examples, an event message may be generated and sent to a video surveillance application in response to selective object detection.
    Type: Application
    Filed: May 21, 2021
    Publication date: November 24, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Damien Kah, Rajeev Nagabhirava, David Berman
  • Patent number: 11509832
    Abstract: Systems and methods for a low light surveillance system with dual video streams are described. A video camera may include an image sensor and an infrared blocking filter. Responsive to detecting low light conditions, the infrared blocking filter may be disabled and the full spectrum video data from the camera may be sent in one video stream that is processed for user display, such as rendering in gray scale, and another video stream that is used for analytics processing, such as event detection.
    Type: Grant
    Filed: February 16, 2021
    Date of Patent: November 22, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Pritam Das, Ramy Ayad
  • Patent number: 11496671
    Abstract: Systems and methods for surveillance video streams with embedded object data from another video camera are described. At least two video cameras are configured with fields of view to provide images of an object from alternative views. Video data for a primary video stream is received from one camera and secondary object data for the object from the other camera is embedded in the primary video stream. The primary video stream is sent to an analytics engine for processing the primary video and embedded secondary object data, such as performing facial recognition on a better image of a human face and/or feature vectors therefrom that are embedded in the primary video stream.
    Type: Grant
    Filed: February 19, 2021
    Date of Patent: November 8, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Toshiki Hirano, Qian Zhong, Haoyu Wu, David Berman
  • Patent number: 11429285
    Abstract: An example system and method may provide for important data files to be stored in differently reliable storage medium based upon an importance score that is representative of the importance of the data file. A data file importance score is generated based on the data file. The data file is stored in a first data storage zone of a storage medium in a data storage device, in response to the data file importance score being greater than a data file importance threshold. The data file is stored in a second data storage zone of the storage medium in the data storage device, in response to the data file importance score being less than the data file importance threshold.
    Type: Grant
    Filed: June 29, 2020
    Date of Patent: August 30, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Toshiki Hirano
  • Patent number: 11429620
    Abstract: An example system and method may provide an importance score for a data file based on the content of the data file. An importance score may be used to determine whether to store the data file in a regular reliability storage media or in a higher reliability storage media. A controller generates a document vector for a data file based on content processed from a data file. The data file includes metadata and the content. The controller generates, using an artificial intelligence (AI) model and the document vector, a data file importance score for the data file. The controller then stores the data file in one of the first data storage zone and the second data storage zone based on the data file importance score.
    Type: Grant
    Filed: June 29, 2020
    Date of Patent: August 30, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Toshiki Hirano
  • Patent number: 11429163
    Abstract: A data storage system may include multiple data storage devices, such as solid-state drives, an enclosure housing the devices, and a thermal bridge positioned in a gap between and in contact with each of the enclosure and a device, where the enclosure is cooler than the device. Thus, heat is conducted away from a hot spot of the device and to the enclosure. The thermal bridge may be flexible enough to bridge different sized gaps, while stiff enough to generate contact forces applied to the enclosure and the device. For example, the thermal bridge may be constructed primarily of copper and configured to function like a compression spring.
    Type: Grant
    Filed: May 20, 2019
    Date of Patent: August 30, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Hussam Zebian, Haoyu Wu, Toshiki Hirano, Jeff Wilke, Timothy Skarzynski
  • Publication number: 20220272255
    Abstract: Systems and methods for surveillance video streams with embedded object data from another video camera are described. At least two video cameras are configured with fields of view to provide images of an object from alternative views. Video data for a primary video stream is received from one camera and secondary object data for the object from the other camera is embedded in the primary video stream. The primary video stream is sent to an analytics engine for processing the primary video and embedded secondary object data, such as performing facial recognition on a better image of a human face and/or feature vectors therefrom that are embedded in the primary video stream.
    Type: Application
    Filed: February 19, 2021
    Publication date: August 25, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Qian Zhong, Haoyu Wu, David Berman
  • Patent number: 11366602
    Abstract: A data storage device is disclosed comprising a non-volatile storage medium (NVSM), and a head configured to access the NVSM. During a first interval, the head is used to write first data to a first segment of the NVSM, and during a second interval, the head is used to read the first data from the first segment of the NVSM and erase at least part of the first data from the first segment of the NVSM.
    Type: Grant
    Filed: June 23, 2020
    Date of Patent: June 21, 2022
    Assignee: WESTERN DIGITAL TECHNOLOGIES, INC.
    Inventors: Shaomin Xiong, Toshiki Hirano, William B. Boyle
  • Publication number: 20220189155
    Abstract: This disclosure relates to an apparatus for object detection. The apparatus comprises a video camera, an object detector, and a controller. The video camera may be configured to generate a video stream of frames. The object detector may be configured to accept the video stream as input data and to perform object detection. The controller may be coupled to the video camera and the object detector. The controller may be configured to manage object detection in order to satisfy a performance metric and/or operate within an operational constraint.
    Type: Application
    Filed: March 1, 2022
    Publication date: June 16, 2022
    Inventors: Haoyu WU, Shaomin XIONG, Toshiki HIRANO
  • Publication number: 20220174200
    Abstract: Systems and methods for a low light surveillance system with dual video streams are described. A video camera may include an image sensor and an infrared blocking filter. Responsive to detecting low light conditions, the infrared blocking filter may be disabled and the full spectrum video data from the camera may be sent in one video stream that is processed for user display, such as rendering in gray scale, and another video stream that is used for analytics processing, such as event detection.
    Type: Application
    Filed: February 16, 2021
    Publication date: June 2, 2022
    Inventors: Shaomin Xiong, Pritam Das, Ramy Ayad
  • Publication number: 20220172700
    Abstract: Systems and methods for audio privacy in network video surveillance systems are described. A video camera may include an image sensor and a microphone to generate a video stream. Responsive to detecting a human speaking condition in the video stream, the audio data may be selectively modified to mask a human voice component of the audio data for storing and/or displaying the surveillance video stream.
    Type: Application
    Filed: February 16, 2021
    Publication date: June 2, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Pritam Das, Ramy Ayad, Rajeev Nagabhirava
  • Publication number: 20220150310
    Abstract: In one embodiment, a sensor device includes a sensor configured to obtain sensor data and a communication interface configured to communicate with a set of secondary sensor devices. The set of secondary sensor devices comprise a set of secondary sensors and a set of secondary processing devices. The set of secondary sensors obtain additional sensor data. The set of secondary processing devices process the additional sensor data based on a set of secondary machine learning models, and generate activation data based on the additional sensor data. The sensor device also includes a processing device. The processing device is also configured to receive the activation data from the set of secondary sensor devices and cause the sensor to obtain first sensor data based on the activation data. The processing device is further configured to generate one or more inferences based on the first sensor data and a machine learning model.
    Type: Application
    Filed: January 24, 2022
    Publication date: May 12, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Haoyu Wu
  • Publication number: 20220116569
    Abstract: A surveillance system includes at least one image or video capture device and a controller configured to determine a change in location for the at least one image or video capture device from a first location to a second location. First image or video data is received from the at least one image or video capture device at the second location, and in response to the location change, a preconfigured neural network is obtained or weights for a neural network are obtained based at least in part on the received first image or video data. Second image or video data is received from the at least one image or video capture device at the second location and an inference operation is performed on the second image or video data by processing the second image or video data using the obtained weights or the obtained preconfigured neural network.
    Type: Application
    Filed: December 24, 2021
    Publication date: April 14, 2022
    Inventors: Shaomin Xiong, Toshiki Hirano, Haoyu Wu
  • Patent number: 11301705
    Abstract: This disclosure relates to an apparatus for object detection. The apparatus comprises a video camera, an object detector, and a controller. The video camera may be configured to generate a video stream of frames. The object detector may be configured to accept the video stream as input data and to perform object detection. The controller may be coupled to the video camera and the object detector. The controller may be configured to manage object detection in order to satisfy a performance metric and/or operate within an operational constraint.
    Type: Grant
    Filed: February 27, 2020
    Date of Patent: April 12, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Haoyu Wu, Shaomin Xiong, Toshiki Hirano
  • Patent number: 11233956
    Abstract: In one embodiment, a sensor device includes a sensor configured to obtain sensor data and a communication interface configured to communicate with a set of secondary sensor devices. The set of secondary sensor devices comprise a set of secondary sensors and a set of secondary processing devices. The set of secondary sensors obtain additional sensor data. The set of secondary processing devices process the additional sensor data based on a set of secondary machine learning models, and generate activation data based on the additional sensor data. The sensor device also includes a processing device. The processing device is also configured to receive the activation data from the set of secondary sensor devices and cause the sensor to obtain first sensor data based on the activation data. The processing device is further configured to generate one or more inferences based on the first sensor data and a machine learning model.
    Type: Grant
    Filed: March 31, 2020
    Date of Patent: January 25, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Toshiki Hirano, Haoyu Wu
  • Patent number: 11232327
    Abstract: Remote neural network retraining for surveillance systems is disclosed. In some cases, systems and methods enable remote training and retraining of neural networks, while processing data in real life using the trained and retrained neural networks locally at the surveillance system. The surveillance system can determine a change of location and can retrain the neural network and/or initiate the retraining of the neural network remotely, such as on a cloud server, to retrain the neural network based on new image and/or video data taken from the new location. The remote server can transmit the retrained neural network and/or weights for nodes of the retrained neural network back to the surveillance system. The surveillance system can then update its neural network and process future image and/or video data based on the retrained neural network and/or weights.
    Type: Grant
    Filed: June 19, 2019
    Date of Patent: January 25, 2022
    Assignee: Western Digital Technologies, Inc.
    Inventors: Shaomin Xiong, Toshiki Hirano, Haoyu Wu
  • Publication number: 20210405887
    Abstract: An example system and method may provide for important data files to be stored in differently reliable storage medium based upon an importance score that is representative of the importance of the data file. A data file importance score is generated based on the data file. The data file is stored in a first data storage zone of a storage medium in a data storage device, in response to the data file importance score being greater than a data file importance threshold. The data file is stored in a second data storage zone of the storage medium in the data storage device, in response to the data file importance score being less than the data file importance threshold.
    Type: Application
    Filed: June 29, 2020
    Publication date: December 30, 2021
    Inventors: Shaomin Xiong, Toshiki Hirano