Patents by Inventor Yuanchao SHU

Yuanchao SHU has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240119089
    Abstract: This document relates to performing live video stream analytics on edge devices. One example determines resources available to the system, and a video analytics configuration is selected that distributes work between edge devices and cloud devices in a cascading manner, where edge device processing is prioritized over cloud processing in order to conserve resources. This example can dynamically modify the allocation of processing depending on changing conditions, such as network availability.
    Type: Application
    Filed: December 12, 2023
    Publication date: April 11, 2024
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Shadi NOGHABI, Paramvir BAHL, Landon COX, Alexander CROWN
  • Publication number: 20240121081
    Abstract: An access control system is disclosed for controlling access to a resource. A request is received by a location attribute policy (LAP) server to access an encrypted resource. The LAP server accesses a resource policy that identifies requirements for granting access to the encrypted resource, such as a list of attributes of the requestor that are required and a dynamic attribute requirement of the requestor. The LAP server receives a cryptographic proof from the computing device that the requestor possesses the attributes and validates the proof based at least on information obtained from a trusted ledger. Once the proof is validated, the LAP server provides a shared secret associated with the dynamic attribute requirement to a decryption algorithm. The decryption algorithm uses the dynamic attribute shared secret in combination with one or more attribute shared secrets from the requestor to generate a decryption key for the encrypted resource.
    Type: Application
    Filed: October 10, 2022
    Publication date: April 11, 2024
    Inventors: Ramarathnam VENKATESAN, Nishanth CHANDRAN, Ganesh ANANTHANARAYANAN, Panagiotis ANTONOPOULOS, Srinath T.V. SETTY, Daniel John CARROLL, JR., Kiran MUTHABATULLA, Yuanchao SHU, Sanjeev MEHROTRA
  • Publication number: 20240096063
    Abstract: Systems and methods are provided for reusing and retraining an image recognition model for video analytics. The image recognition model is used for inferring a frame of video data that is captured at edge devices. The edge devices periodically or under predetermined conditions transmits a captured frame of video data to perform inferencing. The disclosed technology is directed to select an image recognition model from a model store for reusing or for retraining. A model selector uses a gating network model to determine ranked candidate models for validation. The validation includes iterations of retraining the image recognition model and stopping the iteration when a rate of improving accuracy by retraining becomes smaller than the previous iteration step. Retraining a model includes generating reference data using a teacher model and retraining the model using the reference data. Integrating reuse and retraining of models enables improvement in accuracy and efficiency.
    Type: Application
    Filed: December 9, 2022
    Publication date: March 21, 2024
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Paramvir BAHL, Tsuwang HSIEH
  • Patent number: 11860975
    Abstract: Provided are aspects relating to methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service includes a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.
    Type: Grant
    Filed: September 20, 2022
    Date of Patent: January 2, 2024
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ganesh Ananthanarayanan, Yuanchao Shu, Tsu-wang Hsieh, Nikolaos Karianakis, Paramvir Bahl, Romil Bhardwaj
  • Patent number: 11831698
    Abstract: Systems and methods are provided for reducing stream data according to a data streaming protocol under a multi-access edge computing. In particular, an IoT device, such as a video image sensing device, may capture stream data and generate inference data by applying a machine-learning model trained to infer data based on the captured stream data. The inference data represents the captured stream data in a reduced data size based on performing data analytics on the captured data. The IoT device formats the inference data according to the data streaming protocol. In contrast to video data compression, the data streaming protocol includes instructions for transmitting the reduced volume of inference data through a data analytics pipeline.
    Type: Grant
    Filed: June 29, 2021
    Date of Patent: November 28, 2023
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ganesh Ananthanarayanan, Yu Yan, Yuanchao Shu
  • Patent number: 11627095
    Abstract: Computing resources are managed in a computing environment comprising a computing service provider and an edge computing network. The edge computing network comprises computing and storage devices configured to extend computing resources of the computing service provider to remote users of the computing service provider. The edge computing network collects capacity and usage data for computing and network resources at the edge computing network. The capacity and usage data is sent to the computing service provider. Based on the capacity and usage data, the computing service provider, using a cost function, determines a distribution of workloads pertaining to a processing pipeline that has been partitioned into the workloads. The workloads can be executed at the computing service provider or the edge computing network.
    Type: Grant
    Filed: June 15, 2021
    Date of Patent: April 11, 2023
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Ganesh Ananthanarayanan, Yuanchao Shu, Paramvir Bahl
  • Publication number: 20230030499
    Abstract: Examples are disclosed that relate to methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service comprises a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.
    Type: Application
    Filed: September 20, 2022
    Publication date: February 2, 2023
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Tsu-wang HSIEH, Nikolaos KARIANAKIS, Paramvir BAHL, Romil BHARDWAJ
  • Publication number: 20220414534
    Abstract: Systems and methods are provided for continuous learning of models across hierarchies under a multi-access edge computing. In particular, an on-premises edge server, using a model, generates inference data associated with captured stream data. A data drift determiner determines a data drift in the inference data by comparing the data against reference data generated using a golden model. The data drift indicates a loss of accuracy in the inference data. A gateway model maintains one or more models in a model cache for update the model. The gateway model instructs the one or more servers to train the new model. The gateway model transmits the trained model to update the model in the on-premises edge server. Training the new model includes determining an on-premises edge server with computing resources available to train the new model while generating other inference data for incoming stream data in the data analytic pipeline.
    Type: Application
    Filed: June 29, 2021
    Publication date: December 29, 2022
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Paramvir BAHL, Tsuwang HSIEH
  • Publication number: 20220417306
    Abstract: Systems and methods are provided for reducing stream data according to a data streaming protocol under a multi-access edge computing. In particular, an IoT device, such as a video image sensing device, may capture stream data and generate inference data by applying a machine-learning model trained to infer data based on the captured stream data. The inference data represents the captured stream data in a reduced data size based on performing data analytics on the captured data. The IoT device formats the inference data according to the data streaming protocol. In contrast to video data compression, the data streaming protocol includes instructions for transmitting the reduced volume of inference data through a data analytics pipeline.
    Type: Application
    Filed: June 29, 2021
    Publication date: December 29, 2022
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yu YAN, Yuanchao SHU
  • Publication number: 20220400085
    Abstract: Computing resources are managed in a computing environment comprising a computing service provider and an edge computing network. The edge computing network comprises computing and storage devices configured to extend computing resources of the computing service provider to remote users of the computing service provider. The edge computing network collects capacity and usage data for computing and network resources at the edge computing network. The capacity and usage data is sent to the computing service provider. Based on the capacity and usage data, the computing service provider, using a cost function, determines a distribution of workloads pertaining to a processing pipeline that has been partitioned into the workloads. The workloads can be executed at the computing service provider or the edge computing network.
    Type: Application
    Filed: June 15, 2021
    Publication date: December 15, 2022
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Paramvir BAHL
  • Publication number: 20220383188
    Abstract: Systems and methods are provided for merging models for use in an edge server under the multi-access edge computing environment. In particular, a model merger selects a layer of a model based on a level of memory consumption in the edge server and determines sharable layers based on common properties of the selected layer. The model merger generates a merged model by generating a single instantiation of a layer that corresponds to the sharable layers. A model trainer trains the merged model based on training data for the respective models to attain a level of accuracy of data analytics above a predetermined threshold. The disclosed technology further refreshes the merged model upon observing a level of data drift that exceeds a predetermined threshold. The refreshing of the merged model includes detaching and/or splitting consolidated sharable layers of sub-models in the merged model.
    Type: Application
    Filed: September 10, 2021
    Publication date: December 1, 2022
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Anand PADMANABHA IYER, Yuanchao SHU, Nikolaos KARIANAKIS, Arthi Hema PADMANABHAN
  • Patent number: 11461591
    Abstract: Methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service includes a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.
    Type: Grant
    Filed: December 16, 2020
    Date of Patent: October 4, 2022
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ganesh Ananthanarayanan, Yuanchao Shu, Tsu-wang Hsieh, Nikolaos Karianakis, Paramvir Bahl, Romil Bhardwaj
  • Publication number: 20220188569
    Abstract: Examples are disclosed that relate to methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service comprises a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.
    Type: Application
    Filed: December 16, 2020
    Publication date: June 16, 2022
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Tsu-wang HSIEH, Nikolaos KARIANAKIS, Paramvir BAHL, Romil BHARDWAJ
  • Patent number: 11272423
    Abstract: A multi-hop relay network comprises a set of data nodes arranged in a relay topology, wherein the set of data nodes communicate with one another via a data plane comprising a set of high frequency data links, such as mmWave links. An edge node communicates with one or more of the set of data nodes via the data plane and communicates with the set of data nodes over a control plane comprising a set of low frequency wireless links, such as a Wi-Fi network. The edge node determines a path utilization for the set of high frequency wireless links. When one of the high frequency wireless links is over-utilized, the edge node communicates, to a data node in the set of data nodes via the control plane, a command to change the relay topology. The edge node also determines whether the relay topology is operating at a target accuracy. When it is not, the edge node adjusts a data analytics parameter for a node to improve the overall accuracy of the network.
    Type: Grant
    Filed: April 29, 2020
    Date of Patent: March 8, 2022
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ganesh Ananthanarayanan, Yuanchao Shu, Paramvir Bahl, Longfei Shangguan, Zhuqi Li
  • Publication number: 20210345222
    Abstract: A multi-hop relay network comprises a set of data nodes arranged in a relay topology, wherein the set of data nodes communicate with one another via a data plane comprising a set of high frequency data links, such as mmWave links. An edge node communicates with one or more of the set of data nodes via the data plane and communicates with the set of data nodes over a control plane comprising a set of low frequency wireless links, such as a Wi-Fi network. The edge node determines a path utilization for the set of high frequency wireless links. When one of the high frequency wireless links is over-utilized, the edge node communicates, to a data node in the set of data nodes via the control plane, a command to change the relay topology. The edge node also determines whether the relay topology is operating at a target accuracy. When it is not, the edge node adjusts a data analytics parameter for a node to improve the overall accuracy of the network.
    Type: Application
    Filed: April 29, 2020
    Publication date: November 4, 2021
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Paramvir BAHL, Longfei SHANGGUAN, Zhuqi LI
  • Patent number: 11085772
    Abstract: In accordance with implementations of the subject matter described herein, a new approach for generating indoor navigation is proposed. Generally speaking, a reference signal that includes time series data collected by at least one environment sensor along a reference path from a start point to a destination is obtained. For example, the reference signal may be obtained by environment sensors equipped in a user's mobile device or another movable entity. Then, a movement event by identifying a pattern from the reference signal, the pattern describing measurements of the at least one environment sensor associated with a specific movement is extracted. Next, a navigation instruction is generated to indicate that the movement event occurs during a movement of the at least one environment sensor along the reference path. Further, the navigation instruction may be provided to a person for indoor navigation.
    Type: Grant
    Filed: September 7, 2016
    Date of Patent: August 10, 2021
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Yuanchao Shu, Börje Felipe Fernandes Karlsson, Thomas Moscibroda, Yundie Zhang, Zhuqi Li
  • Publication number: 20210190505
    Abstract: In implementations of the subject matter described herein, a solution for providing an indoor location-based service is provided. In this solution, a floor plan about a first floor of a first building comprising at least one floor is obtained. A first map for the first floor is generated based on the floor plan. The first map includes a plurality of vertices and edges, where one vertex corresponds to a position on the first floor, an edge connecting two vertices represents a physical object or a passable path on the first floor, and two positions corresponding to the two vertices are located at the physical object or on the passable path. Moreover, a location-based service is provided to a user at least based on the first map.
    Type: Application
    Filed: June 18, 2019
    Publication date: June 24, 2021
    Inventors: Börje Felipe Fernandes Karlsson, Yuanchao SHU, Ruogu DU, Yi JIANG
  • Publication number: 20200387539
    Abstract: This document relates to performing live video stream analytics on edge devices. One example determines resources available to the system, and a video analytics configuration is selected that distributes work between edge devices and cloud devices in a cascading manner, where edge device processing is prioritized over cloud processing in order to conserve resources. This example can dynamically modify the allocation of processing depending on changing conditions, such as network availability.
    Type: Application
    Filed: June 4, 2019
    Publication date: December 10, 2020
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ganesh ANANTHANARAYANAN, Yuanchao SHU, Shadi NOGHABI, Paramvir BAHL, Landon COX, Alexander CROWN
  • Patent number: 10697778
    Abstract: In accordance with implementations of the subject matter described herein, a new approach for creating a navigation trace is proposed. In these implementations, a reference signal is obtained, where the reference signal includes time series data collected by at least one environment sensor along a reference path from a start point to a destination. Then, a navigation trace is created for the reference path based on the obtained reference signal, where magnitudes at locations of the navigation trace reflect measurements collected by the at least one environment sensor at respective time points of the reference signal. In accordance with implementations of the subject matter described herein, a new approach for generating a navigation instruction from a navigation trace.
    Type: Grant
    Filed: September 7, 2016
    Date of Patent: June 30, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Yuanchao Shu, Börje Felipe Fernandes Karlsson, Thomas Moscibroda, Yundie Zhang, Zhuqi Li
  • Publication number: 20190195635
    Abstract: In accordance with implementations of the subject matter described herein, a new approach for generating indoor navigation is proposed. Generally speaking, a reference signal that includes time series data collected by at least one environment sensor along a reference path from a start point to a destination is obtained. For example, the reference signal may be obtained by environment sensors equipped in a user's mobile device or another movable entity. Then, a movement event by identifying a pattern from the reference signal, the pattern describing measurements of the at least one environment sensor associated with a specific movement is extracted. Next, a navigation instruction is generated to indicate that the movement event occurs during a movement of the at least one environment sensor along the reference path. Further, the navigation instruction may be provided to a person for indoor navigation.
    Type: Application
    Filed: September 7, 2016
    Publication date: June 27, 2019
    Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Yuanchao SHU, Börje Felipe Fernandes KARLSSON, Thomas MOSCIBRODA, Yundie ZHANG, Zhuqi LI