Patents by Inventor Marco PAVONE

Marco PAVONE has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240085914
    Abstract: In various examples, techniques for determining perception zones for object detection are described. For instance, a system may use a dynamic model associated with an ego-machine, a dynamic model associated with an object, and one or more possible interactions between the ego-machine and the object to determine a perception zone. The system may then perform one or more processes using the perception zone. For instance, if the system is validating a perception system of the ego-machine, the system may determine whether a detection error associated with the object is a safety-critical error based on whether the object is located within the perception zone. Additionally, if the system is executing within the ego-machine, the system may determine whether the object is a safety-critical object based on whether the object is located within the perception zone.
    Type: Application
    Filed: September 12, 2022
    Publication date: March 14, 2024
    Inventors: Sever Ioan Topan, Karen Yan Ming Leung, Yuxiao Chen, Pritish Tupekar, Edward Fu Schmerling, Hans Jonas Nilsson, Michael Cox, Marco Pavone
  • Patent number: 11887317
    Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.
    Type: Grant
    Filed: August 30, 2021
    Date of Patent: January 30, 2024
    Assignees: Ford Global Technologies, LLC, The Board of Trustees of the Leland Stanford Junior University
    Inventors: Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
  • Publication number: 20240028673
    Abstract: In various examples, robust trajectory predictions against adversarial attacks in autonomous machines and applications are described herein. Systems and methods are disclosed that perform adversarial training for trajectory predictions determined using a neural network(s). In order to improve the training, the systems and methods may devise a deterministic attach that creates a deterministic gradient path within a probabilistic model to generate adversarial samples for training. Additionally, the systems and methods may introduce a hybrid objective that interleaves the adversarial training and learning from clean data to anchor the output from the neural network(s) on stable, clean data distribution. Furthermore, the systems and methods may use a domain-specific data augmentation technique that generates diverse, realistic, and dynamically-feasible samples for additional training of the neural network(s).
    Type: Application
    Filed: March 8, 2023
    Publication date: January 25, 2024
    Inventors: Chaowei Xiao, Yolong Cao, Danfei Xu, Animashree Anandkumar, Marco Pavone, Xinshuo Weng
  • Publication number: 20240017743
    Abstract: In various examples, cost probability distributions corresponding to predicted locations of an object in an environment and potential locations for a machine in the environment and may be evaluated using corresponding observed costs corresponding to the machine and the object. The cost probability distributions may be evaluated based on comparing the observed costs to threshold values, which may be determined based on sampling a predicted cost function. A threshold value may be selected to provide false-positive rate and/or false-negative rate guarantees for anomaly detection. Control operations may be performed based on results of the evaluation of the cost probability distributions. For example, based on the results, a motion planner may reuse a planned trajectory for a future planning cycle (e.g., thereby avoiding re-planning computations) or generate and/or select a new planned trajectory (e.g., based at least on one or more anomalies being detected).
    Type: Application
    Filed: March 14, 2023
    Publication date: January 18, 2024
    Inventors: Alec Farid, Sushant Veer, Boris Ivanovic, Karen Leung, Marco Pavone
  • Publication number: 20240017745
    Abstract: Apparatuses, systems, and techniques to generate trajectory data for moving objects. In at least one embodiment, adversarial trajectories are generated to evaluate a trajectory prediction model and are based, at least in part, on a differentiable dynamic model.
    Type: Application
    Filed: July 14, 2022
    Publication date: January 18, 2024
    Inventors: Yulong Cao, Chaowei Xiao, Danfei Xu, Anima Anandkumar, Marco Pavone
  • Publication number: 20240010196
    Abstract: In various examples, control policies for controlling agents may be learned from demonstrations capturing joint states of entities navigating through the environment. A control policy may be learned mapping joint states to control actions, where the joint states are between agents, and the control actions are of at least one of the agents. The control policy may be learned to define the mappings as control invariant sets of the joint sates and the control actions. The control policy may be used to determine one or more functions that compute, based at least on a joint state between entities, output indicating a likelihood of collision between the entities operating in accordance with the control policy. Using the output, current and/or potential states of the environment may be evaluated to determine control operations for a machine, such as a vehicle.
    Type: Application
    Filed: March 14, 2023
    Publication date: January 11, 2024
    Inventors: Karen Yan Ming Leung, Sushant Veer, Edward Fu Schmerling, Marco Pavone
  • Publication number: 20240010232
    Abstract: In various examples, a motion planner include an analytical function to predict motion plans for a machine based on predicted trajectories of actors in an environment, where the predictions are differentiable with respect to parameters of a neural network of a motion predictor used to predict the trajectories. The analytical function may be used to determine candidate trajectories for the machine based on a predicted trajectory, to compute cost values for the candidate trajectories, and to select a reference trajectory from the candidate trajectories. For differentiability, a term of the analytical function may correspond to the predicted trajectory. A motion controller may use the reference trajectory to predict a control sequence for the machine using an analytical function trained to generate predictions that are differentiable with respect to at least one parameter of the analytical function used to compute the cost values.
    Type: Application
    Filed: May 16, 2023
    Publication date: January 11, 2024
    Inventors: Peter Karkus, Boris Ivanovic, Shie Mannor, Marco Pavone
  • Publication number: 20230394823
    Abstract: Apparatuses, systems, and techniques to perform trajectory predictions within one or more images. In at least one embodiment, a processor comprises one or more circuits to cause one or more neural networks to perform trajectory predictions of two or more objects detected within a plurality of frames without tracking the two or more objects based, at least in part, on processing a sequence of data of the one or more objects as a whole.
    Type: Application
    Filed: March 8, 2023
    Publication date: December 7, 2023
    Inventors: Xinshuo Weng, Boris Ivanovic, Marco Pavone
  • Publication number: 20230391374
    Abstract: Apparatuses, systems, and techniques to generate trajectory predictions. In at least one embodiment, trajectory predictions are generated based on, for example, one or more neural networks.
    Type: Application
    Filed: February 2, 2023
    Publication date: December 7, 2023
    Inventors: Yuxiao CHEN, Boris IVANOVIC, Marco PAVONE
  • Publication number: 20230391365
    Abstract: In various examples, techniques for generating simulations for autonomous machines and applications are described herein. Systems and methods are disclosed that use various models to generate simulations. For instance, a first model(s) may process input data, such as input data representing maps indicating the locations of objects and state history of the objects within the environment, to determine navigation goals for the objects. Additionally, a second model(s) may then process the input data and data representing the navigation goals in order to determine possible trajectories (e.g., action samples) for the objects within the environment. Furthermore, a third model(s) may process the input data to predict trajectories of the objects within the environment. The systems and methods may then use at least the possible trajectories and the predicted trajectories to simulate the motion (e.g., one or more trajectories) of one or more of the objects.
    Type: Application
    Filed: February 24, 2023
    Publication date: December 7, 2023
    Inventors: Boris Ivanovic, Danfei Xu, Yuxiao Chen, Marco Pavone
  • Patent number: 11829275
    Abstract: Systems and methods for generating and evaluating driving scenarios with varying difficulty levels is provided. The disclosed systems and methods may be used to develop a suite of regression tests that track the progress of an autonomous driving stack. A robustness trace of a temporal logic formula may be computed from an always-eventually fragment using a computation graph. The robustness trace may be approximated by a smoothly differentiable computation graph, which can be implemented in existing machine learning programming frameworks. The systems and methods provided herein may be useful in automatic test case generation for autonomous or semi-autonomous vehicles.
    Type: Grant
    Filed: September 13, 2019
    Date of Patent: November 28, 2023
    Assignees: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
    Inventors: Nikos Arechiga Gonzalez, Karen Y. Leung, Marco Pavone
  • Patent number: 11807267
    Abstract: Systems, methods, and other embodiments described herein relate to improving controls in a device according to risk. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying objects from the sensor data that are present in the surrounding environment. The method includes generating a control sequence for controlling the device according to a risk-sensitivity parameter to navigate toward a destination while considering risk associated with encountering the objects defined by the risk-sensitivity parameter. The method includes controlling the device according to the control sequence.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: November 7, 2023
    Assignees: Toyota Research Institute, Inc., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
    Inventors: Haruki Nishimura, Boris Ivanovic, Adrien David Gaidon, Marco Pavone, Mac Schwager
  • Publication number: 20230308432
    Abstract: A method for authenticating and authorizing a user identifier to access a service is disclosed. The service is activated by a service application programming interface which requires a two-factor authorization to initiate execution of the service for the requesting user identifier. The method receives, by the service API, a service request together with first and second data for a first and a second authentication method, confirming a correctness of the first data as a first identity pass key using the first authentication method, confirming a correctness of the second data as a second identity pass key using the second authentication method and the first identity pass key is input to the second authentication method, and the second authentication method differs from the first authentication method. Having received the confirmed correctness of both the first identity pass key and the second identity pass key, executing the service.
    Type: Application
    Filed: March 23, 2022
    Publication date: September 28, 2023
    Inventors: Martin Smolny, Marco Pavone
  • Patent number: 11689375
    Abstract: Certificate and key management is provided. A signed certificate corresponding to an enterprise is deployed to a plurality of cryptographic communication protocol endpoint proxies located in a heterogeneous distributed computing environment where a private key corresponding to the enterprise is not placed in any of the plurality of cryptographic communication protocol endpoint proxies. Offload of cryptographic communications from the plurality of cryptographic communication protocol endpoint proxies to the hardware security module is received by the hardware security module where the hardware security module verifies connection authenticity for the plurality of cryptographic communication protocol endpoint proxies across the heterogeneous distributed computing environment using the private key corresponding to the enterprise that remains within a security boundary of the hardware security module.
    Type: Grant
    Filed: May 21, 2021
    Date of Patent: June 27, 2023
    Assignee: International Business Machines Corporation
    Inventors: Nataraj Nagaratnam, Christopher S. Smith, David Nguyen, Martin Schmatz, Marco Pavone, Navaneeth Rameshan
  • Publication number: 20230081068
    Abstract: An approach is provided for distributing a root key to a hardware security module (HSM) of an HSM cluster. A signed first command is transmitted to a source HSM to create a master key. A fingerprint of the master key is received in a response signed by the source HSM using a module signing key hardcoded into the source HSM at manufacturing time. A second command is transmitted to a first HSM to generate an importer key pair. A request is transmitted to the source HSM to create and export a wrapped master key. The master key wrapped with a transport key is received. The wrapped master key is transmitted to the first HSM. The master key is activated in the first HSM.
    Type: Application
    Filed: September 10, 2021
    Publication date: March 16, 2023
    Inventors: DAVID NGUYEN, Marco Pavone, Clifford Lee Hansen, Garry Joseph Sullivan, Ross Martin Heninger
  • Publication number: 20230080445
    Abstract: A system, method, and computer program product for implementing encryption key management is provided. The method includes connecting a hardware device to a keystore agent comprising a system configured to manage one or more keystores holding one or more cryptographic key instances. A key template is configured to define an attribute for generating cryptographic keys. The key template is modified such that the keystore component is added to the key template and instances of associated cryptographic keys are generated. Each instance is installed within the keystore component and associated attributes associated with data for consumption are generated. A key event log defining all events associated with a given key of the associated cryptographic keys with respect to a lifetime of the given key is generated and a repository comprising key templates and associated key data is maintained.
    Type: Application
    Filed: September 10, 2021
    Publication date: March 16, 2023
    Inventors: ISABEL ARNOLD, Søren Peen, Troels Nørgaard, Jakub Karol Jelonek, Blazej Pawlak, Christopher S. Smith, Nataraj Nagaratnam, Marco Pavone, Leo Moesgaard
  • Publication number: 20230074293
    Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.
    Type: Application
    Filed: August 30, 2021
    Publication date: March 9, 2023
    Applicants: Ford Global Technologies, LLC, The Board of Trustees of the Leland Stanford Junior University
    Inventors: Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
  • Publication number: 20220376929
    Abstract: Certificate and key management is provided. A signed certificate corresponding to an enterprise is deployed to a plurality of cryptographic communication protocol endpoint proxies located in a heterogeneous distributed computing environment where a private key corresponding to the enterprise is not placed in any of the plurality of cryptographic communication protocol endpoint proxies. Offload of cryptographic communications from the plurality of cryptographic communication protocol endpoint proxies to the hardware security module is received by the hardware security module where the hardware security module verifies connection authenticity for the plurality of cryptographic communication protocol endpoint proxies across the heterogeneous distributed computing environment using the private key corresponding to the enterprise that remains within a security boundary of the hardware security module.
    Type: Application
    Filed: May 21, 2021
    Publication date: November 24, 2022
    Inventors: Nataraj Nagaratnam, Christopher S. Smith, David Nguyen, Martin Schmatz, Marco Pavone, Navaneeth Rameshan
  • Publication number: 20220204030
    Abstract: System, methods, and other embodiments described herein relate to improving controls in a device according to risk. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying objects from the sensor data that are present in the surrounding environment. The method includes generating a control sequence for controlling the device according to a risk-sensitivity parameter to navigate toward a destination while considering risk associated with encountering the objects defined by the risk-sensitivity parameter. The method includes controlling the device according to the control sequence.
    Type: Application
    Filed: December 31, 2020
    Publication date: June 30, 2022
    Inventors: Haruki Nishimura, Boris Ivanovic, Adrien David Gaidon, Marco Pavone, Mac Schwager
  • Publication number: 20220108015
    Abstract: A method includes: federating, by a computer device, a proxy hardware security module from a physical hardware security module; storing, by the computer device, the proxy hardware security module; receiving, by the computer device, a first one of a plurality of periodic identifying communications from the physical hardware security module; and erasing, by the computer device, the proxy hardware security module as a result of the computer device not receiving a second one of the plurality of periodic identifying communications.
    Type: Application
    Filed: December 16, 2021
    Publication date: April 7, 2022
    Inventors: Erlander LO, Karunakar BOJJIREDDY, Angel NUNEZ MENCIAS, Marco PAVONE