Patents by Inventor Marco PAVONE
Marco PAVONE has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240085914Abstract: In various examples, techniques for determining perception zones for object detection are described. For instance, a system may use a dynamic model associated with an ego-machine, a dynamic model associated with an object, and one or more possible interactions between the ego-machine and the object to determine a perception zone. The system may then perform one or more processes using the perception zone. For instance, if the system is validating a perception system of the ego-machine, the system may determine whether a detection error associated with the object is a safety-critical error based on whether the object is located within the perception zone. Additionally, if the system is executing within the ego-machine, the system may determine whether the object is a safety-critical object based on whether the object is located within the perception zone.Type: ApplicationFiled: September 12, 2022Publication date: March 14, 2024Inventors: Sever Ioan Topan, Karen Yan Ming Leung, Yuxiao Chen, Pritish Tupekar, Edward Fu Schmerling, Hans Jonas Nilsson, Michael Cox, Marco Pavone
-
Patent number: 11887317Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.Type: GrantFiled: August 30, 2021Date of Patent: January 30, 2024Assignees: Ford Global Technologies, LLC, The Board of Trustees of the Leland Stanford Junior UniversityInventors: Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
-
Publication number: 20240028673Abstract: In various examples, robust trajectory predictions against adversarial attacks in autonomous machines and applications are described herein. Systems and methods are disclosed that perform adversarial training for trajectory predictions determined using a neural network(s). In order to improve the training, the systems and methods may devise a deterministic attach that creates a deterministic gradient path within a probabilistic model to generate adversarial samples for training. Additionally, the systems and methods may introduce a hybrid objective that interleaves the adversarial training and learning from clean data to anchor the output from the neural network(s) on stable, clean data distribution. Furthermore, the systems and methods may use a domain-specific data augmentation technique that generates diverse, realistic, and dynamically-feasible samples for additional training of the neural network(s).Type: ApplicationFiled: March 8, 2023Publication date: January 25, 2024Inventors: Chaowei Xiao, Yolong Cao, Danfei Xu, Animashree Anandkumar, Marco Pavone, Xinshuo Weng
-
Publication number: 20240017743Abstract: In various examples, cost probability distributions corresponding to predicted locations of an object in an environment and potential locations for a machine in the environment and may be evaluated using corresponding observed costs corresponding to the machine and the object. The cost probability distributions may be evaluated based on comparing the observed costs to threshold values, which may be determined based on sampling a predicted cost function. A threshold value may be selected to provide false-positive rate and/or false-negative rate guarantees for anomaly detection. Control operations may be performed based on results of the evaluation of the cost probability distributions. For example, based on the results, a motion planner may reuse a planned trajectory for a future planning cycle (e.g., thereby avoiding re-planning computations) or generate and/or select a new planned trajectory (e.g., based at least on one or more anomalies being detected).Type: ApplicationFiled: March 14, 2023Publication date: January 18, 2024Inventors: Alec Farid, Sushant Veer, Boris Ivanovic, Karen Leung, Marco Pavone
-
Publication number: 20240017745Abstract: Apparatuses, systems, and techniques to generate trajectory data for moving objects. In at least one embodiment, adversarial trajectories are generated to evaluate a trajectory prediction model and are based, at least in part, on a differentiable dynamic model.Type: ApplicationFiled: July 14, 2022Publication date: January 18, 2024Inventors: Yulong Cao, Chaowei Xiao, Danfei Xu, Anima Anandkumar, Marco Pavone
-
Publication number: 20240010196Abstract: In various examples, control policies for controlling agents may be learned from demonstrations capturing joint states of entities navigating through the environment. A control policy may be learned mapping joint states to control actions, where the joint states are between agents, and the control actions are of at least one of the agents. The control policy may be learned to define the mappings as control invariant sets of the joint sates and the control actions. The control policy may be used to determine one or more functions that compute, based at least on a joint state between entities, output indicating a likelihood of collision between the entities operating in accordance with the control policy. Using the output, current and/or potential states of the environment may be evaluated to determine control operations for a machine, such as a vehicle.Type: ApplicationFiled: March 14, 2023Publication date: January 11, 2024Inventors: Karen Yan Ming Leung, Sushant Veer, Edward Fu Schmerling, Marco Pavone
-
Publication number: 20240010232Abstract: In various examples, a motion planner include an analytical function to predict motion plans for a machine based on predicted trajectories of actors in an environment, where the predictions are differentiable with respect to parameters of a neural network of a motion predictor used to predict the trajectories. The analytical function may be used to determine candidate trajectories for the machine based on a predicted trajectory, to compute cost values for the candidate trajectories, and to select a reference trajectory from the candidate trajectories. For differentiability, a term of the analytical function may correspond to the predicted trajectory. A motion controller may use the reference trajectory to predict a control sequence for the machine using an analytical function trained to generate predictions that are differentiable with respect to at least one parameter of the analytical function used to compute the cost values.Type: ApplicationFiled: May 16, 2023Publication date: January 11, 2024Inventors: Peter Karkus, Boris Ivanovic, Shie Mannor, Marco Pavone
-
Publication number: 20230394823Abstract: Apparatuses, systems, and techniques to perform trajectory predictions within one or more images. In at least one embodiment, a processor comprises one or more circuits to cause one or more neural networks to perform trajectory predictions of two or more objects detected within a plurality of frames without tracking the two or more objects based, at least in part, on processing a sequence of data of the one or more objects as a whole.Type: ApplicationFiled: March 8, 2023Publication date: December 7, 2023Inventors: Xinshuo Weng, Boris Ivanovic, Marco Pavone
-
Publication number: 20230391374Abstract: Apparatuses, systems, and techniques to generate trajectory predictions. In at least one embodiment, trajectory predictions are generated based on, for example, one or more neural networks.Type: ApplicationFiled: February 2, 2023Publication date: December 7, 2023Inventors: Yuxiao CHEN, Boris IVANOVIC, Marco PAVONE
-
Publication number: 20230391365Abstract: In various examples, techniques for generating simulations for autonomous machines and applications are described herein. Systems and methods are disclosed that use various models to generate simulations. For instance, a first model(s) may process input data, such as input data representing maps indicating the locations of objects and state history of the objects within the environment, to determine navigation goals for the objects. Additionally, a second model(s) may then process the input data and data representing the navigation goals in order to determine possible trajectories (e.g., action samples) for the objects within the environment. Furthermore, a third model(s) may process the input data to predict trajectories of the objects within the environment. The systems and methods may then use at least the possible trajectories and the predicted trajectories to simulate the motion (e.g., one or more trajectories) of one or more of the objects.Type: ApplicationFiled: February 24, 2023Publication date: December 7, 2023Inventors: Boris Ivanovic, Danfei Xu, Yuxiao Chen, Marco Pavone
-
Patent number: 11829275Abstract: Systems and methods for generating and evaluating driving scenarios with varying difficulty levels is provided. The disclosed systems and methods may be used to develop a suite of regression tests that track the progress of an autonomous driving stack. A robustness trace of a temporal logic formula may be computed from an always-eventually fragment using a computation graph. The robustness trace may be approximated by a smoothly differentiable computation graph, which can be implemented in existing machine learning programming frameworks. The systems and methods provided herein may be useful in automatic test case generation for autonomous or semi-autonomous vehicles.Type: GrantFiled: September 13, 2019Date of Patent: November 28, 2023Assignees: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Nikos Arechiga Gonzalez, Karen Y. Leung, Marco Pavone
-
Patent number: 11807267Abstract: Systems, methods, and other embodiments described herein relate to improving controls in a device according to risk. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying objects from the sensor data that are present in the surrounding environment. The method includes generating a control sequence for controlling the device according to a risk-sensitivity parameter to navigate toward a destination while considering risk associated with encountering the objects defined by the risk-sensitivity parameter. The method includes controlling the device according to the control sequence.Type: GrantFiled: December 31, 2020Date of Patent: November 7, 2023Assignees: Toyota Research Institute, Inc., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Haruki Nishimura, Boris Ivanovic, Adrien David Gaidon, Marco Pavone, Mac Schwager
-
Publication number: 20230308432Abstract: A method for authenticating and authorizing a user identifier to access a service is disclosed. The service is activated by a service application programming interface which requires a two-factor authorization to initiate execution of the service for the requesting user identifier. The method receives, by the service API, a service request together with first and second data for a first and a second authentication method, confirming a correctness of the first data as a first identity pass key using the first authentication method, confirming a correctness of the second data as a second identity pass key using the second authentication method and the first identity pass key is input to the second authentication method, and the second authentication method differs from the first authentication method. Having received the confirmed correctness of both the first identity pass key and the second identity pass key, executing the service.Type: ApplicationFiled: March 23, 2022Publication date: September 28, 2023Inventors: Martin Smolny, Marco Pavone
-
Patent number: 11689375Abstract: Certificate and key management is provided. A signed certificate corresponding to an enterprise is deployed to a plurality of cryptographic communication protocol endpoint proxies located in a heterogeneous distributed computing environment where a private key corresponding to the enterprise is not placed in any of the plurality of cryptographic communication protocol endpoint proxies. Offload of cryptographic communications from the plurality of cryptographic communication protocol endpoint proxies to the hardware security module is received by the hardware security module where the hardware security module verifies connection authenticity for the plurality of cryptographic communication protocol endpoint proxies across the heterogeneous distributed computing environment using the private key corresponding to the enterprise that remains within a security boundary of the hardware security module.Type: GrantFiled: May 21, 2021Date of Patent: June 27, 2023Assignee: International Business Machines CorporationInventors: Nataraj Nagaratnam, Christopher S. Smith, David Nguyen, Martin Schmatz, Marco Pavone, Navaneeth Rameshan
-
Publication number: 20230081068Abstract: An approach is provided for distributing a root key to a hardware security module (HSM) of an HSM cluster. A signed first command is transmitted to a source HSM to create a master key. A fingerprint of the master key is received in a response signed by the source HSM using a module signing key hardcoded into the source HSM at manufacturing time. A second command is transmitted to a first HSM to generate an importer key pair. A request is transmitted to the source HSM to create and export a wrapped master key. The master key wrapped with a transport key is received. The wrapped master key is transmitted to the first HSM. The master key is activated in the first HSM.Type: ApplicationFiled: September 10, 2021Publication date: March 16, 2023Inventors: DAVID NGUYEN, Marco Pavone, Clifford Lee Hansen, Garry Joseph Sullivan, Ross Martin Heninger
-
Publication number: 20230080445Abstract: A system, method, and computer program product for implementing encryption key management is provided. The method includes connecting a hardware device to a keystore agent comprising a system configured to manage one or more keystores holding one or more cryptographic key instances. A key template is configured to define an attribute for generating cryptographic keys. The key template is modified such that the keystore component is added to the key template and instances of associated cryptographic keys are generated. Each instance is installed within the keystore component and associated attributes associated with data for consumption are generated. A key event log defining all events associated with a given key of the associated cryptographic keys with respect to a lifetime of the given key is generated and a repository comprising key templates and associated key data is maintained.Type: ApplicationFiled: September 10, 2021Publication date: March 16, 2023Inventors: ISABEL ARNOLD, Søren Peen, Troels Nørgaard, Jakub Karol Jelonek, Blazej Pawlak, Christopher S. Smith, Nataraj Nagaratnam, Marco Pavone, Leo Moesgaard
-
Publication number: 20230074293Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.Type: ApplicationFiled: August 30, 2021Publication date: March 9, 2023Applicants: Ford Global Technologies, LLC, The Board of Trustees of the Leland Stanford Junior UniversityInventors: Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
-
Publication number: 20220376929Abstract: Certificate and key management is provided. A signed certificate corresponding to an enterprise is deployed to a plurality of cryptographic communication protocol endpoint proxies located in a heterogeneous distributed computing environment where a private key corresponding to the enterprise is not placed in any of the plurality of cryptographic communication protocol endpoint proxies. Offload of cryptographic communications from the plurality of cryptographic communication protocol endpoint proxies to the hardware security module is received by the hardware security module where the hardware security module verifies connection authenticity for the plurality of cryptographic communication protocol endpoint proxies across the heterogeneous distributed computing environment using the private key corresponding to the enterprise that remains within a security boundary of the hardware security module.Type: ApplicationFiled: May 21, 2021Publication date: November 24, 2022Inventors: Nataraj Nagaratnam, Christopher S. Smith, David Nguyen, Martin Schmatz, Marco Pavone, Navaneeth Rameshan
-
Publication number: 20220204030Abstract: System, methods, and other embodiments described herein relate to improving controls in a device according to risk. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying objects from the sensor data that are present in the surrounding environment. The method includes generating a control sequence for controlling the device according to a risk-sensitivity parameter to navigate toward a destination while considering risk associated with encountering the objects defined by the risk-sensitivity parameter. The method includes controlling the device according to the control sequence.Type: ApplicationFiled: December 31, 2020Publication date: June 30, 2022Inventors: Haruki Nishimura, Boris Ivanovic, Adrien David Gaidon, Marco Pavone, Mac Schwager
-
Publication number: 20220108015Abstract: A method includes: federating, by a computer device, a proxy hardware security module from a physical hardware security module; storing, by the computer device, the proxy hardware security module; receiving, by the computer device, a first one of a plurality of periodic identifying communications from the physical hardware security module; and erasing, by the computer device, the proxy hardware security module as a result of the computer device not receiving a second one of the plurality of periodic identifying communications.Type: ApplicationFiled: December 16, 2021Publication date: April 7, 2022Inventors: Erlander LO, Karunakar BOJJIREDDY, Angel NUNEZ MENCIAS, Marco PAVONE