Patents by Inventor Marco PAVONE
Marco PAVONE has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250029489Abstract: In various examples, a traffic model including one or more traffic scenarios may be generated and/or updated based on using human feedback. Human feedback may be provided indicating a preference for various traffic scenarios to identify which scenarios in a model are more realistic. A reward model may capture the preference information and rank the realism of one or more traffic scenarios.Type: ApplicationFiled: October 12, 2023Publication date: January 23, 2025Inventors: Yulong Cao, Chaowei Xiao, Marco Pavone, Boris Ivanovic
-
Publication number: 20250005447Abstract: One embodiment of a method for processing data includes performing one or more operations to determine a performance of one or more predefined rules based on data that is received and one or more first predictions generated using the one or more predefined rules, performing one or more operations to determine a performance of a trained machine learning model based on the data and one or more second predictions generated using the trained machine learning model, processing the data using the one or more predefined rules to generate one or more third predictions, processing the data using the trained machine learning model to generate one or more fourth predictions, and generating one or more fifth predictions based on the one or more third predictions, the one or more fourth predictions, the performance of the one or more predefined rules, and the performance of the trained machine learning model.Type: ApplicationFiled: April 11, 2024Publication date: January 2, 2025Inventors: Sushant VEER, Apoorva SHARMA, Marco PAVONE
-
Publication number: 20240419979Abstract: One embodiment of a method for controlling a system includes generating a plurality of initializations using a trained machine learning model, performing a plurality of instances of an iterative technique based on the plurality of initializations to generate a plurality of results, generating a control signal based on one or more results included in the plurality of results, and transmitting the control signal to the system to cause the system to perform one or more operations.Type: ApplicationFiled: January 18, 2024Publication date: December 19, 2024Inventors: Peter KARKUS, Tong CHE, Christopher MAES, Shie MANNOR, Marco PAVONE, Yunfei SHI, Heng YANG
-
Publication number: 20240400101Abstract: In various examples, systems and methods are disclosed relating to refinement of safety zones and improving evaluation metrics for the perception modules of autonomous and semi-autonomous systems. Example implementations can exclude areas in the state space that are not safety critical, while retaining the areas that are safety-critical. This can be accomplished by leveraging ego maneuver information and conditioning safety zone computations on ego maneuvers. A maneuver-based decomposition of perception safety zones may leverage a temporal convolution operation with the capability to account for collision at any intermediate time along the way to maneuver completion. This provides a significant reduction in zone volume while maintaining completeness, thus optimizing or otherwise enhancing obstacle perception performance requirements by filtering out regions of state space not relevant to a system's route of travel.Type: ApplicationFiled: June 2, 2023Publication date: December 5, 2024Applicant: NVIDIA CorporationInventors: Sever Ioan TOPAN, Yuxiao CHEN, Edward FU SCHMERLING, Karen Yan Ming LEUNG, Hans Jonas NILSSON, Michael COX, Marco PAVONE
-
Patent number: 12124580Abstract: A method includes: federating, by a computer device, a proxy hardware security module from a physical hardware security module; storing, by the computer device, the proxy hardware security module; receiving, by the computer device, a first one of a plurality of periodic identifying communications from the physical hardware security module; and erasing, by the computer device, the proxy hardware security module as a result of the computer device not receiving a second one of the plurality of periodic identifying communications.Type: GrantFiled: December 16, 2021Date of Patent: October 22, 2024Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATIONInventors: Erlander Lo, Karunakar Bojjireddy, Angel Nunez Mencias, Marco Pavone
-
Patent number: 12095917Abstract: An approach is provided for distributing a root key to a hardware security module (HSM) of an HSM cluster. A signed first command is transmitted to a source HSM to create a master key. A fingerprint of the master key is received in a response signed by the source HSM using a module signing key hardcoded into the source HSM at manufacturing time. A second command is transmitted to a first HSM to generate an importer key pair. A request is transmitted to the source HSM to create and export a wrapped master key. The master key wrapped with a transport key is received. The wrapped master key is transmitted to the first HSM. The master key is activated in the first HSM.Type: GrantFiled: September 10, 2021Date of Patent: September 17, 2024Assignee: International Business Machines CorporationInventors: David Nguyen, Marco Pavone, Clifford Lee Hansen, Garry Joseph Sullivan, Ross Martin Heninger
-
Publication number: 20240273802Abstract: In various examples, frequency regularization and/or occlusion regularization techniques may be used to train Neural Radiance Fields (NeRF) to determine neural renderings based at least on sparse inputs in a way that reduces overfitting, underfitting, and/or occlusions. For example, while training a NeRF, a linearly increased frequency mask may be applied to regularize a visible frequency spectrum of training data based on training time steps. In examples, as training of the NeRF progresses, the visible frequency may be increased in a way that reduces the risk of overfitting and/or avoids underfitting. Additionally, the disclosed techniques may also include masking one or more density scores located within a threshold proximity of an origin of a ray to reduce floaters, walls, and other occlusions in the neural rendering output.Type: ApplicationFiled: January 25, 2024Publication date: August 15, 2024Inventors: Yue Wang, Marco Pavone, Jiawei Yang
-
Patent number: 12061480Abstract: A mobile robot can be caused to move according to a planned trajectory. The mobile robot can be a vehicle. Information about agents in an environment of the mobile robot can be received from sensors. At a first time, a spatiotemporal graph can be produced. The spatiotemporal graph can represent relationships among the agents in the environment. The mobile robot can be one of the agents in the environment. Information from the spatiotemporal graph can be input to neural networks to produce information for a mixture of affine time-varying systems. The mixture of affine time-varying systems can represent an evolution of agent states of the agents. Using the mixture of affine time-varying systems and information associated with the first time, a prediction of the agent states at a second time can be calculated. The mobile robot can be caused to move according to the planned trajectory determined from the prediction.Type: GrantFiled: April 12, 2021Date of Patent: August 13, 2024Assignees: Toyota Research Institute, Inc., The Board of Trustees of the Leland Stanford Junior UniversityInventors: Boris Ivanovic, Amine Elhafsi, Guy Rosman, Adrien David Gaidon, Marco Pavone
-
Patent number: 12041164Abstract: A system, method, and computer program product for implementing encryption key management is provided. The method includes connecting a hardware device to a keystore agent comprising a system configured to manage one or more keystores holding one or more cryptographic key instances. A key template is configured to define an attribute for generating cryptographic keys. The key template is modified such that the keystore component is added to the key template and instances of associated cryptographic keys are generated. Each instance is installed within the keystore component and associated attributes associated with data for consumption are generated. A key event log defining all events associated with a given key of the associated cryptographic keys with respect to a lifetime of the given key is generated and a repository comprising key templates and associated key data is maintained.Type: GrantFiled: September 10, 2021Date of Patent: July 16, 2024Assignee: International Business Machines CorporationInventors: Isabel Arnold, Søren Peen, Troels Nørgaard, Jakub Karol Jelonek, Blazej Pawlak, Christopher S. Smith, Nataraj Nagaratnam, Marco Pavone, Leo Moesgaard
-
Publication number: 20240199068Abstract: Apparatuses, systems, and techniques to obtain prediction set(s) (e.g., region(s)) for keypoint prediction(s) based at least in part on data associated with an object, compute a set of candidate poses for the object based at least in part on the prediction set(s), and estimate an estimated object pose based at least in part on the set of candidate poses. The estimated object pose may be used to move a device. For example the estimated object pose may be used to provide collision-free motion generation for a real-world or virtual device (e.g., a robot, an autonomous machine, or a semi-autonomous machine). In at least one embodiment, at least a portion of the object pose estimation and/or at least a portion of the collision-free motion generation is performed in parallel.Type: ApplicationFiled: September 8, 2023Publication date: June 20, 2024Inventors: Marco Pavone, Heng Yang
-
Publication number: 20240199074Abstract: Autonomous vehicles (AVs) may need to contend with conflicting traveling rules and the AV controller would need to select the least objectionable control action. A rank-preserving reward function can be applied to trajectories derived from a rule hierarchy. The reward function can be correlated to a robustness vector derived for each trajectory. Thereby the highest ranked rules would result in the highest reward, and the lower ranked rules would result in lower reward. In some aspects, one or more optimizers, such as a stochastic optimizer can be utilized to improve the results of the reward calculation. In some aspects, a sigmoid function can be applied to the calculation to smooth out the step function used to calculate the robustness vector. The preferred trajectory selected using the results from the reward function can be communicated to an AV controller for implementation as a control action.Type: ApplicationFiled: June 14, 2023Publication date: June 20, 2024Inventors: Sushant Veer, Karen Leung, Ryan Cosner, Yuxiao Chen, Marco Pavone
-
Publication number: 20240193848Abstract: Apparatuses, systems, and techniques to use one or more machine learning processes to obtain a set of feature values based at least in part on a set of locations along a ray that intersects an object. A color value is obtained based at least in part on the set of feature values. A view of the object may be generated using the color value. A path of motion may be determined based at least in part on the color value and used to cause a device to move.Type: ApplicationFiled: November 21, 2023Publication date: June 13, 2024Inventors: Yue Wang, Marco Pavone
-
Publication number: 20240182082Abstract: In various examples, policy planning using behavior models for autonomous and semi-autonomous systems and applications is described herein. Systems and methods are disclosed that determine a policy for navigating a vehicle, such as a semi-autonomous vehicle or an autonomous vehicle (or other machine), where the policy allows for multistage reasoning that leverages future reactive behaviors of one or more other objects. For instance, a first behavior model (e.g., a trajectory tree) may be generated that represents candidate trajectories for the vehicle and one or more second behavior models (e.g., one or more scenario trees) may be generated that respectively represent future behaviors of the other object(s). The first behavior model and the second behavior model(s) may then be processed, such as in a closed-loop simulation based on a realistic data-driven traffic model, to determine the policy for navigating the vehicle.Type: ApplicationFiled: July 19, 2023Publication date: June 6, 2024Inventors: Yuxiao Chen, Peter Karkus, Boris Ivanovic, Xinshuo Weng, Marco Pavone
-
Publication number: 20240160913Abstract: In various examples, learning responsibility allocations for machine interactions is described herein. Systems and methods are disclosed that train a neural network(s) to generate outputs indicating estimated levels of responsibilities associated with interactions between vehicles or machines and other objects (e.g., other vehicles, machines, pedestrians, animals, etc.). In some examples, the neural network(s) is trained using real-world data, such as data representing scenes depicting actual interactions between vehicles and objects and/or parameters (e.g., velocities, positions, directions, etc.) associated with the interactions. Then, in practice, a vehicle (e.g., an autonomous vehicle, a semi-autonomous vehicle, etc.) may use the neural network(s) to generate an output indicating a proposed or estimated level of responsibility associated with an interaction between the vehicle and an object. The vehicle may then use the output to determine one or more controls for the vehicle to use when navigating.Type: ApplicationFiled: October 31, 2022Publication date: May 16, 2024Inventors: Ryan Cosner, Yuxiao Chen, Karen Yan Ming Leung, Marco Pavone
-
Publication number: 20240085914Abstract: In various examples, techniques for determining perception zones for object detection are described. For instance, a system may use a dynamic model associated with an ego-machine, a dynamic model associated with an object, and one or more possible interactions between the ego-machine and the object to determine a perception zone. The system may then perform one or more processes using the perception zone. For instance, if the system is validating a perception system of the ego-machine, the system may determine whether a detection error associated with the object is a safety-critical error based on whether the object is located within the perception zone. Additionally, if the system is executing within the ego-machine, the system may determine whether the object is a safety-critical object based on whether the object is located within the perception zone.Type: ApplicationFiled: September 12, 2022Publication date: March 14, 2024Inventors: Sever Ioan Topan, Karen Yan Ming Leung, Yuxiao Chen, Pritish Tupekar, Edward Fu Schmerling, Hans Jonas Nilsson, Michael Cox, Marco Pavone
-
Patent number: 11887317Abstract: A plurality of agent locations can be determined at a plurality of time steps by inputting a plurality of images to a perception algorithm that inputs the plurality of images and outputs agent labels and the agent locations. A plurality of first uncertainties corresponding to the agent locations can be determined at the plurality of time steps by inputting the plurality of agent locations to a filter algorithm that inputs the agent locations and outputs the plurality of first uncertainties corresponding to the plurality of agent locations. A plurality of predicted agent trajectories and potential trajectories corresponding to the predicted agent trajectories can be determined by inputting the plurality of agent locations at the plurality of time steps and the first uncertainties corresponding to the agent locations at the plurality of time steps to a variational autoencoder.Type: GrantFiled: August 30, 2021Date of Patent: January 30, 2024Assignees: Ford Global Technologies, LLC, The Board of Trustees of the Leland Stanford Junior UniversityInventors: Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
-
Publication number: 20240028673Abstract: In various examples, robust trajectory predictions against adversarial attacks in autonomous machines and applications are described herein. Systems and methods are disclosed that perform adversarial training for trajectory predictions determined using a neural network(s). In order to improve the training, the systems and methods may devise a deterministic attach that creates a deterministic gradient path within a probabilistic model to generate adversarial samples for training. Additionally, the systems and methods may introduce a hybrid objective that interleaves the adversarial training and learning from clean data to anchor the output from the neural network(s) on stable, clean data distribution. Furthermore, the systems and methods may use a domain-specific data augmentation technique that generates diverse, realistic, and dynamically-feasible samples for additional training of the neural network(s).Type: ApplicationFiled: March 8, 2023Publication date: January 25, 2024Inventors: Chaowei Xiao, Yolong Cao, Danfei Xu, Animashree Anandkumar, Marco Pavone, Xinshuo Weng
-
Publication number: 20240017743Abstract: In various examples, cost probability distributions corresponding to predicted locations of an object in an environment and potential locations for a machine in the environment and may be evaluated using corresponding observed costs corresponding to the machine and the object. The cost probability distributions may be evaluated based on comparing the observed costs to threshold values, which may be determined based on sampling a predicted cost function. A threshold value may be selected to provide false-positive rate and/or false-negative rate guarantees for anomaly detection. Control operations may be performed based on results of the evaluation of the cost probability distributions. For example, based on the results, a motion planner may reuse a planned trajectory for a future planning cycle (e.g., thereby avoiding re-planning computations) or generate and/or select a new planned trajectory (e.g., based at least on one or more anomalies being detected).Type: ApplicationFiled: March 14, 2023Publication date: January 18, 2024Inventors: Alec Farid, Sushant Veer, Boris Ivanovic, Karen Leung, Marco Pavone
-
Publication number: 20240017745Abstract: Apparatuses, systems, and techniques to generate trajectory data for moving objects. In at least one embodiment, adversarial trajectories are generated to evaluate a trajectory prediction model and are based, at least in part, on a differentiable dynamic model.Type: ApplicationFiled: July 14, 2022Publication date: January 18, 2024Inventors: Yulong Cao, Chaowei Xiao, Danfei Xu, Anima Anandkumar, Marco Pavone
-
Publication number: 20240010196Abstract: In various examples, control policies for controlling agents may be learned from demonstrations capturing joint states of entities navigating through the environment. A control policy may be learned mapping joint states to control actions, where the joint states are between agents, and the control actions are of at least one of the agents. The control policy may be learned to define the mappings as control invariant sets of the joint sates and the control actions. The control policy may be used to determine one or more functions that compute, based at least on a joint state between entities, output indicating a likelihood of collision between the entities operating in accordance with the control policy. Using the output, current and/or potential states of the environment may be evaluated to determine control operations for a machine, such as a vehicle.Type: ApplicationFiled: March 14, 2023Publication date: January 11, 2024Inventors: Karen Yan Ming Leung, Sushant Veer, Edward Fu Schmerling, Marco Pavone