Patents by Inventor Sertac Karaman
Sertac Karaman has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240119857Abstract: System, methods, and other embodiments described herein relate to training a scene simulator for rendering 2D scenes using data from real and simulated agents. In one embodiment, a method includes acquiring trajectories and three-dimensional (3D) views for multiple agents from observations of real vehicles. The method also includes generating a 3D scene having the multiple agents using the 3D views and information from simulated agents. The method also includes training a scene simulator to render scene projections using the 3D scene. The method also includes outputting a 2D scene having simulated observations for a driving scene using the scene simulator.Type: ApplicationFiled: September 27, 2022Publication date: April 11, 2024Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha, Massachusetts Institute of TechnologyInventors: Tsun-Hsuan Wang, Alexander Amini, Wilko Schwarting, Igor Gilitschenski, Sertac Karaman, Daniela Rus
-
Patent number: 11884302Abstract: Understanding the intent of human drivers and adapting to their driving styles is used to increased efficiency and safety of autonomous vehicles (AVs) by enabling them to behave in safe and predictable ways without requiring explicit inter-vehicle communication. A Social Value Orientation (SVO), which quantifies the degree of an agent's selfishness or altruism, is estimated by the AV for other vehicles to better predict how they will interact and cooperate with others. Interactions between agents are modeled as a best response game wherein each agent negotiates to maximize their own utility. A dynamic game solution uses the Nash equilibrium, yielding an online method of predicting multi-agent interactions given their SVOs. This approach allows autonomous vehicles to observe human drivers, estimate their SVOs, and generate an autonomous control policy in real time.Type: GrantFiled: November 16, 2020Date of Patent: January 30, 2024Assignee: Massachusetts Institute of TechnologyInventors: Daniela Rus, Sertac Karaman, Javier Alonso Mora, Alyssa Pierson, Wilko Schwarting
-
Patent number: 11808590Abstract: An approach to autonomous navigation of a vehicle augments a static map of an environment with a clutter map characterizing a risk of encountering an object that is not represented in the static map of the environment. For example, the clutter map may be based on locations and velocities of those objects, and route planning may avoid planning a path through locations that have a high risk of occupancy, and therefore potential delay or collision.Type: GrantFiled: January 13, 2020Date of Patent: November 7, 2023Assignee: Massachusetts Institute of TechnologyInventors: Daniela Rus, Sertac Karaman, Wilko Schwarting, Anshula Gandhi, Cristian-Ioan Vasile, Alyssa Pierson
-
Patent number: 11724691Abstract: Systems and methods described herein relate to estimating risk associated with a vehicular maneuver. One embodiment acquires a geometric representation of an intersection including a lane in which a vehicle is traveling and at least one other lane; discretizes the at least one other lane into a plurality of segments; determines a trajectory along which the vehicle will travel; estimates a probability density function for whether a road agent external to the vehicle is present in the respective segments; estimates a traffic-conflict probability of a traffic conflict in the respective segments conditioned on whether an external road agent is present; estimates a risk associated with the vehicle following the trajectory by integrating a product of the probability density function and the traffic-conflict probability over the at least one other lane and the plurality of segments; and controls operation of the vehicle based, at least in part, on the estimated risk.Type: GrantFiled: June 13, 2019Date of Patent: August 15, 2023Assignees: Toyota Research Institute, Inc., Massachusetts Institute of TechnologyInventors: Stephen G. McGill, Jr., Guy Rosman, Moses Theodore Ort, Alyssa Pierson, Igor Gilitschenski, Minoru Brandon Araki, Luke S. Fletcher, Sertac Karaman, Daniela Rus, John Joseph Leonard
-
Publication number: 20230110082Abstract: A vehicular control system includes a plurality of electronic control units (ECUs), each providing a respective quantity of computational units representative of an amount of processing power of the respective ECU. The ECUs operate a vehicle in a nominal autonomous operational mode when a sum of the quantity of computational units exceeds a threshold. The system, while the ECUs operate the vehicle in the nominal autonomous operational mode, and responsive to detecting a failure of one of the ECUs, determines whether a sum of the quantity of computational units of the remaining ECUs that do not have a failure exceeds the threshold. The ECUs, responsive to the system determining that the sum of the quantity of computational units of the remaining ECUs fails to exceed the threshold, switches from operating the vehicle in the nominal autonomous operational mode to operating the vehicle in a degraded autonomous operational mode.Type: ApplicationFiled: November 23, 2022Publication date: April 13, 2023Inventors: Albert Huang, Sertac Karaman, Ryan C.C. Chin, Jenny Larios Berlin, Ramiro Almeida
-
Publication number: 20230062810Abstract: A method of generating an output trajectory of an ego vehicle is described. The method includes extracting high-level features from a bird-view image of a traffic environment of the ego vehicle. The method also includes generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features. The method further includes generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.Type: ApplicationFiled: July 9, 2021Publication date: March 2, 2023Applicants: TOYOTA RESEARCH INSTITUTE, INC., MASSACHUSETTS INSTITUTE OF TECHNOLOGY, LEHIGH UNIVERITYInventors: Xiao LI, Brandon ARAKI, Sertac KARAMAN, Daniela RUS, Guy ROSMAN, Igor GILITSCHENSKI, Cristian-Ioan VASILE
-
Patent number: 11511762Abstract: A method, computer program product, and computing system for operating an autonomous vehicle; monitoring the operation of a plurality of computing devices within the autonomous vehicle; and in response to detecting the failure of one or more of the plurality of computing devices, switching the autonomous vehicle from a nominal autonomous operational mode to a degraded autonomous operational mode.Type: GrantFiled: November 13, 2019Date of Patent: November 29, 2022Assignee: MAGNA ELECTRONICS INC.Inventors: Albert Huang, Sertac Karaman, Ryan C. C. Chin, Jenny Larios-Berlin, Ramiro Almeida
-
Patent number: 11436839Abstract: The present disclosure provides systems and methods to detect occluded objects using shadow information to anticipate moving obstacles that are occluded behind a corner or other obstacle. The system may perform a dynamic threshold analysis on enhanced images allowing the detection of even weakly visible shadows. The system may classify an image sequence as either “dynamic” or “static”, enabling an autonomous vehicle, or other moving platform, to react and respond to a moving, yet occluded object by slowing down or stopping.Type: GrantFiled: November 2, 2018Date of Patent: September 6, 2022Assignees: TOYOTA RESEARCH INSTITUTE, INC., MASSACHUSETTS INSTITUE OF TECHNOLOGYInventors: Felix Maximilian Naser, Igor Gilitschenski, Guy Rosman, Alexander Andre Amini, Fredo Durand, Antonio Torralba, Gregory Wornell, William Freeman, Sertac Karaman, Daniela Rus
-
Patent number: 11427210Abstract: Systems and methods for predicting the trajectory of an object are disclosed herein. One embodiment receives sensor data that includes a location of the object in an environment of the object; accesses a location-specific latent map, the location-specific latent map having been learned together with a neural-network-based trajectory predictor during a training phase, wherein the neural-network-based trajectory predictor is deployed in a robot; inputs, to the neural-network-based trajectory predictor, the location of the object and the location-specific latent map, the location-specific latent map providing, to the neural-network-based trajectory predictor, a set of location-specific biases regarding the environment of the object; and outputs, from the neural-network-based trajectory predictor, a predicted trajectory of the object.Type: GrantFiled: March 31, 2020Date of Patent: August 30, 2022Assignees: Toyota Research Institute, Inc., Massachusetts Institute of TechnologyInventors: Guy Rosman, Igor Gilitschenski, Arjun Gupta, Sertac Karaman, Daniela Rus
-
Patent number: 11300968Abstract: A method is disclosed for use in a planning agent, the method including: identifying a first agent in a vicinity of the planning agent; identifying a location of the first agent and a velocity of the first agent; calculating a set of occupancy costs for the first agent, each occupancy cost in the set of occupancy costs being associated with a different respective location in the vicinity of the planning agent, each occupancy cost in the set of occupancy costs being calculated at least in part based on a cost function that depends on the location of the first agent and the velocity of the first agent; and changing at least one of speed or direction of travel of the planning agent based on the set of occupancy costs.Type: GrantFiled: February 22, 2019Date of Patent: April 12, 2022Assignee: Massachusetts Institute of TechnologyInventors: Alyssa Pierson, Wilko Schwarting, Sertac Karaman, Daniela L. Rus
-
Patent number: 11295162Abstract: An approach to place recognition from an image makes use of the detection of objects at a set of known places as well as at an unknown place. Images of the detected objects in an image of the unknown place are processed to yield respective numerical descriptors, and these descriptors are used to compare the unknown place to the known places to recognize the unknown place. At least some embodiments make use of a trained parameterized image processor to transform an image of an object to an object descriptor, and the training of the processor is meant to preserve distinctions between different instances of a type of object, as well as distinctions between entirely different types of objects.Type: GrantFiled: November 1, 2019Date of Patent: April 5, 2022Assignees: Massachusetts Institute of Technology, ETH ZurichInventors: Daniela Rus, Sertac Karaman, Igor Gilitschenski, Andrei Cramariuc, Cesar Cadena, Roland Siegwart
-
Publication number: 20210389776Abstract: A controller for an autonomous vehicle is trained using simulated paths on a roadway and simulated observations that are formed by transforming images previously acquired on similar paths on that roadway. Essentially an unlimited number of paths may be simulated, enabling optimization approaches including reinforcement learning to be applied to optimize the controller.Type: ApplicationFiled: June 11, 2021Publication date: December 16, 2021Inventors: Daniela Rus, Sertac Karaman, Igor Gilitschenski, Alexander Amini, Julia Moseyko, Jacob Phillips
-
Patent number: 11181383Abstract: Systems and methods described herein relate to vehicular navigation and localization. One embodiment extracts perceptual features from sensor data; extracts unrouted-map features from unrouted map data; combines the perceptual features and the unrouted-map features to produce first combined features data; outputs, based at least in part on the first combined features data, parameters of a probability distribution for one or more steering trajectories that are available to a vehicle; and performs a localization of the vehicle based, at least in part, on the parameters of the probability distribution.Type: GrantFiled: May 8, 2019Date of Patent: November 23, 2021Assignees: Toyota Research Institute, Inc., Massachusetts Institute of TechnologyInventors: Alexander Amini, Guy Rosman, Sertac Karaman, Daniela Rus
-
Publication number: 20210339767Abstract: A method, computer program product, and computing system for receiving metric data that is based, at least in part, upon sensor data generated by various sensors of an autonomous vehicle; processing the metric data; and generating a temporal understanding with respect to the autonomous vehicle based, at least in part, upon the metric data.Type: ApplicationFiled: May 4, 2021Publication date: November 4, 2021Inventors: Sertac Karaman, Albert Huang
-
Publication number: 20210341941Abstract: A method, computer program product, and computing system for receiving metric data that is based, at least in part, upon sensor data generated by various sensors of an autonomous vehicle; and processing the metric data to generate a semantic understanding of the autonomous vehicle.Type: ApplicationFiled: May 4, 2021Publication date: November 4, 2021Inventors: Sertac Karaman, Albert Huang
-
Publication number: 20210339766Abstract: A method, computer program product, and computing system for receiving situational data from an infrastructure system; processing the situational data to identify one or more AV-impacting conditions; generating AV instructions based, at least in part, upon the one or more AV-impacting conditions; and providing the AV instructions to one or more autonomous vehicles.Type: ApplicationFiled: May 4, 2021Publication date: November 4, 2021Inventors: SERTAC KARAMAN, Albert Huang
-
Publication number: 20210146964Abstract: Understanding the intent of human drivers and adapting to their driving styles is used to increased efficiency and safety of autonomous vehicles (AVs) by enabling them to behave in safe and predictable ways without requiring explicit inter-vehicle communication. A Social Value Orientation (SVO), which quantifies the degree of an agent's selfishness or altruism, is estimated by the AV for other vehicles to better predict how they will interact and cooperate with others. Interactions between agents are modeled as a best response game wherein each agent negotiates to maximize their own utility. A dynamic game solution uses the Nash equilibrium, yielding an online method of predicting multi-agent interactions given their SVOs. This approach allows autonomous vehicles to observe human drivers, estimate their SVOs, and generate an autonomous control policy in real time.Type: ApplicationFiled: November 16, 2020Publication date: May 20, 2021Inventors: Daniela Rus, Sertac Karaman, Javier Alonso Mora, Alyssa Pierson, Wilko Schwarting
-
Patent number: 11010622Abstract: A method of non-line-of-sight (NLoS) obstacle detection for an ego vehicle is described. The method includes capturing a sequence of images over a period with an image capture device. The method also includes storing the sequence of images in a cyclic buffer. The method further includes registering each image in the cyclic buffer to a projected image. The method includes performing the registering by estimating a homography H for each frame of the sequence of images to project to a view point of a first frame in the sequence of images and remove motion of the ego vehicle in the projected image. The method also includes enhancing the projected image. The method further includes classifying the projected image based on a scene determination. The method also includes issuing a control signal to the vehicle upon classifying the projected image.Type: GrantFiled: December 30, 2019Date of Patent: May 18, 2021Assignees: TOYOTA RESEARCH INSTITUTE, INC., MASSACHUSETTS INSTITUE OF TECHNOLOGYInventors: Felix Maximilian Naser, Igor Gilitschenski, Alexander Andre Amini, Christina Liao, Guy Rosman, Sertac Karaman, Daniela Rus
-
Publication number: 20210133480Abstract: An approach to place recognition from an image makes use of the detection of objects at a set of known places as well as at an unknown place. Images of the detected objects in an image of the unknown place are processed to yield respective numerical descriptors, and these descriptors are used to compare the unknown place to the known places to recognize the unknown place. At least some embodiments make use of a trained parameterized image processor to transform an image of an object to an object descriptor, and the training of the processor is meant to preserve distinctions between different instances of a type of object, as well as distinctions between entirely different types of objects.Type: ApplicationFiled: November 1, 2019Publication date: May 6, 2021Inventors: Daniela Rus, Sertac Karaman, Igor Gilitschenski, Andrei Cramariuc, Cesar Cadena, Roland Siegwart
-
Publication number: 20210081715Abstract: Systems and methods for predicting the trajectory of an object are disclosed herein. One embodiment receives sensor data that includes a location of the object in an environment of the object; accesses a location-specific latent map, the location-specific latent map having been learned together with a neural-network-based trajectory predictor during a training phase, wherein the neural-network-based trajectory predictor is deployed in a robot; inputs, to the neural-network-based trajectory predictor, the location of the object and the location-specific latent map, the location-specific latent map providing, to the neural-network-based trajectory predictor, a set of location-specific biases regarding the environment of the object; and outputs, from the neural-network-based trajectory predictor, a predicted trajectory of the object.Type: ApplicationFiled: March 31, 2020Publication date: March 18, 2021Inventors: Guy Rosman, Igor Gilitschenski, Arjun Gupta, Sertac Karaman, Daniela Rus