Patents by Inventor Stephen G. McGill

Stephen G. McGill has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11904854
    Abstract: A method includes receiving data relating to pedestrian activity at one or more locations outside of a crosswalk, analyzing the data, based on the data, identifying at least one location of the one or more locations as a constructive crosswalk, and controlling operation of an autonomous vehicle based on the at least one location of the constructive crosswalk.
    Type: Grant
    Filed: March 30, 2020
    Date of Patent: February 20, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Paul Drews
  • Publication number: 20240043020
    Abstract: System, methods, and other embodiments described herein relate to coordinating and optimizing prediction by a model using intermediate data and vehicle-to-vehicle (V2V) communications that improves bandwidth utilization. In one embodiment, a method includes processing acquired data, by a subject vehicle using a prediction model, to execute a vehicle task associated with navigating a driving scene. The method also includes extracting processed data according to the acquired data from intermediate activations of layers within the prediction model according to a cooperation model. The method also includes quantifying relevancy of the processed data, for a target vehicle, to select a subset using the cooperation model. The method also includes communicating the subset to a selected vehicle having the prediction model for additional prediction according to the relevancy and available resources to transmit the subset.
    Type: Application
    Filed: August 8, 2022
    Publication date: February 8, 2024
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Guy Rosman, Stephen G. McGill, JR., Paul Drews
  • Publication number: 20230415762
    Abstract: A method for indicating occlusion information at an ego agent includes observing a spatial area from a first viewpoint of one or more first sensors associated with the ego agent. The method also includes identifying the spatial area as an occluded area in accordance with observing the spatial area from a second viewpoint of the one or more first sensors after observing the spatial area from the first viewpoint. The method further includes receiving, from a target agent, a message indicating the spatial area is occluded from one or more second sensors associated with the target agent. The method still further includes transmitting, to the target agent in accordance with receiving the message, the occlusion information indicating information associated the spatial area based on identifying the spatial area as the occluded area.
    Type: Application
    Filed: September 8, 2023
    Publication date: December 28, 2023
    Applicant: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. MCGILL, Guy ROSMAN, Luke S. FLETCHER
  • Patent number: 11794762
    Abstract: A method for prioritizing occlusion information is presented. The method includes determining, at a first time period, a first sensor's view of a spatial area is occluded. The method also includes observing, at a second time period, the spatial area. The method further includes determining a level of risk associated with the spatial area based on the observation. The method still further includes prioritizing transmission of the occlusion information corresponding to the spatial area based on the determined level of risk and transmitting the occlusion information corresponding to the spatial area based on the priority.
    Type: Grant
    Filed: April 10, 2020
    Date of Patent: October 24, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher
  • Patent number: 11724691
    Abstract: Systems and methods described herein relate to estimating risk associated with a vehicular maneuver. One embodiment acquires a geometric representation of an intersection including a lane in which a vehicle is traveling and at least one other lane; discretizes the at least one other lane into a plurality of segments; determines a trajectory along which the vehicle will travel; estimates a probability density function for whether a road agent external to the vehicle is present in the respective segments; estimates a traffic-conflict probability of a traffic conflict in the respective segments conditioned on whether an external road agent is present; estimates a risk associated with the vehicle following the trajectory by integrating a product of the probability density function and the traffic-conflict probability over the at least one other lane and the plurality of segments; and controls operation of the vehicle based, at least in part, on the estimated risk.
    Type: Grant
    Filed: June 13, 2019
    Date of Patent: August 15, 2023
    Assignees: Toyota Research Institute, Inc., Massachusetts Institute of Technology
    Inventors: Stephen G. McGill, Jr., Guy Rosman, Moses Theodore Ort, Alyssa Pierson, Igor Gilitschenski, Minoru Brandon Araki, Luke S. Fletcher, Sertac Karaman, Daniela Rus, John Joseph Leonard
  • Patent number: 11718303
    Abstract: Vehicle systems and methods for autonomously controlling a vehicle to create a reaction by one or more surrounding vehicles, where the reaction is used to build one or more vehicle profiles are disclosed. In one embodiment, a vehicle includes an object detection system configured to output an object signal in response to detecting one or more vehicles operating in an environment surrounding the vehicle, an autonomous control system configured to autonomously control one or more vehicle systems of the vehicle, one or more processors, and one or more non-transitory memory modules communicatively coupled to the processors and storing machine-readable instructions that, when executed, cause the one or more processors to perform at least determining the vehicle is operating in an autonomous driving mode, and in response to determining the vehicle is operating in the autonomous driving mode, determine a presence of the one or more vehicles.
    Type: Grant
    Filed: January 3, 2018
    Date of Patent: August 8, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventor: Stephen G. McGill
  • Patent number: 11703874
    Abstract: Systems and methods to systematically identify and collect operational vehicle data for selective transmission to a non-local storage location for further analysis and use in training autonomous and semi-autonomous vehicles are provided. The systems and methods provided overcome limitations in storage and transmission of collected data by selectively archiving only that collected driving data warranting further analysis.
    Type: Grant
    Filed: March 10, 2022
    Date of Patent: July 18, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher
  • Patent number: 11663860
    Abstract: A method of providing dynamic and variable learning for an ego vehicle by determining and using most-trustworthy inputs includes determining, based on ambient conditions of an environment of the ego vehicle, a level of trustworthiness of sensor values obtained from one or more ego vehicle sensors. The method also includes determining a level of confidence in an accuracy of an output of a subsystem of the ego vehicle. The output of the subsystem is based on the sensor values. The level of confidence is based on the level of trustworthiness of the sensor values.
    Type: Grant
    Filed: October 25, 2019
    Date of Patent: May 30, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher
  • Patent number: 11654934
    Abstract: A system and method for generating a predicted vehicle trajectory includes a generative adversarial network configured to receive a trajectory vector of a target vehicle and generate a set of latent state vectors using the received trajectory vector and an artificial neural network. The latent state vectors each comprise a high-level sub-vector, ZH. The GAN enforces ZH to be correlated to an annotation coding representing semantic categories of vehicle trajectories. The GAN selects a subset, from the set of latent state vectors, using farthest point sampling and generates a predicted vehicle trajectory based on the selected subset of latent state vectors.
    Type: Grant
    Filed: November 25, 2020
    Date of Patent: May 23, 2023
    Assignees: TOYOTA RESEARCH INSTITUTE, INC., MASSACHUSETTS INSTITUTE OF TECHNOLOGY
    Inventors: Xin Huang, Stephen G. McGill, Jonathan A. DeCastro, Brian C. Williams, Luke S. Fletcher, John J. Leonard, Guy Rosman
  • Patent number: 11634123
    Abstract: A method includes receiving sensor data associated with one or more inputs associated with a road portion, determining a level of risk associated with each of the one or more inputs, determining an estimated amount of computing resources that each of a plurality of candidate computing methods will consume, and selecting one or more computing methods from the plurality of candidate computing methods to associate with the one or more inputs based on the levels of risk associated with the one or more inputs and the estimated amount of computing resources that the candidate computing methods will consume.
    Type: Grant
    Filed: July 9, 2020
    Date of Patent: April 25, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Nicholas Charles Fishwick
  • Patent number: 11628848
    Abstract: Systems and methods for training a neural network for estimating a trajectory of a vehicle are disclosed. In one embodiment, a system includes one or more processors, and a non-transitory computer-readable medium storing computer-readable instructions. The computer-readable instructions cause the one or more processors to receive sensor data of a plurality of examples from a plurality of vehicle sensors, input the sensor data into a sensor data neural network to generate a sensor data intermediate space and receive structured data of the plurality of examples. The computer-readable instructions cause the one or more processors to input the structured data into a structured data neural network to generate a structured data intermediate space, calculate a first loss between the sensor data intermediate space and the structured data intermediate space using a first loss function, and provide the first loss to the sensor data neural network and the structured data neural network.
    Type: Grant
    Filed: March 31, 2020
    Date of Patent: April 18, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman
  • Patent number: 11618476
    Abstract: A method including generating a curvilinear coordinate system from a current position of the vehicle. The method further includes generating a probability distribution from the curvilinear coordinate system and predicting a future vehicle behavior of the vehicle based on the probability distribution.
    Type: Grant
    Filed: June 5, 2020
    Date of Patent: April 4, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventors: Stephen G. McGill, Guy Rosman, Paul Drews
  • Publication number: 20230085422
    Abstract: A method for task-informed planning by a behavior planning system of a vehicle includes observing a previous trajectory of an agent within a distance from the vehicle. The method also includes predicting, by the behavior planning system, a set of potential trajectories for the agent and/or the vehicle based on observing the previous trajectory. The method further includes selecting, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value based on the respective potential action and the set of potential trajectories, the selected potential action being associated with a highest utility value of respective utility values associated with the set of potential actions. The method still further includes controlling the vehicle to perform an action associated with the potential action selected by the behavior planning system.
    Type: Application
    Filed: July 22, 2022
    Publication date: March 16, 2023
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA, MASSACHUSETTS INSTITUE OF TECHNOLOGY
    Inventors: Xin HUANG, Guy ROSMAN, Ashkan Mohammadzadeh JASOUR, Stephen G. McGILL, JR., John J. LEONARD, Brian C. WILLIAMS
  • Publication number: 20230056475
    Abstract: A method for scenario-based event triggers is described. The method includes generating, by a first machine-learning (ML) model, feature vectors encoding driving scenarios surrounding an ego vehicle. The method also includes detecting, by a second machine-learning (ML) model, a unique driving scenario outside of pre-programmed event triggers corresponding to one of the feature vectors encoding driving scenarios surrounding the ego vehicle. The method further includes triggering uploading of the unique driving scenario outside of pre-programmed event triggers to a central scenario-based event control server.
    Type: Application
    Filed: August 20, 2021
    Publication date: February 23, 2023
    Applicant: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Xiongyi CUI, Stephen G. MCGILL, Guy ROSMAN, Simon A. I. STENT
  • Patent number: 11548521
    Abstract: Systems, vehicles and methods for determining wrong direction driving are disclosed. In one embodiment, a system for determining a vehicle traveling in a wrong direction includes one or more sensors that produce sensor data, one or more processors, and one or more non-transitory computer-readable medium storing computer readable-instructions. When the computer-readable instructions are executed by the one or more processors, the computer-readable instructions cause the one or more processors to determine one or more lanes within a roadway using the sensor data, determine a direction of travel of the one or more lanes using the sensor data, and identify a non-compliant vehicle traveling in a direction in the one or more lanes that is different from the determined direction of travel in the one or more lanes.
    Type: Grant
    Filed: March 31, 2020
    Date of Patent: January 10, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher, Simon A. I. Stent
  • Publication number: 20220410938
    Abstract: Systems and methods for predicting a trajectory of a moving object are disclosed herein. One embodiment downloads, to a robot, a probabilistic hybrid discrete-continuous automaton (PHA) model learned as a deep neural network; uses the deep neural network to infer a sequence of high-level discrete modes and a set of associated low-level samples, wherein the high-level discrete modes correspond to candidate maneuvers for the moving object and the low-level samples are candidate trajectories; uses the sequence of high-level discrete modes and the set of associated low-level samples, via a learned proposal distribution in the deep neural network, to adaptively sample the sequence of high-level discrete modes to produce a reduced set of low-level samples; applies a sample selection technique to the reduced set of low-level samples to select a predicted trajectory for the moving object; and controls operation of the robot based, at least in part, on the predicted trajectory.
    Type: Application
    Filed: December 1, 2021
    Publication date: December 29, 2022
    Applicants: Toyota Research Institute, Inc., Massachusetts Institute of Technology
    Inventors: Xin Huang, Igor Gilitschenski, Guy Rosman, Stephen G. McGill, JR., John Joseph Leonard, Ashkan Mohammadzadeh Jasour, Brian C. Williams
  • Patent number: 11529966
    Abstract: A method for controlling a driving behavior performed by a first agent includes navigating, by the first agent, according to a trajectory and a velocity. The method also includes receiving, from a second agent, a risk identification message identifying a third agent as a potential risk based on the third agent performing a behavior associated with a probability that is less than a threshold. The method further includes autonomously engaging a defensive driving mode in response to receiving the risk identification message. The method still further includes adjusting one or both of the trajectory or the velocity in response to autonomously engaging the defensive driving mode.
    Type: Grant
    Filed: April 1, 2021
    Date of Patent: December 20, 2022
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher
  • Patent number: 11531865
    Abstract: Systems and methods for parallel autonomy of a vehicle are disclosed herein. One embodiment receives input data, the input data including at least one of sensor data and structured input data; encodes the input data into an intermediate embedding space using a first neural network; inputs the intermediate embedding space to a first behavior model and a second behavior model, the first behavior model producing a first behavior output, the second behavior model producing a second behavior output; combines the first behavior output and the second behavior output using an ideal-behavior model to produce an ideal behavior for the vehicle; and controls one or more aspects of operation of the vehicle based, at least in part, on the ideal behavior.
    Type: Grant
    Filed: February 28, 2020
    Date of Patent: December 20, 2022
    Assignee: Toyota Research Institute, Inc.
    Inventors: Stephen G. McGill, Jr., Guy Rosman, Luke S. Fletcher, John Joseph Leonard
  • Patent number: 11524701
    Abstract: A method includes encapsulating a current state of a road portion into a voxel grid, a first dimension of the voxel grid being associated with a first spatial dimension of the road portion, a second dimension of the voxel grid being associated with a second spatial dimension of the road portion, and a third dimension of the voxel grid comprising a plurality of feature layers, wherein each feature layer is associated with a feature of the road portion. The voxel grid may be input into a trained neural network and a future state of the road portion may be predicted based on an output of the neural network.
    Type: Grant
    Filed: March 30, 2020
    Date of Patent: December 13, 2022
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Paul Drews, Guy Rosman
  • Patent number: 11475720
    Abstract: An embodiment takes the form of a vehicle that generates a data collection configuration for one or more vehicle sensors of a vehicle based on an estimated information gain to a neural network were the vehicle to provision the neural network with notional sensor data, and based on a vehicle resource consumption by the vehicle were the vehicle to provision the neural network with the notional sensor data. The notional sensor data comprises sensor data that would be collected from a given sensor among the vehicle sensors according to a respective sensor configuration of the given sensor. The vehicle collects sensor data from the vehicle sensors according to the generated data collection configuration.
    Type: Grant
    Filed: March 31, 2020
    Date of Patent: October 18, 2022
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. McGill, Guy Rosman, Luke S. Fletcher, John J. Leonard, Simon Stent