Patents Assigned to Toyota Research Institute
  • Publication number: 20240010218
    Abstract: Systems and methods for learning and managing robot user interfaces are disclosed herein. One embodiment generates, based on input data including information about past interactions of a particular user with a robot and with existing HMIs of the robot, a latent space using one or more encoder neural networks, wherein the latent space is a reduced-dimensionality representation of learned behavior and characteristics of the particular user, and uses the latent space as input to train a decoder neural network associated with (1) a new HMI distinct from the existing HMIs or (2) a particular HMI among the existing HMIs to alter operation of the particular HMI. The trained first decoder neural network is deployed in the robot to control, at least in part, operation of the new HMI or the particular HMI in accordance with the learned behavior and characteristics of the particular user.
    Type: Application
    Filed: August 26, 2022
    Publication date: January 11, 2024
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Guy Rosman, Daniel J. Brooks, Simon A.I. Stent, Tiffany Chen, Emily Sarah Sumner, Shabnam Hakimi, Jonathan DeCastro, Deepak Edakkattil Gopinath
  • Publication number: 20240012964
    Abstract: A method of closed loop simulation for accelerated material discovery is described. The method includes ranking a plurality of candidate systems according to corresponding properties of interest predicted by a first prediction model. The method also includes simulating a first top-N of the plurality of candidate systems according to the corresponding properties of interest predicted by the first prediction model. The method further includes re-ranking the plurality of candidate systems according to the corresponding properties of interest predicted by a second prediction model. The method also includes simulating a second top-N of the plurality of candidate systems according to the corresponding properties of interest predicted by the second prediction model.
    Type: Application
    Filed: July 7, 2022
    Publication date: January 11, 2024
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA
    Inventors: Daniel SCHWEIGERT, Ha-Kyung KWON, Arash KHAJEH
  • Publication number: 20240010225
    Abstract: A method of representation learning for object detection from unlabeled point cloud sequences is described. The method includes detecting moving object traces from temporally-ordered, unlabeled point cloud sequences. The method also includes extracting a set of moving objects based on the moving object traces detected from the sequence of temporally-ordered, unlabeled point cloud sequences. The method further includes classifying the set of moving objects extracted from on the moving object traces detected from the sequence of temporally-ordered, unlabeled point cloud sequences. The method also includes estimating 3D bounding boxes for the set of moving objects based on the classifying of the set of moving objects.
    Type: Application
    Filed: July 7, 2022
    Publication date: January 11, 2024
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA, MASSACHUSETTS INSTITUE OF TECHNOLOGY
    Inventors: Xiangru HUANG, Yue WANG, Vitor GUIZILINI, Rares Andrei AMBRUS, Adrien David GAIDON, Justin SOLOMON
  • Patent number: 11864897
    Abstract: Systems and methods for determining whether a user employs System 1 type thinking or System 2 type thinking when engaged in a task are disclosed. The systems and methods include determining one or more properties of the task based on information regarding the task received from a database storing information regarding the task, determining one or more properties of the user with respect to the task, determining a state of the user based on one or more physiological sensors configured to sense one or more characteristics of the user, and determining that the user employs System 1 type thinking or System 2 type thinking when engaged in the task based on the determined one or more properties of the task, the determined one or more properties of the user, and the determined state of the user.
    Type: Grant
    Filed: April 12, 2021
    Date of Patent: January 9, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Rumen Iliev, Kent Lyons, Charlene C. Wu, Matthew Lee, Yanxia Zhang, Yue Weng
  • Patent number: 11868439
    Abstract: Systems, methods, and other embodiments described herein relate to training a multi-task network using real and virtual data. In one embodiment, a method includes acquiring training data that includes real data and virtual data for training a multi-task network that performs at least depth prediction and semantic segmentation. The method includes generating a first output from the multi-task network using the real data and second output from the multi-task network using the virtual data. The method includes generating a mixed loss by analyzing the first output to produce a real loss and the second output to produce a virtual loss. The method includes updating the multi-task network using the mixed loss.
    Type: Grant
    Filed: March 29, 2021
    Date of Patent: January 9, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Vitor Guizilini, Adrien David Gaidon, Jie Li, Rares A. Ambrus
  • Patent number: 11865965
    Abstract: System, methods, and other embodiments described herein relate to communicating alerts about a recommended speed by adapting headlight color. In one embodiment, a method includes computing a recommended speed using an automated driving system (ADS) during an operator controlling a vehicle while the ADS is disengaged. The method also includes, responsive to determining that a vehicle speed satisfies a threshold associated with the recommended speed and an offset value set according to a driving environment, adapting an alert color projected by headlights of the vehicle according to the vehicle speed and the alert color is corrected for operator perception of visible colors associated with the driving environment.
    Type: Grant
    Filed: February 23, 2022
    Date of Patent: January 9, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Hiroshi Yasuda, Manuel Ludwig Kuehner, Guillermo Pita Gil
  • Patent number: 11868600
    Abstract: Methods, systems, and non-transitory machine-readable mediums for ranking on an absolute scale include displaying, on an electronic display, a first handle, a second handle, and an interactor, determining a value of the first handle and a value of the second handle based on their respective positions on the interactor, in response to a user dragging the first and second handles on the interactor, and determining a rank of the first and second handles based on the values of the first and second handles, in response to the user dragging the first and second handles on the interactor.
    Type: Grant
    Filed: August 4, 2021
    Date of Patent: January 9, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Scott Carter, Alex Filipowicz, Shabnam Hakimi
  • Publication number: 20240001566
    Abstract: A robotic arm includes a suction tool including a suction mechanism, a first pump, and a second pump. The first pump is operable to provide a first suction force and a first flow rate to the suction mechanism, and the second pump is operable to provide a second suction force and a second flow rate to the suction mechanism. A connection member places the first pump and the second pump in fluid communication with the suction mechanism.
    Type: Application
    Filed: October 14, 2022
    Publication date: January 4, 2024
    Applicant: Toyota Research Institute, Inc.
    Inventor: Lukas S. Kaul
  • Publication number: 20240005627
    Abstract: A method of conditional neural ground planes for static-dynamic disentanglement is described. The method includes extracting, using a convolutional neural network (CNN), CNN image features from an image to form a feature tensor. The method also includes resampling unprojected 2D features of the feature tensor to form feature pillars. The method further includes aggregating the feature pillars to form an entangled neural ground plane. The method also includes decomposing the entangled neural ground plane into a static neural ground plane and a dynamic neural ground plane.
    Type: Application
    Filed: April 18, 2023
    Publication date: January 4, 2024
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA, MASSACHUSETTS INSTITUTE OF TECHNOLOGY
    Inventors: Prafull SHARMA, Ayush TEWARI, Yilun DU, Sergey ZAKHAROV, Rares Andrei AMBRUS, Adrien David GAIDON, William Tafel FREEMAN, Frederic Pierre DURAND, Joshua B. TENENBAUM, Vincent SITZMANN
  • Publication number: 20240005540
    Abstract: System, methods, and other embodiments described herein relate to an improved approach to training a depth model to derive depth estimates from monocular images using cost volumes. In one embodiment, a method includes predicting, using a depth model, depth values from at least one input image that is a monocular image. The method includes generating a cost volume by sampling the depth values corresponding to bins of the cost volume. The method includes determining loss values for the bins of the cost volume. The method includes training the depth model according to the loss values of the cost volume.
    Type: Application
    Filed: May 27, 2022
    Publication date: January 4, 2024
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Vitor Guizilini, Rares A. Ambrus, Sergey Zakharov
  • Patent number: 11858497
    Abstract: Systems and methods of controlling a vehicle in a stable drift are provided. With the goal of enhancing the driver experience, the disclosed drift control systems provide an interactive drift driving experience for the driver of a vehicle. In some embodiments, a driver is allowed to take manual control of a vehicle after a stable drift is initiated. For safety reasons, an assisted driving system may provide corrective assistance to prevent the vehicle from entering an unstable/unsafe drift. In other embodiments, an autonomous driving system retains control of the vehicle throughout the drift. However, the driver may perform “simulated drift maneuvers” such as counter-steering, and clutch kicking in order to communicate their desire to drift more or less aggressively. Accordingly, the autonomous driving system will effectuate the driver's communicated desire in a manner that keeps the vehicle in a safe/stable drift.
    Type: Grant
    Filed: June 14, 2021
    Date of Patent: January 2, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Avinash Balachandran, Yan Ming Jonathan Goh, John Subosits, Michael Thompson, Alexander R. Green
  • Patent number: 11858395
    Abstract: Systems and methods are provided for improved and intuitive adjustment of a vehicle headrest. A vehicle headrest adjustment system may include touch sensors and/or motion sensors to determine a driver or passenger intends to adjust a vehicle headrest. Upon detection of touch consistent with intent to adjust a vehicle headrest and/or a hand gesture consistent with intent to adjust a vehicle headrest, a system may unlock a vehicle headrest. A driver or passenger may move the unlocked vehicle headrest into the desired position. Upon detection of a condition indicating the driver or passenger is finished and/or upon a set period of time elapsing, the system may re-lock the vehicle headrest into place. A vehicle headrest adjustment system may also include a power-assist function. The power-assist function may include motors and force sensors to assist a driver or passenger in moving a vehicle headrest into a desired position.
    Type: Grant
    Filed: January 13, 2022
    Date of Patent: January 2, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Manuel Ludwig Kuehner, Daniel J. Brooks, Hiroshi Yasuda, Jaime Camhi
  • Publication number: 20230415762
    Abstract: A method for indicating occlusion information at an ego agent includes observing a spatial area from a first viewpoint of one or more first sensors associated with the ego agent. The method also includes identifying the spatial area as an occluded area in accordance with observing the spatial area from a second viewpoint of the one or more first sensors after observing the spatial area from the first viewpoint. The method further includes receiving, from a target agent, a message indicating the spatial area is occluded from one or more second sensors associated with the target agent. The method still further includes transmitting, to the target agent in accordance with receiving the message, the occlusion information indicating information associated the spatial area based on identifying the spatial area as the occluded area.
    Type: Application
    Filed: September 8, 2023
    Publication date: December 28, 2023
    Applicant: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Stephen G. MCGILL, Guy ROSMAN, Luke S. FLETCHER
  • Patent number: 11854280
    Abstract: A method for 3D object detection is described. The method includes detecting semantic keypoints from monocular images of a video stream capturing a 3D object. The method also includes inferring a 3D bounding box of the 3D object corresponding to the detected semantic vehicle keypoints. The method further includes scoring the inferred 3D bounding box of the 3D object. The method also includes detecting the 3D object according to a final 3D bounding box generated based on the scoring of the inferred 3D bounding box.
    Type: Grant
    Filed: April 27, 2021
    Date of Patent: December 26, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Arjun Bhargava, Haofeng Chen, Adrien David Gaidon, Rares A. Ambrus, Sudeep Pillai
  • Patent number: 11851084
    Abstract: Systems and methods for controlling an autonomous vehicle are disclosed herein. One embodiment determines a reference path for the autonomous vehicle along a roadway segment and steers the autonomous vehicle along a path that includes controlled back and forth lateral deviations from the reference path along the roadway segment to provide feedback to an occupant of the autonomous vehicle, the feedback indicating to the occupant that the autonomous vehicle is in an autonomous driving mode and that the autonomous driving mode is operating correctly.
    Type: Grant
    Filed: April 16, 2021
    Date of Patent: December 26, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventors: Manuel Ludwig Kuehner, Daniel J. Brooks, Hiroshi Yasuda, Jaime Camhi
  • Publication number: 20230409880
    Abstract: Systems and methods for generating predicted preferences are disclosed. The method includes implementing, with a computing device having a processor and a non-transitory computer-readable memory, a conjoint architecture comprising: an autoencoder trained to transform input data including one or more choices and one or more features into a latent representation, and a choice classification network trained to predict one or more predicted preferences from the latent representation extracted by the autoencoder. The method further includes outputting, from the choice classification network, the one or more predicted preferences.
    Type: Application
    Filed: February 24, 2023
    Publication date: December 21, 2023
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Yanxia Zhang, Francine R. Chen, Rumen Iliev, Totte Harinen, Alexandre L.S. Filipowicz, Yin-Ying Chen, Nikos Arechiga Gonzalez, Shabnam Hakimi, Kenton Michael Lyons, Charlene C. Wu, Matthew E. Klenk
  • Patent number: 11847127
    Abstract: A method of identifying causal relationships includes receiving data comprising a set of values corresponding to one or more variables, and generating a list of candidate causal models of relationships between or within the variables. The list is ranked based on a likelihood of each candidate causal model, wherein the likelihood includes at least a correlation value. The method further includes receiving feedback identifying a candidate causal model and a change in rank of the candidate causal model, re-ranking the list based on the feedback, and displaying the re-ranked list. The method generates an intervention comprising a suggested modification corresponding to a variable of a selected causal model among the candidate causal models in the re-ranked list, receives additional data corresponding to the variable of the suggested modification and evaluates the additional data to determine whether the likelihood of the selected causal model has changed.
    Type: Grant
    Filed: May 12, 2021
    Date of Patent: December 19, 2023
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Rumen Iliev, Totte Harri Harinen
  • Patent number: 11847840
    Abstract: A distracted driver can be informed of his or her distraction by a visual notification. It can be detected whether a driver of a vehicle is focused on a non-critical object located within the vehicle. In response to detecting that the driver of the vehicle is focused on a non-critical object located within the vehicle, an amount of time the driver is focused on the non-critical object can be determined. When the amount of time exceeds a threshold amount of time, a visual notification of distracted driving can be caused to be presented on or visually adjacent to the non-critical object.
    Type: Grant
    Filed: August 24, 2021
    Date of Patent: December 19, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventors: Hiroshi Yasuda, Manuel Ludwig Kuehner
  • Patent number: 11845457
    Abstract: A system for training an operator of a vehicle includes a processor and a memory in communication with the processor, which includes a safety module and a training module. The safety module has instructions that, when executed by the processor, cause the processor to determine when the vehicle is operating within a safe area based on at least one of: a location of the vehicle and a location of one or more objects with respect to the vehicle. The training module has instructions that, when executed by the processor, cause the processor to apply at least one brake of the vehicle when the vehicle is operating within the safe area to cause the vehicle to engage in an oversteer event, and collect operator response information when the vehicle engages in the oversteer event.
    Type: Grant
    Filed: February 19, 2021
    Date of Patent: December 19, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventors: John Subosits, Yan Ming Jonathan Goh, Michael Thompson, Alexander R. Green, Avinash Balachandran
  • Publication number: 20230398696
    Abstract: A robotic system is contemplated. The robotic system comprises a robot comprising a camera, a microphone, memory, and a controller that is configured to receive a natural language command for performing an action within a real world environment, parse the natural language command, categorize the action as being associated with guidance for performing the action, receive the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action, and store, in the memory, the natural language command in correlation with the motion that is applied to the at least one portion of the robot.
    Type: Application
    Filed: June 14, 2022
    Publication date: December 14, 2023
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventor: Thomas Kollar