Patents by Inventor Kuan-Hui Lee

Kuan-Hui Lee has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12148223
    Abstract: A method for generating a dense light detection and ranging (LiDAR) representation by a vision system includes receiving, at a sparse depth network, one or more sparse representations of an environment. The method also includes generating a depth estimate of the environment depicted in an image captured by an image capturing sensor. The method further includes generating, via the sparse depth network, one or more sparse depth estimates based on receiving the one or more sparse representations. The method also includes fusing the depth estimate and the one or more sparse depth estimates to generate a dense depth estimate. The method further includes generating the dense LiDAR representation based on the dense depth estimate and controlling an action of the vehicle based on identifying a three-dimensional object in the dense LiDAR representation.
    Type: Grant
    Filed: April 28, 2022
    Date of Patent: November 19, 2024
    Assignees: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA
    Inventors: Arjun Bhargava, Chao Fang, Charles Christopher Ochoa, Kun-Hsin Chen, Kuan-Hui Lee, Vitor Guizilini
  • Publication number: 20240363000
    Abstract: A method for vehicle prediction, planning, and control is described. The method includes separately encoding traffic state information at an intersection into corresponding traffic state latent spaces. The method also includes aggregating the corresponding traffic state latent spaces to form a generalized traffic geometry latent space. The method further includes interpreting the generalized traffic geometry latent space to form a traffic flow map including current and future vehicle trajectories. The method also includes decoding the generalized traffic geometry latent space to predict a vehicle behavior according to the traffic flow map based on the current and future vehicle trajectories.
    Type: Application
    Filed: July 11, 2024
    Publication date: October 31, 2024
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA
    Inventors: Kuan-Hui LEE, Charles Christopher OCHOA, Arun BHARGAVA, Chao FANG, Kun-Hsin CHEN
  • Publication number: 20240355983
    Abstract: A display apparatus includes a driving backplane and a light emitting component. The driving backplane has a first pad and a second pad. The light emitting component is disposed on the driving backplane. The light emitting component includes a first semiconductor layer, a second semiconductor layer, an active layer, a first electrode, a second electrode, a first solder and a second solder. The first solder and the second solder of the light-emitting component are respectively disposed on the first pad and the second pad of the driving backplane and electrically connected to the first pad and the second pad respectively. A volume of the first solder is larger than a volume of the second solder, and an area of the first pad is smaller than an area of the second pad.
    Type: Application
    Filed: December 13, 2023
    Publication date: October 24, 2024
    Inventors: Chia-Hui Pai, Wen-Hsien Tseng, Chien-Hung Kuo, Wei-Chieh Chen, Kuan-Yi Lee, Chih-Chun Yang
  • Publication number: 20240355960
    Abstract: A light emitting element substrate includes a substrate and a light emitting element disposed on the substrate and including first and second semiconductor layers, an active layer, first and second electrodes, and first and second solders. The second semiconductor layer is disposed opposite to the first semiconductor layer. The active layer is disposed between the first and second semiconductor layers. The first and second electrodes are electrically connected to the first and second semiconductor layers respectively. The first and second solders are respectively disposed on and electrically connected to the first and second electrodes respectively. The first electrode includes a first under barrier pattern. The first solder covers the first under barrier pattern. A projection area of the first solder on the substrate is greater than a projection area of the first under barrier pattern on the substrate. A display apparatus is provided.
    Type: Application
    Filed: November 9, 2023
    Publication date: October 24, 2024
    Applicant: AUO Corporation
    Inventors: Chia-Hui Pai, Wen-Hsien Tseng, Chien-Hung Kuo, Wei-Chieh Chen, Kuan-Yi Lee, Chih-Chun Yang
  • Patent number: 12112551
    Abstract: In one embodiment, a signal light state detection system includes one or more processors, an a non-transitory memory module storing computer-readable instructions. The computer-readable instructions are configured to cause the one or more processors to receive a first image of a vehicle and receiving a second image of the vehicle, wherein the second image is later in time from the first image, and generate a warped image from the first image and the second image, wherein the warped image has individual pixels of one of the first image and the second image that are shifted to locations of corresponding pixels of the other of the first image and the second image. The one or more processors further generate a difference image from the warped image and one of the first image and the second image, and determine, using a classifier module, a probability of a state of vehicle signal lights.
    Type: Grant
    Filed: February 9, 2022
    Date of Patent: October 8, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Naoki Nagasaka, Blake Wulfe, Kuan-Hui Lee, Jia-En Marcus Pan
  • Publication number: 20240319529
    Abstract: An electronic device includes: a first panel, a second panel, and a first adhesive member. The first panel includes a fist alignment mark. The second panel is disposed on the first panel, and includes a second alignment mark. The first adhesive member is disposed between the first panel and the second panel, and in a top view of the electronic device, a first distance is between the first adhesive member and the first alignment mark, a second distance is between the first adhesive member and the second alignment mark, the first distance is greater than zero, and the second distance is greater than zero.
    Type: Application
    Filed: February 21, 2024
    Publication date: September 26, 2024
    Inventors: Yi-Hui LEE, Kuan-Chou CHEN
  • Patent number: 12087063
    Abstract: System, methods, and other embodiments described herein relate to detection of traffic lights corresponding to a driving lane from views captured by multiple cameras. In one embodiment, a method includes estimating, by a first model using images from multiple cameras, positions and state confidences of traffic lights corresponding to a driving lane of a vehicle. The method also includes aggregating, by a second model, the state confidences and a multi-view stereo composition from geometric representations associated with the positions of the traffic lights. The method also includes assigning, by the second model according to the aggregating, a relevancy score computed for a candidate traffic light of the traffic lights to the driving lane. The method also includes executing a task by the vehicle according to the relevancy score.
    Type: Grant
    Filed: April 28, 2022
    Date of Patent: September 10, 2024
    Assignees: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Kun-Hsin Chen, Kuan-Hui Lee, Chao Fang, Charles Christopher Ochoa
  • Patent number: 12080161
    Abstract: A method for vehicle prediction, planning, and control is described. The method includes separately encoding traffic state information at an intersection into corresponding traffic state latent spaces. The method also includes aggregating the corresponding traffic state latent spaces to form a generalized traffic geometry latent space. The method further includes interpreting the generalized traffic geometry latent space to form a traffic flow map including current and future vehicle trajectories. The method also includes decoding the generalized traffic geometry latent space to predict a vehicle behavior according to the traffic flow map based on the current and future vehicle trajectories.
    Type: Grant
    Filed: April 28, 2022
    Date of Patent: September 3, 2024
    Assignees: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA
    Inventors: Kuan-Hui Lee, Charles Christopher Ochoa, Arjun Bhargava, Chao Fang, Kun-Hsin Chen
  • Patent number: 12073633
    Abstract: System, methods, and other embodiments described herein relate to improving the detection of traffic lights associated with a driving lane using a camera instead of map data. In one embodiment, a method includes estimating, from an image using a first model, depth and orientation information of traffic lights relative to a driving lane of a vehicle. The method also includes computing, using a second model, relevancy scores for the traffic lights according to geometric inferences between the depth and the orientation information. The method also includes assigning, using the second model, a primary relevancy score for a light of the traffic lights associated with the driving lane according to the depth and the orientation information. The method also includes executing a control task by the vehicle according to the primary relevancy score and a state confidence, computed by the first model, for the light.
    Type: Grant
    Filed: April 22, 2022
    Date of Patent: August 27, 2024
    Assignees: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Kun-Hsin Chen, Kuan-Hui Lee, Chao Fang, Charles Christopher Ochoa
  • Publication number: 20240251524
    Abstract: An immersion cooling system is provided and includes a storage tank, an adapter frame, and an electronic device. The storage tank includes an accommodating tank and a sealing cover for sealing the opening of the accommodating tank. The adapter frame includes a first wire channel attached to the side of the adapter frame, and is disposed in the accommodating tank in a vertical direction. The electronic device includes at least one power wire, and is inserted into the adapter frame along the above direction. The side of the electronic device is attached to the side of the first wire channel. A second wire channel is formed between the bottoms of the electronic device and the adapter frame, and the first and second wire channels communicate with each other. The power wire is connected to the circuit of the external power apparatus along the first and second wire channels.
    Type: Application
    Filed: November 16, 2023
    Publication date: July 25, 2024
    Inventors: Hsuan-Ting LIU, Yan-Hui JIAN, Chia-Hsing CHEN, Chen-Hsiu LEE, Kuan-Lung WU
  • Patent number: 12046050
    Abstract: System, methods, and other embodiments described herein relate to accurately distinguishing a traffic light from other illuminated objects in the traffic scene and detecting states using hierarchical modeling. In one embodiment, a method includes detecting, using a machine learning (ML) model, two-dimensional (2D) coordinates of illuminated objects identified from a monocular image of a traffic scene for control adaptation by a control model. The method also includes assigning, using the ML model, computed probabilities to the illuminated objects for categories within a hierarchical ontology of environmental lights associated with the traffic scene, wherein one of the probabilities indicates existence of a traffic light instead of a brake light in the traffic scene. The method also includes executing a task by the control model for a vehicle according to the 2D coordinates and the computed probabilities.
    Type: Grant
    Filed: April 15, 2022
    Date of Patent: July 23, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Kun-Hsin Chen, Kuan-Hui Lee, Chao Fang, Charles Christopher Ochoa
  • Patent number: 12039438
    Abstract: Systems, methods, and other embodiments described herein relate to improving trajectory forecasting in a device. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying an object from the sensor data that is present in the surrounding environment. The method includes determining category probabilities for the object, the category probabilities indicating semantic classes for classifying the object and probabilities that the object belongs to the semantic classes. The method includes forecasting trajectories for the object based, at least in part, on the category probabilities and the sensor data. The method includes controlling the device according to the trajectories.
    Type: Grant
    Filed: December 4, 2020
    Date of Patent: July 16, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Boris Ivanovic, Kuan-Hui Lee, Jie Li, Adrien David Gaidon, Pavel Tokmakov
  • Patent number: 12014549
    Abstract: A vehicle light classification system captures a sequence of images of a scene that includes a front/rear view of a vehicle with front/rear-side lights, determines semantic keypoints, in the images and associated with the front/rear-side lights, based on inputting the images into a first neural network, obtains multiple difference images that are each a difference between successive images from among the sequence of images, the successive images being aligned based on their respective semantic keypoints, and determines a classification of the front/rear-side lights based at least in part on the difference images by inputting the difference images into a second neural network.
    Type: Grant
    Filed: March 4, 2021
    Date of Patent: June 18, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Jia-En Pan, Kuan-Hui Lee, Chao Fang, Kun-Hsin Chen, Arjun Bhargava, Sudeep Pillai
  • Patent number: 11948310
    Abstract: Systems and methods described herein relate to jointly training a machine-learning-based monocular optical flow, depth, and scene flow estimator. One embodiment processes a pair of temporally adjacent monocular image frames using a first neural network structure to produce a first optical flow estimate; processes the pair of temporally adjacent monocular image frames using a second neural network structure to produce an estimated depth map and an estimated scene flow; processes the estimated depth map and the estimated scene flow using the second neural network structure to produce a second optical flow estimate; and imposes a consistency loss between the first optical flow estimate and the second optical flow estimate that minimizes a difference between the first optical flow estimate and the second optical flow estimate to improve performance of the first neural network structure in estimating optical flow and the second neural network structure in estimating depth and scene flow.
    Type: Grant
    Filed: September 29, 2021
    Date of Patent: April 2, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Vitor Guizilini, Rares A. Ambrus, Kuan-Hui Lee, Adrien David Gaidon
  • Patent number: 11948309
    Abstract: Systems and methods described herein relate to jointly training a machine-learning-based monocular optical flow, depth, and scene flow estimator. One embodiment processes a pair of temporally adjacent monocular image frames using a first neural network structure to produce an optical flow estimate and to extract, from at least one image frame in the pair of temporally adjacent monocular image frames, a set of encoded image context features; triangulates the optical flow estimate to generate a depth map; extracts a set of encoded depth context features from the depth map using a depth context encoder; and combines the set of encoded image context features and the set of encoded depth context features to improve performance of a second neural network structure in estimating depth and scene flow.
    Type: Grant
    Filed: September 29, 2021
    Date of Patent: April 2, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Vitor Guizilini, Rares A. Ambrus, Kuan-Hui Lee, Adrien David Gaidon
  • Patent number: 11922640
    Abstract: A method for 3D object tracking is described. The method includes inferring first 2D semantic keypoints of a 3D object within a sparsely annotated video stream. The method also includes matching the first 2D semantic keypoints of a current frame with second 2D semantic keypoints in a next frame of the sparsely annotated video stream using embedded descriptors within the current frame and the next frame. The method further includes warping the first 2D semantic keypoints to the second 2D semantic keypoints to form warped 2D semantic keypoints in the next frame. The method also includes labeling a 3D bounding box in the next frame according to the warped 2D semantic keypoints in the next frame.
    Type: Grant
    Filed: March 8, 2021
    Date of Patent: March 5, 2024
    Assignee: TOYOTA RESEARCH INSTITUTE, INC.
    Inventors: Arjun Bhargava, Sudeep Pillai, Kuan-Hui Lee
  • Patent number: 11878684
    Abstract: A system for trajectory prediction using a predicted endpoint conditioned network includes one or more processors and a memory that includes a sensor input module, an endpoint distribution module, and a future trajectory module. The modules cause the one or more processors to the one or more processors to obtain sensor data of a scene having a plurality of pedestrians, determine endpoint distributions of the plurality of pedestrians within the scene, the endpoint distributions representing desired end destinations of the plurality of pedestrians from the scene, and determine future trajectory points for at least one of the plurality of pedestrians based on prior trajectory points of the plurality of pedestrians and the endpoint distributions of the plurality of pedestrians. The future trajectory points may be conditioned not only on the pedestrian and their immediate neighbors' histories (observed trajectories) but also on all the other pedestrian's estimated endpoints.
    Type: Grant
    Filed: September 30, 2020
    Date of Patent: January 23, 2024
    Assignee: Toyota Research Institute, Inc.
    Inventors: Karttikeya Mangalam, Kuan-Hui Lee, Adrien David Gaidon
  • Patent number: 11810367
    Abstract: Described herein are systems and methods for determining if a vehicle is parked. In one example, a system includes a processor, a sensor system, and a memory. Both the sensor system and the memory are in communication with the processor. The memory includes a parking determination module having instructions that, when executed by the processor, cause the processor to determine, using a random forest model, when the vehicle is parked based on vehicle estimated features, vehicle learned features, and vehicle taillight features of the vehicle that are based on sensor data from the sensor system.
    Type: Grant
    Filed: June 29, 2021
    Date of Patent: November 7, 2023
    Assignee: Toyota Research Institute, Inc.
    Inventors: Chao Fang, Kuan-Hui Lee, Logan Michael Ellis, Jia-En Pan, Kun-Hsin Chen, Sudeep Pillai, Daniele Molinari, Constantin Franziskus Dominik Hubmann, T. Wolfram Burgard
  • Publication number: 20230351886
    Abstract: A method for vehicle prediction, planning, and control is described. The method includes separately encoding traffic state information at an intersection into corresponding traffic state latent spaces. The method also includes aggregating the corresponding traffic state latent spaces to form a generalized traffic geometry latent space. The method further includes interpreting the generalized traffic geometry latent space to form a traffic flow map including current and future vehicle trajectories. The method also includes decoding the generalized traffic geometry latent space to predict a vehicle behavior according to the traffic flow map based on the current and future vehicle trajectories.
    Type: Application
    Filed: April 28, 2022
    Publication date: November 2, 2023
    Applicants: TOYOTA RESEARCH INSTITUTE, INC., TOYOTA JIDOSHA KABUSHIKI KAISHA
    Inventors: Kuan-Hui LEE, Charles Christopher OCHOA, Arjun BHARGAVA, Chao FANG, Kun-Hsin CHEN
  • Publication number: 20230351739
    Abstract: Systems, methods, and other embodiments described herein relate to a multi-task model that integrates recurrent models to improve handling of multi-sweep inputs. In one embodiment, a method includes acquiring sensor data from multiple modalities. The method includes separately encoding respective segments of the sensor data according to an associated one of the different modalities to form encoded features using separate encoders of a network. The method includes accumulating, in a detector, sparse features associated with sparse sensor inputs of the multiple modalities to densify the sparse features into dense features. The method includes providing observations according to the encoded features and the sparse features using the network.
    Type: Application
    Filed: April 29, 2022
    Publication date: November 2, 2023
    Applicants: Toyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
    Inventors: Kuan-Hui Lee, Charles Christopher Ochoa, Arjun Bhargava, Chao Fang, Kun-Hsin Chen