Patents by Inventor Kratarth Goel
Kratarth Goel has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11893750Abstract: A machine-learning (ML) architecture for determining three or more outputs, such as a two and/or three-dimensional region of interest, semantic segmentation, direction logits, depth data, and/or instance segmentation associated with an object in an image. The ML architecture may output these outputs at a rate of 30 or more frames per second on consumer grade hardware.Type: GrantFiled: December 31, 2019Date of Patent: February 6, 2024Assignee: ZOOX, INC.Inventors: Kratarth Goel, James William Vaisey Philbin, Praveen Srinivasan, Sarah Tariq
-
Publication number: 20230406360Abstract: Methods, systems, and apparatus for generating trajectory predictions for one or more target agents. In one aspect, a system comprises one or more computers configured to obtain scene context data characterizing a scene in an environment at a current time point, where the scene includes multiple agents that include a target agent and one or more context agents, and the scene context data includes respective context data for each of multiple different modalities of context data. The one or more computers then generate an encoded representation of the scene in the environment that includes one or more embeddings and process the encoded representation of the scene context data using a decoder neural network to generate a trajectory prediction output for the target agent that predicts a future trajectory of the target after the current time point.Type: ApplicationFiled: June 15, 2023Publication date: December 21, 2023Inventors: Rami Al-Rfou, Nigamaa Nayakanti, Kratarth Goel, Aurick Qikun Zhou, Benjamin Sapp, Khaled Refaat
-
Publication number: 20230266771Abstract: Techniques for utilizing multiple scales of images as input to machine learning (ML) models are discussed herein. Operations can include providing an image associated with a first scale to a first ML model. An output of the first ML model can include a first bounding box indicative of a first region of the image representing a first object, with the first bounding box falling within a first range of sizes. Next, a scaled image can be generated by scaling the image. The scaled image can be provided to a second ML model, which can output a second bounding box indicative of a second region of the image representing a second object, the second bounding falling within a second range of sizes. Thus, inputting a scaled image to a same ML model (or to different ML models) can result in different detected features in the images.Type: ApplicationFiled: February 27, 2023Publication date: August 24, 2023Inventors: Sarah Tariq, Kratarth Goel, James William Vaisey Philbin
-
Patent number: 11710352Abstract: Techniques for detecting attributes and/or gestures associated with pedestrians in an environment are described herein. The techniques may include receiving sensor data associated with a pedestrian in an environment of a vehicle and inputting the sensor data into a machine-learned model that is configured to determine a gesture and/or an attribute of the pedestrian. Based on the input data, an output may be received from the machine-learned model that indicates the gesture and/or the attribute of the pedestrian and the vehicle may be controlled based at least in part on the gesture and/or the attribute of the pedestrian. The techniques may also include training the machine-learned model to detect the attribute and/or the gesture of the pedestrian.Type: GrantFiled: May 14, 2021Date of Patent: July 25, 2023Assignee: Zoox, Inc.Inventors: Oytun Ulutan, Xin Wang, Kratarth Goel, Vasiliy Karasev, Sarah Tariq, Yi Xu
-
Patent number: 11681046Abstract: Techniques for training a machine learned (ML) model to determine depth data based on image data are discussed herein. Training can use stereo image data and depth data (e.g., lidar data). A first (e.g., left) image can be input to a ML model, which can output predicted disparity and/or depth data. The predicted disparity data can be used with second image data (e.g., a right image) to reconstruct the first image. Differences between the first and reconstructed images can be used to determine a loss. Losses may include pixel, smoothing, structural similarity, and/or consistency losses. Further, differences between the depth data and the predicted depth data and/or differences between the predicted disparity data and the predicted depth data can be determined, and the ML model can be trained based on the various losses. Thus, the techniques can use self-supervised training and supervised training to train a ML model.Type: GrantFiled: October 22, 2021Date of Patent: June 20, 2023Assignee: Zoox, Inc.Inventors: Thomas Oscar Dudzik, Kratarth Goel, Praveen Srinivasan, Sarah Tariq
-
Publication number: 20230091924Abstract: Techniques for utilizing a depth completion algorithm to determine dense depth data are discussed are discussed herein. Two-dimensional image data representing an environment can be captured or otherwise received. Depth data representing the environment can be captured or otherwise received. The depth data can be projected into the image data and processed using the depth completion algorithm. The depth completion algorithm can be utilized to determine the dense depth values based on intensity values of pixels, and other depth values. A vehicle can be controlled based on the determined depth values.Type: ApplicationFiled: September 17, 2021Publication date: March 23, 2023Inventors: Jonathan Tyler Dowdall, Kratarth Goel, Adam Edward Pollack, Scott M. Purdy, Bharadwaj Raghavan
-
Patent number: 11610078Abstract: Using detection of low variance regions for improving detection is described. In an example, sensor data can be received from a sensor associated with a vehicle. The sensor data can represent an environment. An indication of a low variance region associated with the sensor data can be determined and an indication of a high variance region associated with the sensor data can be determined based at least in part on the indication of the low variance region. The vehicle can be controlled based on at least one of the sensor data or the indication of the high variance region.Type: GrantFiled: December 6, 2019Date of Patent: March 21, 2023Assignee: Zoox, Inc.Inventors: Kratarth Goel, James William Vaisey Philbin, Sarah Tariq
-
Patent number: 11605236Abstract: Low variance detection training is described herein. In an example, annotated data can be determined based on sensor data received from a sensor associated with a vehicle. The annotated data can comprise an annotated low variance region and/or an annotated high variance region. The sensor data can be input into a model, and the model can determine an output comprising a high variance output and a low variance output. In an example, a difference between the annotated data and the output can be determined and one or more parameters associated with the model can be altered based at least in part on the difference. The model can be transmitted to a vehicle configured to be controlled by another output of the model.Type: GrantFiled: December 6, 2019Date of Patent: March 14, 2023Assignee: Zoox, Inc.Inventors: Kratarth Goel, James William Vaisey Philbin, Sarah Tariq
-
Patent number: 11592818Abstract: Techniques for utilizing multiple scales of images as input to machine learning (ML) models are discussed herein. Operations can include providing an image associated with a first scale to a first ML model. An output of the first ML model can include a first bounding box indicative of a first region of the image representing a first object, with the first bounding box falling within a first range of sizes. Next, a scaled image can be generated by scaling the image. The scaled image can be provided to a second ML model, which can output a second bounding box indicative of a second region of the image representing a second object, the second bounding falling within a second range of sizes. Thus, inputting a scaled image to a same ML model (or to different ML models) can result in different detected features in the images.Type: GrantFiled: June 20, 2018Date of Patent: February 28, 2023Assignee: Zoox, Inc.Inventors: Sarah Tariq, James William Vaisey Philbin, Kratarth Goel
-
Publication number: 20230029900Abstract: Techniques are discussed herein for generating three-dimensional (3D) representations of an environment based on two-dimensional (2D) image data, and using the 3D representations to perform 3D object detection and other 3D analyses of the environment. 2D image data may be received, along with depth estimation data associated with the 2D image data. Using the 2D image data and associated depth data, an image-based object detector may generate 3D representations, including point clouds and/or 3D pixel grids, for the 2D image or particular regions of interest. In some examples, a 3D point cloud may be generated by projecting pixels from the 2D image into 3D space followed by a trained 3D convolutional neural network (CNN) performing object detection. Additionally or alternatively, a top-down view of a 3D pixel grid representation may be used to perform object detection using 2D convolutions.Type: ApplicationFiled: July 30, 2021Publication date: February 2, 2023Inventor: Kratarth Goel
-
Publication number: 20230033177Abstract: Techniques are discussed herein for generating three-dimensional (3D) representations of an environment based on two-dimensional (2D) image data, and using the 3D representations to perform 3D object detection and other 3D analyses of the environment. 2D image data may be received, along with depth estimation data associated with the 2D image data. Using the 2D image data and associated depth data, an image-based object detector may generate 3D representations, including point clouds and/or 3D pixel grids, for the 2D image or particular regions of interest. In some examples, a 3D point cloud may be generated by projecting pixels from the 2D image into 3D space followed by a trained 3D convolutional neural network (CNN) performing object detection. Additionally or alternatively, a top-down view of a 3D pixel grid representation may be used to perform object detection using 2D convolutions.Type: ApplicationFiled: July 30, 2021Publication date: February 2, 2023Inventor: Kratarth Goel
-
Patent number: 11450117Abstract: The techniques discussed herein may comprise refining a classification of an object detected as being represented in sensor data. For example, refining the classification may comprise determining a sub-classification of the object.Type: GrantFiled: March 29, 2021Date of Patent: September 20, 2022Assignee: Zoox, Inc.Inventors: Kratarth Goel, Sarah Tariq
-
Publication number: 20220114395Abstract: Techniques for training a machine learned (ML) model to determine depth data based on image data are discussed herein. Training can use stereo image data and depth data (e.g., lidar data). A first (e.g., left) image can be input to a ML model, which can output predicted disparity and/or depth data. The predicted disparity data can be used with second image data (e.g., a right image) to reconstruct the first image. Differences between the first and reconstructed images can be used to determine a loss. Losses may include pixel, smoothing, structural similarity, and/or consistency losses. Further, differences between the depth data and the predicted depth data and/or differences between the predicted disparity data and the predicted depth data can be determined, and the ML model can be trained based on the various losses. Thus, the techniques can use self-supervised training and supervised training to train a ML model.Type: ApplicationFiled: October 22, 2021Publication date: April 14, 2022Inventors: Thomas Oscar Dudzik, Kratarth Goel, Praveen Srinivasan, Sarah Tariq
-
Patent number: 11163990Abstract: Techniques described herein relate to using head detection to improve pedestrian detection. In an example, a head can be detected in sensor data received from a sensor associated with a vehicle using a machine learned model. Based at least partly on detecting the head in the sensor data, a pedestrian can be determined to be present in an environment within which the vehicle is positioned. In an example, an indication of the pedestrian can be provided to at least one system of the vehicle, for instance, for use by the at least one system to make a determination associated with controlling the vehicle.Type: GrantFiled: June 28, 2019Date of Patent: November 2, 2021Assignee: Zoox, Inc.Inventor: Kratarth Goel
-
Patent number: 11157774Abstract: Techniques for training a machine learned (ML) model to determine depth data based on image data are discussed herein. Training can use stereo image data and depth data (e.g., lidar data). A first (e.g., left) image can be input to a ML model, which can output predicted disparity and/or depth data. The predicted disparity data can be used with second image data (e.g., a right image) to reconstruct the first image. Differences between the first and reconstructed images can be used to determine a loss. Losses may include pixel, smoothing, structural similarity, and/or consistency losses. Further, differences between the depth data and the predicted depth data and/or differences between the predicted disparity data and the predicted depth data can be determined, and the ML model can be trained based on the various losses. Thus, the techniques can use self-supervised training and supervised training to train a ML model.Type: GrantFiled: November 14, 2019Date of Patent: October 26, 2021Assignee: Zoox, Inc.Inventors: Thomas Oscar Dudzik, Kratarth Goel, Praveen Srinivasan, Sarah Tariq
-
Publication number: 20210216793Abstract: The techniques discussed herein may comprise refining a classification of an object detected as being represented in sensor data. For example, refining the classification may comprise determining a sub-classification of the object.Type: ApplicationFiled: March 29, 2021Publication date: July 15, 2021Inventors: Kratarth Goel, Sarah Tariq
-
Publication number: 20210181757Abstract: A machine-learning (ML) architecture for determining three or more outputs, such as a two and/or three-dimensional region of interest, semantic segmentation, direction logits, depth data, and/or instance segmentation associated with an object in an image. The ML architecture may output these outputs at a rate of 30 or more frames per second on consumer grade hardware.Type: ApplicationFiled: December 31, 2019Publication date: June 17, 2021Inventors: Kratarth Goel, James William Vaisey Philbin, Praveen Srinivasan, Sarah Tariq
-
Publication number: 20210150279Abstract: Techniques for training a machine learned (ML) model to determine depth data based on image data are discussed herein. Training can use stereo image data and depth data (e.g., lidar data). A first (e.g., left) image can be input to a ML model, which can output predicted disparity and/or depth data. The predicted disparity data can be used with second image data (e.g., a right image) to reconstruct the first image. Differences between the first and reconstructed images can be used to determine a loss. Losses may include pixel, smoothing, structural similarity, and/or consistency losses. Further, differences between the depth data and the predicted depth data and/or differences between the predicted disparity data and the predicted depth data can be determined, and the ML model can be trained based on the various losses. Thus, the techniques can use self-supervised training and supervised training to train a ML model.Type: ApplicationFiled: November 14, 2019Publication date: May 20, 2021Inventors: Thomas Oscar Dudzik, Kratarth Goel, Praveen Srinivasan, Sarah Tariq
-
Publication number: 20210150278Abstract: Techniques for training a machine learned (ML) model to determine depth data based on image data are discussed herein. Training can use stereo image data and depth data (e.g., lidar data). A first (e.g., left) image can be input to a ML model, which can output predicted disparity and/or depth data. The predicted disparity data can be used with second image data (e.g., a right image) to reconstruct the first image. Differences between the first and reconstructed images can be used to determine a loss. Losses may include pixel, smoothing, structural similarity, and/or consistency losses. Further, differences between the depth data and the predicted depth data and/or differences between the predicted disparity data and the predicted depth data can be determined, and the ML model can be trained based on the various losses. Thus, the techniques can use self-supervised training and supervised training to train a ML model.Type: ApplicationFiled: November 14, 2019Publication date: May 20, 2021Inventors: Thomas Oscar Dudzik, Kratarth Goel, Praveen Srinivasan, Sarah Tariq
-
Patent number: 10984290Abstract: Training a machine-learning (ML) architecture to determine three or more outputs at a rate of 30 or more frames per second on consumer grade hardware may comprise jointly training components of the ML using loss(es) determined across the components and/or consistency losses determined between outputs of two or more components. The ML architecture discussed herein may comprise one or more sets of neural network layers and/or respective components for determining a two and/or three-dimensional region of interest, semantic segmentation, direction logits, depth data, and/or instance segmentation associated with an object in an image.Type: GrantFiled: December 31, 2019Date of Patent: April 20, 2021Assignee: Zoox, Inc.Inventors: Kratarth Goel, James William Vaisey Philbin, Praveen Srinivasan, Sarah Tariq