Patents by Inventor Juan Carlos Niebles
Juan Carlos Niebles has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240104809Abstract: Embodiments described herein provide systems and methods for multimodal layout generations for digital publications. The system may receive as inputs, a background image, one or more foreground texts, and one or more foreground images. Feature representations of the background image may be generated. The foreground inputs may be input to a layout generator which has cross attention to the background image feature representations in order to generate a layout comprising of bounding box parameters for each input item. A composite layout may be generated based on the inputs and generated bounding boxes. The resulting composite layout may then be displayed on a user interface.Type: ApplicationFiled: January 30, 2023Publication date: March 28, 2024Inventors: Ning Yu, Chia-Chih Chen, Zeyuan Chen, Caiming Xiong, Juan Carlos Niebles Duque, Ran Xu, Rui Meng
-
Patent number: 11918370Abstract: Many embodiments of the invention include systems and methods for evaluating motion from a video, the method includes identifying a target individual in a set of one or more frames in a video, analyzing the set of frames to determine a set of pose parameters, generating a 3D body mesh based on the pose parameters, identifying joint positions for the target individual in the set of frames based on the generated 3D body mesh, predicting a motion evaluation score based on the identified join positions, providing an output based on the motion evaluation score.Type: GrantFiled: May 19, 2021Date of Patent: March 5, 2024Assignee: The Board of Trustees of the Leland Stanford Junior UniversityInventors: Ehsan Adeli-Mosabbeb, Mandy Lu, Kathleen Poston, Juan Carlos Niebles
-
Publication number: 20240070868Abstract: Embodiments described herein provide an open-vocabulary instance segmentation framework that adopts a pre-trained vision-language model to develop a pipeline in detecting novel categories of instances.Type: ApplicationFiled: January 25, 2023Publication date: February 29, 2024Inventors: Ning Yu, Vibashan Vishnukumar Sharmini, Chen Xing, Juan Carlos Niebles Duque, Ran Xu
-
Publication number: 20240021018Abstract: Systems and methods of capturing privacy protected images and performing machine vision tasks are described. An embodiment includes a system that includes an optical component and an image processing application configured to capture distorted video using the optical component, where the optical component includes a set of optimal camera lens parameters ?*o learned using machine learning, performing a machine vision task on the distorted video, where the machine vision task includes a set of optimal action recognition parameters ?*c learned using the machine learning, and generating a classification based on the machine vision task, where the machine learning is jointly trained to optimize the optical element and the machine vision task.Type: ApplicationFiled: July 13, 2023Publication date: January 18, 2024Applicants: The Board of Trustees of the Leland Stanford Junior University, Universidad Industrial de SantanderInventors: Juan Carlos Niebles, Carlos Hinojosa, Henry Arguello, Miguel Marquez, Ehsan Adeli-Mosabbeb, Fei-Fei Li
-
Publication number: 20230154139Abstract: Embodiments described herein provide an intelligent method to select instances, by utilizing unsupervised tracking for videos. Using this freely available form of supervision, a temporal constraint is adopted for selecting instances that ensures that different instances contain the same object while sampling the temporal augmentation from the video. In addition, using the information on the spatial extent of the tracked object, spatial constraints are applied to ensure that sampled instances overlap meaningfully with the tracked object. Taken together, these spatiotemporal constraints result in better supervisory signal for contrastive learning from videos.Type: ApplicationFiled: January 31, 2022Publication date: May 18, 2023Inventors: Brian Chen, Ramprasaath Ramasamy Selvaraju, Juan Carlos Niebles Duque, Nikhil Naik
-
Patent number: 11604936Abstract: A method for scene perception using video captioning based on a spatio-temporal graph model is described. The method includes decomposing the spatio-temporal graph model of a scene in input video into a spatial graph and a temporal graph. The method also includes modeling a two branch framework having an object branch and a scene branch according to the spatial graph and the temporal graph to learn object interactions between the object branch and the scene branch. The method further includes transferring the learned object interactions from the object branch to the scene branch as privileged information. The method also includes captioning the scene by aligning language logits from the object branch and the scene branch according to the learned object interactions.Type: GrantFiled: March 23, 2020Date of Patent: March 14, 2023Assignees: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Boxiao Pan, Haoye Cai, De-An Huang, Kuan-Hui Lee, Adrien David Gaidon, Ehsan Adeli-Mosabbeb, Juan Carlos Niebles Duque
-
Publication number: 20220180101Abstract: Systems and methods for multi-view cooperative contrastive self-supervised learning, may include receiving a plurality of video sequences, the video sequences comprising a plurality of image frames; applying selected images of a first and second video sequence of the plurality of video sequences to a plurality of different encoders to derive a plurality of embeddings for different views of the selected images of the first and second video sequences; determining distances of the derived plurality of embeddings for the selected images of the first and second video sequences; detecting inconsistencies in the determined distances; and predicting semantics of a future image based on the determined distances.Type: ApplicationFiled: December 3, 2020Publication date: June 9, 2022Inventors: Nishant Rai, Ehsan Adeli Mosabbeb, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
-
Patent number: 11205082Abstract: A system and method for predicting pedestrian intent is provided. A prediction circuit comprising a plurality of gated recurrent units (GRUB) receives a sequence of images captured by a camera. The prediction circuit parses each frame of the sequence of images to identify one or more pedestrians and one or more objects. Using the parsed data, the prediction circuit generates a pedestrian-centric spatiotemporal graph, the parsed data comprising one or more identified pedestrians and one or more identified object. The prediction circuit uses the pedestrian-centric graph to determine a probability of one or more pedestrians crossing a street for each frame of the sequence of images.Type: GrantFiled: October 8, 2019Date of Patent: December 21, 2021Assignees: TOYOTA RESEARCH INSTITUTE, INC., The Board of Trustees of the Leland Stanford Junior UniversityInventors: Ehsan Adeli-Mosabbeb, Kuan Lee, Adrien Gaidon, Bingbin Liu, Zhangjie Cao, Juan Carlos Niebles
-
Publication number: 20210386359Abstract: Many embodiments of the invention include systems and methods for evaluating motion from a video, the method includes identifying a target individual in a set of one or more frames in a video, analyzing the set of frames to determine a set of pose parameters, generating a 3D body mesh based on the pose parameters, identifying joint positions for the target individual in the set of frames based on the generated 3D body mesh, predicting a motion evaluation score based on the identified join positions, providing an output based on the motion evaluation score.Type: ApplicationFiled: May 19, 2021Publication date: December 16, 2021Applicant: The Board of Trustees of the Leland Stanford Junior UniversityInventors: Ehsan Adeli-Mosabbeb, Mandy Lu, Kathleen Poston, Juan Carlos Niebles
-
Publication number: 20210295093Abstract: A method for scene perception using video captioning based on a spatio-temporal graph model is described. The method includes decomposing the spatio-temporal graph model of a scene in input video into a spatial graph and a temporal graph. The method also includes modeling a two branch framework having an object branch and a scene branch according to the spatial graph and the temporal graph to learn object interactions between the object branch and the scene branch. The method further includes transferring the learned object interactions from the object branch to the scene branch as privileged information. The method also includes captioning the scene by aligning language logits from the object branch and the scene branch according to the learned object interactions.Type: ApplicationFiled: March 23, 2020Publication date: September 23, 2021Applicants: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Boxiao PAN, Haoye CAI, De-An HUANG, Kuan-Hui LEE, Adrien David GAIDON, Ehsan ADELI-MOSABBEB, Juan Carlos NIEBLES DUQUE
-
Patent number: 11074438Abstract: A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting is described. The method includes generating a frame-level supervision for human poses. The method also includes suppressing noise and filling missing joints of the human body using a pose completion module. The method further includes splitting the poses into a global stream and a local stream. Furthermore, the method includes combining the global stream and the local stream to forecast future human locomotion.Type: GrantFiled: October 1, 2019Date of Patent: July 27, 2021Assignees: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Karttikeya Mangalam, Ehsan Adeli-Mosabbeb, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles Duque
-
Publication number: 20210103742Abstract: A system and method for predicting pedestrian intent is provided. A prediction circuit comprising a plurality of gated recurrent units (GRUB) receives a sequence of images captured by a camera. The prediction circuit parses each frame of the sequence of images to identify one or more pedestrians and one or more objects. Using the parsed data, the prediction circuit generates a pedestrian-centric spatiotemporal graph, the parsed data comprising one or more identified pedestrians and one or more identified object. The prediction circuit uses the pedestrian-centric graph to determine a probability of one or more pedestrians crossing a street for each frame of the sequence of images.Type: ApplicationFiled: October 8, 2019Publication date: April 8, 2021Inventors: Ehsan Adeli-Mosabbeb, Kuan Lee, Adrien Gaidon, Bingbin Liu, Zhangjie Cao, Juan Carlos Niebles
-
Publication number: 20210097266Abstract: A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting is described. The method includes generating a frame-level supervision for human poses. The method also includes suppressing noise and filling missing joints of the human body using a pose completion module. The method further includes splitting the poses into a global stream and a local stream. Furthermore, the method includes combining the global stream and the local stream to forecast future human locomotion.Type: ApplicationFiled: October 1, 2019Publication date: April 1, 2021Applicants: TOYOTA RESEARCH INSTITUTE, INC., THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITYInventors: Karttikeya MANGALAM, Ehsan ADELI-MOSABBEB, Kuan-Hui LEE, Adrien GAIDON, Juan Carlos NIEBLES DUQUE
-
Patent number: 10210462Abstract: A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.Type: GrantFiled: November 24, 2014Date of Patent: February 19, 2019Assignee: Google LLCInventors: Juan Carlos Niebles Duque, Hrishikesh Aradhye, Luciano Sbaiz, Jay Yagnik, Reto Strobl
-
Publication number: 20150081604Abstract: A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.Type: ApplicationFiled: November 24, 2014Publication date: March 19, 2015Inventors: Juan Carlos Niebles Duque, Hrishikesh Aradhye, Luciano Sbaiz, Jay Yagnik, Reto Strobl
-
Patent number: 8924993Abstract: A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.Type: GrantFiled: November 10, 2011Date of Patent: December 30, 2014Assignee: Google Inc.Inventors: Juan Carlos Niebles Duque, Hrishikesh Balkrishna Aradhye, Luciano Sbaiz, Jay N. Yagnik, Reto Strobl