Patents by Inventor Tomas Izo
Tomas Izo has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20210166035Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.Type: ApplicationFiled: December 14, 2020Publication date: June 3, 2021Inventors: Sanketh Shetty, Tomas Izo, Min-Hsuan Tsai, Sudheendra Vijayanarasimhan, Apostol Natsev, Sami Abu-El-Haija, George Dan Toderici, Susana Ricco, Balakrishnan Varadarajan, Nicola Muscettola, WeiHsin Gu, Weilong Yang, Nitin Khandelwal, Phuong Le
-
Patent number: 10867183Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.Type: GrantFiled: April 23, 2018Date of Patent: December 15, 2020Assignee: Google LLCInventors: Sanketh Shetty, Tomas Izo, Min-Hsuan Tsai, Sudheendra Vijayanarasimhan, Apostol Natsev, Sami Abu-El-Haija, George Dan Toderici, Susanna Ricco, Balakrishnan Varadarajan, Nicola Muscettola, WeiHsin Gu, Weilong Yang, Nitin Khandelwal, Phuong Le
-
Patent number: 10777229Abstract: Frame-level quality scores for video frames of a video item is determined. A sliding window is applied to the video frames to identify a plurality of groups of the video frames for scoring on a group-level. A plurality of group-level quality scores for the plurality of groups of video frames of the video item is determined using the frame-level quality scores of the video frames. One of the plurality of groups of video frames of the video item is selected based on the plurality of group-level quality scores. A moving thumbnail is created using the selected group of video frames selected based on the respective group level quality score.Type: GrantFiled: July 8, 2019Date of Patent: September 15, 2020Assignee: Google LLCInventors: Weilong Yang, Min-hsuan Tsai, Zheng Sun, Pei Cao, Tomas Izo
-
Publication number: 20190333538Abstract: Frame-level quality scores for video frames of a video item is determined. A sliding window is applied to the video frames to identify a plurality of groups of the video frames for scoring on a group-level. A plurality of group-level quality scores for the plurality of groups of video frames of the video item is determined using the frame-level quality scores of the video frames. One of the plurality of groups of video frames of the video item is selected based on the plurality of group-level quality scores. A moving thumbnail is created using the selected group of video frames selected based on the respective group level quality score.Type: ApplicationFiled: July 8, 2019Publication date: October 31, 2019Inventors: Weilong Yang, Min-hsuan Tsai, Zheng Sun, Pei Cao, Tomas Izo
-
Patent number: 10390067Abstract: Implementations disclose predicting video start times for maximizing user engagement. A method includes receiving a first content item comprising content item segments, processing the first content item using a trained machine learning model that is trained based on interaction signals and audio-visual content features of a training set of training segments of training content items, and obtaining, based on the processing of the first content item using the trained machine learning model, one or more outputs comprising salience scores for the content item segments, the salience scores indicating which content item segment of the content item segments is to be selected as a starting point for playback of the first content item.Type: GrantFiled: May 12, 2017Date of Patent: August 20, 2019Assignee: Google LLCInventors: Sanketh Shetty, Apostol Natsev, Balakrishnan Varadarajan, Tomas Izo
-
Patent number: 10347294Abstract: A method of generating a moving thumbnail is disclosed. The method includes sampling video frames of a video item. The method further includes determining frame-level quality scores for the sampled video frames. The method also includes determining multiple group-level quality scores for multiple groups of the sampled video frames using the frame-level quality scores of the sampled video frames. The method further includes selecting one of the groups of the sampled video frames based on the multiple group-level quality scores. The method includes creating a moving thumbnail using a subset of the video frames that have timestamps within a range from the start timestamp to the end timestamp.Type: GrantFiled: June 30, 2016Date of Patent: July 9, 2019Assignee: GOOGLE LLCInventors: Weilong Yang, Min-Hsuan Tsai, Zheng Sun, Pei Cao, Tomas Izo
-
Publication number: 20180239964Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.Type: ApplicationFiled: April 23, 2018Publication date: August 23, 2018Inventors: Sanketh Shetty, Tomas Izo, Min-Hsuan Tsai, Sudheendra Vijayanarasimhan, Apostol Natsev, Sami Abu-El-Haija, George Dan Toderici, Susanna Ricco, Balakrishnan Varadarajan, Nicola Muscettola, WeiHsin Gu, Weilong Yang, Nitin Khandelwal, Phuong Le
-
Patent number: 9953222Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.Type: GrantFiled: September 8, 2015Date of Patent: April 24, 2018Assignee: Google LLCInventors: Sanketh Shetty, Tomas Izo, Min-Hsuan Tsai, Sudheendra Vijayanarasimhan, Apostol Natsev, Sami Abu-El-Haija, George Dan Toderici, Susanna Ricco, Balakrishnan Varadarajan, Nicola Muscettola, WeiHsin Gu, Weilong Yang, Nitin Khandelwal, Phuong Le
-
Publication number: 20180005666Abstract: A method of generating a moving thumbnail is disclosed. The method includes sampling video frames of a video item. The method further includes determining frame-level quality scores for the sampled video frames. The method also includes determining multiple group-level quality scores for multiple groups of the sampled video frames using the frame-level quality scores of the sampled video frames. The method further includes selecting one of the groups of the sampled video frames based on the multiple group-level quality scores. The method includes creating a moving thumbnail using a subset of the video frames that have timestamps within a range from the start timestamp to the end timestamp.Type: ApplicationFiled: June 30, 2016Publication date: January 4, 2018Inventors: Weilong Yang, Min-Hsuan Tsai, Zheng Sun, Pei Cao, Tomas Izo
-
Patent number: 9659218Abstract: Implementations disclose predicting video start times for maximizing user engagement. A method includes applying a machine-learned model to audio-visual content features of segments of a target content item, the machine-learned model trained based on user interaction signals and audio-visual content features of a training set of content item segments, calculating, based on applying the machine-learned model, a salience score for each of the segments of the target content item, and selecting, based on the calculated salience scores, one of the segments of the target content item as a starting point for playback of the target content item.Type: GrantFiled: April 29, 2015Date of Patent: May 23, 2017Assignee: Google Inc.Inventors: Sanketh Shetty, Apostol Natsev, Balakrishnan Varadarajan, Tomas Izo
-
Patent number: 9607224Abstract: A solution is provided for temporally segmenting a video based on analysis of entities identified in the video frames of the video. The video is decoded into multiple video frames and multiple video frames are selected for annotation. The annotation process identifies entities present in a sample video frame and each identified entity has a timestamp and confidence score indicating the likelihood that the entity is accurately identified. For each identified entity, a time series comprising of timestamps and corresponding confidence scores is generated and smoothed to reduce annotation noise. One or more segments containing an entity over the length of the video are obtained by detecting boundaries of the segments in the time series of the entity. From the individual temporal segmentation for each identified entity in the video, an overall temporal segmentation for the video is generated, where the overall temporal segmentation reflects the semantics of the video.Type: GrantFiled: May 14, 2015Date of Patent: March 28, 2017Assignee: Google Inc.Inventors: Min-hsuan Tsai, Sudheendra Vijayanarasimhan, Tomas Izo, Sanketh Shetty, Balakrishnan Varadarajan
-
Publication number: 20160335499Abstract: A solution is provided for temporally segmenting a video based on analysis of entities identified in the video frames of the video. The video is decoded into multiple video frames and multiple video frames are selected for annotation. The annotation process identifies entities present in a sample video frame and each identified entity has a timestamp and confidence score indicating the likelihood that the entity is accurately identified. For each identified entity, a time series comprising of timestamps and corresponding confidence scores is generated and smoothed to reduce annotation noise. One or more segments containing an entity over the length of the video are obtained by detecting boundaries of the segments in the time series of the entity. From the individual temporal segmentation for each identified entity in the video, an overall temporal segmentation for the video is generated, where the overall temporal segmentation reflects the semantics of the video.Type: ApplicationFiled: May 14, 2015Publication date: November 17, 2016Inventors: Min-hsuan Tsai, Sudheendra Vijayanarasimhan, Tomas Izo, Sanketh Shetty, Balakrishnan Varadarajan
-
Publication number: 20160070962Abstract: A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.Type: ApplicationFiled: September 8, 2015Publication date: March 10, 2016Inventors: Sanketh Shetty, Tomas Izo, Min-Hsuan Tsai, Sudheendra Vijayanarasimhan, Apostol Natsev, Sami Abu-El-Haija, George Dan Toderici, Susanna Ricco, Balakrishnan Varadarajan, Nicola Muscettola, WeiHsin Gu, Weilong Yang, Nitin Khandelwal, Phuong Le
-
Patent number: 9235203Abstract: This disclosure is directed to providing audio playback to a mobile device user. According to one aspect of this disclosure, a mobile device may be to modify audio playback in response to detecting an inclination of the mobile device (and thereby a user) with respect to a reference plane. According to another aspect of this disclosure, a mobile device may be configured to automatically identify an audible sound that may be motivational to a user, and store an indication of the audible sound in response to the identification. According to another aspect of this disclosure, a mobile device may automatically play back a previously identified motivational song in response to detection of user movement.Type: GrantFiled: October 2, 2014Date of Patent: January 12, 2016Assignee: Google Inc.Inventors: Tomas Izo, Gal Chechik
-
Patent number: 9148619Abstract: A system and method provide a soundtrack recommendation service for recommending one or more soundtrack for a video (i.e., a probe video). A feature extractor of the recommendation service extracts a set of content features of the probe video and generates a set of semantic features represented by a signature vector of the probe video. A video search module of the recommendation service is configured to search for a number of video candidates, each of which is semantically similar to the probe video and has an associated soundtrack. A video outlier identification module of the recommendation service identifies video candidates having an atypical use of their soundtracks and ranks the video candidates based on the typicality of their soundtrack usage. A soundtrack recommendation module selects the soundtracks of the top ranked video candidates as the soundtrack recommendations to the probe video.Type: GrantFiled: April 2, 2014Date of Patent: September 29, 2015Assignee: Google Inc.Inventors: Tomas Izo, Hrishikesh Aradhye, Bilson Campana, Gal Chechik
-
Patent number: 8886345Abstract: This disclosure is directed to providing audio playback to a mobile device user. According to one aspect of this disclosure, a mobile device may be to modify audio playback in response to detecting an inclination of the mobile device (and thereby a user) with respect to a reference plane. According to another aspect of this disclosure, a mobile device may be configured to automatically identify an audible sound that may be motivational to a user, and store an indication of the audible sound in response to the identification. According to another aspect of this disclosure, a mobile device may automatically play back a previously identified motivational song in response to detection of user movement.Type: GrantFiled: September 23, 2011Date of Patent: November 11, 2014Assignee: Google Inc.Inventors: Tomas Izo, Gal Chechik
-
Publication number: 20140212106Abstract: A system and method provide a soundtrack recommendation service for recommending one or more soundtrack for a video (i.e., a probe video). A feature extractor of the recommendation service extracts a set of content features of the probe video and generates a set of semantic features represented by a signature vector of the probe video. A video search module of the recommendation service is configured to search for a number of video candidates, each of which is semantically similar to the probe video and has an associated soundtrack. A video outlier identification module of the recommendation service identifies video candidates having an atypical use of their soundtracks and ranks the video candidates based on the typicality of their soundtrack usage. A soundtrack recommendation module selects the soundtracks of the top ranked video candidates as the soundtrack recommendations to the probe video.Type: ApplicationFiled: April 2, 2014Publication date: July 31, 2014Applicant: Google Inc.Inventors: Tomas Izo, Hrishikesh Aradhye, Bilson Campana, Gal Chechik
-
Patent number: 8737817Abstract: A system and method provide a soundtrack recommendation service for recommending one or more soundtrack for a video (i.e., a probe video). A feature extractor of the recommendation service extracts a set of content features of the probe video and generates a set of semantic features represented by a signature vector of the probe video. A video search module of the recommendation service is configured to search for a number of video candidates, each of which is semantically similar to the probe video and has an associated soundtrack. A video outlier identification module of the recommendation service identifies video candidates having an atypical use of their soundtracks and ranks the video candidates based on the typicality of their soundtrack usage. A soundtrack recommendation module selects the soundtracks of the top ranked video candidates as the soundtrack recommendations to the probe video.Type: GrantFiled: February 8, 2011Date of Patent: May 27, 2014Assignee: Google Inc.Inventors: Tomas Izo, Hrishikesh Aradhye, Bilson Campana, Gal Chechik
-
Patent number: 8392007Abstract: This disclosure is directed to providing audio playback to a mobile device user. According to one aspect of this disclosure, a mobile device may be to modify audio playback in response to detecting an inclination of the mobile device (and thereby a user) with respect to a reference plane. According to another aspect of this disclosure, a mobile device may be configured to automatically identify an audible sound that may be motivational to a user, and store an indication of the audible sound in response to the identification. According to another aspect of this disclosure, a mobile device may automatically play back a previously identified motivational song in response to detection of user movement.Type: GrantFiled: September 30, 2011Date of Patent: March 5, 2013Assignee: Google Inc.Inventors: Tomas Izo, Gal Chechik
-
Patent number: 8319087Abstract: Methods and systems for generating playlists of media items with audio data are disclosed. Based on two received feature sets, media items corresponding to each feature set are identified. Transition characteristics are also received. Based on the identified media items and transition characteristics, a dynamic playlist is generated that transitions from media items having characteristics of the first feature set to media items having characteristics of the second feature set. Each time the playlist is generated, it may include a different set of media items.Type: GrantFiled: March 30, 2011Date of Patent: November 27, 2012Assignee: Google Inc.Inventors: Rich Gossweiler, Douglas Eck, Hrishikesh Aradhye, Tomas Izo