Patents by Inventor Shashank Merchant
Shashank Merchant has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11962846Abstract: The disclosure provides for use of steganographically-encoded data as a basis to control content modification. A server could determine through a fingerprint comparison that a portion of media content on a channel being rendered by a content presentation device matches at least one modifiable-content segment, such as at least one replaceable ad. The server could then use watermark data encoded in the media content of the channel as a basis to determine an actual identity of a modifiable-content segment present on the channel, so as to then control content modification.Type: GrantFiled: December 14, 2021Date of Patent: April 16, 2024Assignee: ROKU, INC.Inventors: Shashank Merchant, Do Phuoc Hoang Tuong Lan
-
Patent number: 11937026Abstract: In one aspect, an example method includes (i) providing, by a playback device, replacement media content for display; (ii) determining, by the playback device, that a remote control transmitted to the playback device an instruction configured to cause a modification to operation of the playback device while the playback device displays the replacement media content; (iii) determining, by the playback device based on the instruction, an overlay that the playback device is configured to provide for display in conjunction with the modification; (iv) determining, by the playback device, a region within a display of the playback device corresponding to the overlay; and (v) modifying, by the playback device, a transparency of the region such that the overlay is visible through the replacement media content when the playback device provides the overlay for display.Type: GrantFiled: March 11, 2022Date of Patent: March 19, 2024Assignee: Roku, Inc.Inventors: Kurt R. Thielen, Shashank Merchant, Peter Dunker, Markus K. Cremer, Chungwon Seo, Seunghyeong Lee, Steven D. Scherf
-
Patent number: 11895370Abstract: In one aspect, an example method includes (i) providing, by a playback device, replacement media content for display; (ii) determining, by the playback device that while the playback device is displaying the replacement media content a remote control transmitted an instruction to a media device that provides media content to the playback device; (iii) determining, by the playback device, a playback-modification action corresponding to the instruction and the media device; and (iv) modifying, by the playback device, playback of the replacement media content in accordance with the playback-modification action.Type: GrantFiled: March 28, 2022Date of Patent: February 6, 2024Assignee: Roku, Inc.Inventors: Kurt R. Thielen, Shashank Merchant, Peter Dunker, Markus K. Cremer, Chungwon Seo, Seunghyeong Lee, Steven D Scherf
-
Patent number: 11869261Abstract: Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices, are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.Type: GrantFiled: February 22, 2023Date of Patent: January 9, 2024Assignee: Roku, Inc.Inventors: Jose Pio Pereira, Sunil Suresh Kulkarni, Mihailo M. Stojancic, Shashank Merchant, Peter Wendt
-
Publication number: 20230353822Abstract: In one aspect, an example method includes (i) identifying, by a playback device, a media device that provides media content to the playback device; (ii) providing, by the playback device, replacement media content for display; (iii) determining, by the playback device, that while the playback device is displaying the replacement media content a remote control transmitted an instruction to the identified media device; (iv) determining, by the playback device, a playback-modification action corresponding to the instruction and the identified media device; and (v) modifying, by the playback device, playback of the replacement media content in accordance with the playback-modification action.Type: ApplicationFiled: July 6, 2023Publication date: November 2, 2023Inventors: KURT R. THIELEN, Peter Dunker, Markus K. Cremer, Steven D. Scherf, Shashank Merchant
-
Patent number: 11803591Abstract: A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.Type: GrantFiled: February 16, 2022Date of Patent: October 31, 2023Assignee: Roku, Inc.Inventors: Jose Pio Pereira, Mihailo M. Stojancic, Shashank Merchant
-
Publication number: 20230308703Abstract: In one aspect, a method includes identifying an upcoming content-modification opportunity on a channel, where the upcoming content-modification opportunity relates to a modifiable content-segment transmitted by a content-distribution system on the channel, and where the modifiable content-segment includes an overlay content portion and a non-overlay content portion. The method also includes responsive to identifying the upcoming content-modification opportunity on the channel, identifying a content-presentation device tuned to the channel. The method also includes determining location data associated with the overlay content portion.Type: ApplicationFiled: March 28, 2023Publication date: September 28, 2023Inventors: Andrew Laurence Ashbacher, Steven Michael Cormie, Markus Kurt Peter Cremer, Shashank Merchant, Andreas Schmidt
-
Patent number: 11765426Abstract: In one aspect, an example method includes (i) identifying, by a playback device, a media device based on a control message received from the media device by way of an audio and/or video interface, where the media device provides media content to the playback device; (ii) providing, by the playback device, replacement media content for display; (iii) determining, by the playback device, that while the playback device is displaying the replacement media content a remote control transmitted an instruction to the identified media device; (iv) determining, by the playback device, a playback-modification action corresponding to the instruction and the identified media device; and (v) modifying, by the playback device, playback of the replacement media content in accordance with the playback-modification action.Type: GrantFiled: January 26, 2022Date of Patent: September 19, 2023Assignee: ROKU, INC.Inventors: Kurt R. Thielen, Peter Dunker, Markus K. Cremer, Steven D. Scherf, Shashank Merchant
-
Patent number: 11748055Abstract: In one aspect, an example method includes (i) determining, by a playback device, a first loudness level of a first portion of first media content from a first source while the playback device presents the first media content, with the first portion having a first length; (ii) switching, by the playback device, from presenting the first media content from the first source to presenting second media content from a second source; (iii) based on the switching, determining, by the playback device, second loudness levels of second portions of the first media content while the playback device presents the second media content, with the second portions having a second length that is shorter than the first length; and (iv) while the playback device presents the second media content, adjusting, by the playback device, a volume of the playback device based on one or more of the second loudness levels.Type: GrantFiled: May 20, 2022Date of Patent: September 5, 2023Assignee: ROKU, INC.Inventors: Markus K. Cremer, Shashank Merchant, Robert Coover, Todd J. Hodges, Joshua Ernest Morris
-
Publication number: 20230276090Abstract: Disclosed is a method that includes a media presentation device (i) generating query fingerprint data representing video frames of a channel being rendered by the media presentation device, and transmitting the generated query fingerprint data from the device over a network to a server, (ii) responsive to transmitting the query fingerprint data to the server, receiving, from the server, channel-icon fingerprint data representing channel icons of multiple channels selected based at least on the server having detected that the query fingerprint data representing the video frames of the channel being rendered by the media presentation device matches the reference fingerprint data representing the frames of the multiple channels of video content, and (iii) responsive to receiving the channel-icon fingerprint data from the server, performing channel-icon fingerprint matching to determine which channel in the multiple channels is the channel being rendered by the media presentation device.Type: ApplicationFiled: May 1, 2023Publication date: August 31, 2023Applicant: ROKU, INC.Inventors: Andreas Schmidt, Shashank Merchant, Markus Kurt Peter Cremer
-
Patent number: 11740860Abstract: In one aspect, an example method includes (i) presenting first media content from a first source; (ii) encountering a trigger to switch from presenting the first media content from the first source to presenting second media content from a second source; (iii) determining a first loudness level of the first media content; (iv) determining a second loudness level of the second media content; (v) based on a difference between the first loudness level and the second loudness level, adjusting a loudness level of the second media content so as to generate modified media content having a third loudness level that is different from the second loudness level; and (vi) responsive to encountering the trigger, presenting the modified media content having the third loudness level.Type: GrantFiled: April 27, 2022Date of Patent: August 29, 2023Assignee: ROKU, INC.Inventors: Markus K. Cremer, Shashank Merchant, Aneesh Vartakavi
-
Patent number: 11736762Abstract: A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.Type: GrantFiled: April 18, 2022Date of Patent: August 22, 2023Assignee: ROKU, INC.Inventors: Mihailo M. Stojancic, Jose Pio Pereira, Peter Wendt, Shashank Merchant, Sunil Suresh Kulkarni
-
Patent number: 11729458Abstract: A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.Type: GrantFiled: June 14, 2019Date of Patent: August 15, 2023Assignee: ROKU, INC.Inventors: Mihailo M. Stojancic, Sunil Suresh Kulkarni, Shashank Merchant, Jose Pio Pereira, Oleksiy Bolgarov
-
Patent number: 11722721Abstract: Disclosed is a method that includes a media presentation device (i) generating query fingerprint data representing video frames of a channel being rendered by the media presentation device, and transmitting the generated query fingerprint data from the device over a network to a server, (ii) responsive to transmitting the query fingerprint data to the server, receiving, from the server, channel-icon fingerprint data representing channel icons of multiple channels selected based at least on the server having detected that the query fingerprint data representing the video frames of the channel being rendered by the media presentation device matches the reference fingerprint data representing the frames of the multiple channels of video content, and (iii) responsive to receiving the channel-icon fingerprint data from the server, performing channel-icon fingerprint matching to determine which channel in the multiple channels is the channel being rendered by the media presentation device.Type: GrantFiled: December 16, 2021Date of Patent: August 8, 2023Assignee: ROKU, INC.Inventors: Andreas Schmidt, Shashank Merchant, Markus Kurt Peter Cremer
-
Patent number: 11706481Abstract: A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.Type: GrantFiled: June 14, 2019Date of Patent: July 18, 2023Inventors: Mihailo M. Stojancic, Daniel H. Eakins, Shashank Merchant, Prashant Ramanathan, Jose Pio Pereira
-
Publication number: 20230196809Abstract: Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices, are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.Type: ApplicationFiled: February 22, 2023Publication date: June 22, 2023Applicant: Roku, Inc.Inventors: Jose Pio PEREIRA, Sunil Suresh KULKARNI, Mihailo M. STOJANCIC, Shashank MERCHANT, Peter WENDT
-
Patent number: 11638058Abstract: Disclosed is a method including (i) detecting that query fingerprint data representing frames of video content being rendered by a media presentation device matches reference fingerprint data representing frames of multiple channels of video content, including detecting that the query fingerprint data matches multiple instances of reference fingerprint data each representing frames of a different respective channel in the multiple channels of video content, and (ii) responsive to at least the detecting that the query fingerprint data representing the frames of the video content being rendered by the device matches the reference fingerprint data representing the frames of the multiple channels of video content, configuring the device with channel-icon fingerprint data representing channel icons of the multiple channels of video content, to enable the device to perform channel-icon fingerprint matching to determine which channel of the multiple channels is being rendered by the device.Type: GrantFiled: December 16, 2021Date of Patent: April 25, 2023Assignee: Roku, Inc.Inventors: Andreas Schmidt, Shashank Merchant, Markus Kurt Peter Cremer
-
Patent number: 11630858Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.Type: GrantFiled: August 20, 2021Date of Patent: April 18, 2023Assignee: Roku, Inc.Inventors: Mihailo M. Stojancic, Prashant Ramanathan, Peter Wendt, Jose Pio Pereira, Shashank Merchant
-
Patent number: 11631404Abstract: Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices, are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.Type: GrantFiled: August 12, 2021Date of Patent: April 18, 2023Assignee: ROKU, INC.Inventors: Jose Pio Pereira, Sunil Suresh Kulkarni, Mihailo M. Stojancic, Shashank Merchant, Peter Wendt
-
Patent number: 11617001Abstract: In one aspect, a method includes identifying an upcoming content-modification opportunity on a channel, where the upcoming content-modification opportunity relates to a modifiable content-segment transmitted by a content-distribution system on the channel, and where the modifiable content-segment includes an overlay content portion and a non-overlay content portion. The method also includes responsive to identifying the upcoming content-modification opportunity on the channel, identifying a content-presentation device tuned to the channel. The method also includes determining location data associated with the overlay content portion.Type: GrantFiled: August 11, 2021Date of Patent: March 28, 2023Assignee: ROKU, INC.Inventors: Andrew Laurence Ashbacher, Steven Michael Cormie, Markus Kurt Peter Cremer, Shashank Merchant, Andreas Schmidt