Abstract: Various embodiments enable a process to automatically attempt to select the most relevant words associated with products available for purchase from an electronic marketplace from an image frame. For example, an image frame containing text can be obtained and analyzed with an optical character recognition. The recognized words can then be preprocessed using various filtering and scoring techniques to narrow down a volume of text to a few relevant query terms. These query terms can then be sent to a search engine associated with the electronic marketplace to return relevant products to a user.

Abstract: A video segmentation system can be utilized to automate segmentation of digital video content. Features corresponding to visual, audio, and/or textual content of the video can be extracted from frames of the video. The extracted features of adjacent frames are compared according to a similarity measure to determine boundaries of a first set of shots or video segments distinguished by abrupt transitions. The first set of shots is analyzed according to certain heuristics to recognize a second set of shots distinguished by gradual transitions. Key frames can be extracted from the first and second set of shots, and the key frames can be used by the video segmentation system to group the first and second set of shots by scene. Additional processing can be performed to associate metadata, such as names of actors or titles of songs, with the detected scenes.

Abstract: Image processing techniques for self-captured images are disclosed. An image can be captured during activation of an illumination element of a mobile device. Presence of a representation of the mobile device can be determined in the captured image based at least in part upon locating a reflection of an illumination produced by the illumination element. Dimensions of the representation of the mobile device can be determined and compared to actual dimensions of the mobile device to provide a scaling factor. The scaling factor can be used to determine the size of various objects, including articles of clothing, in the image.

Abstract: Approaches include using a machine learning-based approach to generating search strings and refinements based on a specific item represented in an image. For example, a classifier that is trained on descriptions of images can be provided. An image that includes a representation of an item of interest is obtained. The image is analyzed using the classifier algorithm to determine a first term representing a visual characteristic of the image. Then, the image is analyzed again to determine a second term representing another visual characteristic of the image based at least in part on the first term. Additional terms can be determined to generate a description of the image, including characteristics of the item of interest. Based on the determined characteristics of the item of interest, a search query and one or more refinements can be generated.

Abstract: Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Abstract: Techniques are described for providing functionality to allow a viewer of a television show to watch a “previously on” segment of an episode of the television show and be able to watch the scenes from prior episodes referenced in the “previously on” segment.

Abstract: A video segmentation system can be utilized to automate segmentation of digital video content. Features corresponding to visual, audio, and/or textual content of the video can be extracted from frames of the video. The extracted features of adjacent frames are compared according to a similarity measure to determine boundaries of a first set of shots or video segments distinguished by abrupt transitions. The first set of shots is analyzed according to certain heuristics to recognize a second set of shots distinguished by gradual transitions. Key frames can be extracted from the first and second set of shots, and the key frames can be used by the video segmentation system to group the first and second set of shots by scene. Additional processing can be performed to associate metadata, such as names of actors or titles of songs, with the detected scenes.

Abstract: Various embodiments enable a computing device to perform tasks such as processing an image to recognize text or an object in an image to identify a particular product or related products associated with the text or object. In response to recognizing the text or the object as being associated with a product available for purchase from an electronic marketplace, one or more advertisements or product listings associated with the product can be displayed to the user. Accordingly, additional information for the associated product can be displayed, enabling the user to learn more about and purchase the product from the electronic marketplace through the portable computing device.

Abstract: A user can select an object represented in video content in order to set a magnification level with respect to that object. A portion of the video frames containing a representation of the object is selected to maintain a presentation size of the representation corresponding to the magnification level. The selection provides for a “smart zoom” feature enabling an object of interest, such as a face of an actor, to be used in selecting an appropriate portion of each frame to magnify, such that the magnification results in a portion of the frame being selected that includes the one or more objects of interest to the user. Pre-generated tracking data can be provided for some objects, which can enable a user to select an object and then have predetermined portion selections and magnifications applied that can provide for a smoother user experience than for dynamically-determined data.

Abstract: Various embodiments enable a computing device to perform tasks such as processing an image to recognize text or an object in an image to identify a particular product or related products associated with the text or object. In response to recognizing the text or the object as being associated with a product available for purchase from an electronic marketplace, one or more advertisements or product listings associated with the product can be displayed to the user. Accordingly, additional information for the associated product can be displayed, enabling the user to learn more about and purchase the product from the electronic marketplace through the portable computing device.

Abstract: A user can select an object represented in video content in order to set a magnification level with respect to that object. A portion of the video frames containing a representation of the object is selected to maintain a presentation size of the representation corresponding to the magnification level. The selection provides for a “smart zoom” feature enabling an object of interest, such as a face of an actor, to be used in selecting an appropriate portion of each frame to magnify, such that the magnification results in a portion of the frame being selected that includes the one or more objects of interest to the user. Pre-generated tracking data can be provided for some objects, which can enable a user to select an object and then have predetermined portion selections and magnifications applied that can provide for a smoother user experience than for dynamically-determined data.

Abstract: Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Abstract: A video segmentation system can be utilized to automate segmentation of digital video content. Features corresponding to visual, audio, and/or textual content of the video can be extracted from frames of the video. The extracted features of adjacent frames are compared according to a similarity measure to determine boundaries of a first set of shots or video segments distinguished by abrupt transitions. The first set of shots is analyzed according to certain heuristics to recognize a second set of shots distinguished by gradual transitions. Key frames can be extracted from the first and second set of shots, and the key frames can be used by the video segmentation system to group the first and second set of shots by scene. Additional processing can be performed to associate metadata, such as names of actors or titles of songs, with the detected scenes.

Abstract: Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Abstract: Various embodiments enable a device to perform an adaptive sampling method for locating text regions in images of natural scenes. Since only a fraction of pixels in an input image correspond to text regions, it is desirable to eliminate non-text regions early in the text detection process. Therefore, in at least one embodiment, an image is sampled horizontally to identify connected pixel regions indicative of text. In this example, each sampled row of the image is tested for containing Maximally Stable Extremal Regions (MSERs) in order to identify regions containing text. If a respective row contains a connected pixel region indicative of text, the neighboring rows are recursively sampled at finer levels in order to fully contain the text and sampling is terminated for regions which are unlikely to contain text. This sampling process can also be performed for the vertical dimension for regions which are determined to contain text.

Abstract: Automatic replay or skip ahead functionality can be configured to intelligently navigate to a portion of a video a user desires to view. The context at which a user selects intelligent navigation can be analyzed to determine where to initiate automatic replay or skip ahead. The context for intelligent navigation can be based on scene or shot segmentation data, closed captioning, aggregate video navigation data from a community of users of shared demographic traits and/or interest, and/or other metadata. In the case of automatic replay, playback of a portion of a video can include enhancements for that portion, such as providing closed captioning, display at a decreased frame rate (“slow motion”), zooming in/out on a portion of the frames of a video segment, among other enhancements.

Abstract: Automatic replay or skip ahead functionality can be configured to intelligently navigate to a portion of a video a user desires to view. The context at which a user selects intelligent navigation can be analyzed to determine where to initiate automatic replay or skip ahead. The context for intelligent navigation can be based on scene or shot segmentation data, closed captioning, aggregate video navigation data from a community of users of shared demographic traits and/or interest, and/or other metadata. In the case of automatic replay, playback of a portion of a video can include enhancements for that portion, such as providing closed captioning, display at a decreased frame rate (“slow motion”), zooming in/out on a portion of the frames of a video segment, among other enhancements.

Abstract: Disclosed are various embodiments for detection of cast members in video content such as movies, television shows, and other programs. Data indicating cast members who appear in a video program is obtained. Each cast member is associated with a reference image depicting a face of the cast member. A frame is obtained from the video program, and a face is detected in the frame. The frame can correspond to a scene in the video program. The detected face in the frame is recognized as being a particular cast member based at least in part on the reference image depicting the cast member. An association between the cast member and the frame is generated in response to the detected face in the frame being recognized as the cast member.

Abstract: Disclosed are various embodiments for detection of cast members in video content such as movies, television shows, and other programs. Data indicating cast members who appear in a video program is obtained. Each cast member is associated with a reference image depicting a face of the cast member. A frame is obtained from the video program, and a face is detected in the frame. The detected face in the frame is recognized as being a particular cast member based at least in part on the reference image depicting the cast member. An association between the cast member and the frame is generated in response to the detected face in the frame being recognized as the cast member.

Abstract: A video segmentation system can be utilized to automate segmentation of digital video content. Features corresponding to visual, audio, and/or textual content of the video can be extracted from frames of the video. The extracted features of adjacent frames are compared according to a similarity measure to determine boundaries of a first set of shots or video segments distinguished by abrupt transitions. The first set of shots is analyzed according to certain heuristics to recognize a second set of shots distinguished by gradual transitions. Key frames can be extracted from the first and second set of shots, and the key frames can be used by the video segmentation system to group the first and second set of shots by scene. Additional processing can be performed to associate metadata, such as names of actors or titles of songs, with the detected scenes.