Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Computer vision and image characteristic search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and comparing the determined characteristics to visual characteristics of other images, e.g., to identify similar visual characteristics in the other images. In some aspects, the described system performs searches that leverage a digital image as part of a search query to locate digital content of interest. In some aspects, the described system surfaces multiple user interface instrumentalities that include images of patterns, textures, or materials and that are selectable to initiate a visual search of digital content having a similar pattern, texture, or material. The described aspects also include pattern-based authentication in which the system determines authenticity of an item in an image based on a similarity of its visual characteristics to visual characteristics of known authentic items.

Abstract: Computer vision and image characteristic search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and comparing the determined characteristics to visual characteristics of other images, e.g., to identify similar visual characteristics in the other images. In some aspects, the described system performs searches that leverage a digital image as part of a search query to locate digital content of interest. In some aspects, the described system surfaces multiple user interface instrumentalities that include images of patterns, textures, or materials and that are selectable to initiate a visual search of digital content having a similar pattern, texture, or material. The described aspects also include pattern-based authentication in which the system determines authenticity of an item in an image based on a similarity of its visual characteristics to visual characteristics of known authentic items.

Abstract: Computer vision for unsuccessful queries and iterative search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and describing them, e.g., using feature vectors. In some aspects, these visual characteristics are determined for search queries that are identified as not being successful. Aggregated information describing visual characteristics of images of unsuccessful search queries is used to determine common visual characteristics and objects depicted in those images. This information can be used to inform other users about unmet needs of searching users. In some aspects, these visual characteristics are used in connection with iterative image searches where users select an initial query image and then the search results are iteratively refined.

Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Computer vision and image characteristic search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and comparing the determined characteristics to visual characteristics of other images, e.g., to identify similar visual characteristics in the other images. In some aspects, the described system performs searches that leverage a digital image as part of a search query to locate digital content of interest. In some aspects, the described system surfaces multiple user interface instrumentalities that include images of patterns, textures, or materials and that are selectable to initiate a visual search of digital content having a similar pattern, texture, or material. The described aspects also include pattern-based authentication in which the system determines authenticity of an item in an image based on a similarity of its visual characteristics to visual characteristics of known authentic items.

Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Computer vision for unsuccessful queries and iterative search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and describing them, e.g., using feature vectors. In some aspects, these visual characteristics are determined for search queries that are identified as not being successful. Aggregated information describing visual characteristics of images of unsuccessful search queries is used to determine common visual characteristics and objects depicted in those images. This information can be used to inform other users about unmet needs of searching users. In some aspects, these visual characteristics are used in connection with iterative image searches where users select an initial query image and then the search results are iteratively refined.

Abstract: Computer vision and image characteristic search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and comparing the determined characteristics to visual characteristics of other images, e.g., to identify similar visual characteristics in the other images. In some aspects, the described system performs searches that leverage a digital image as part of a search query to locate digital content of interest. In some aspects, the described system surfaces multiple user interface instrumentalities that include images of patterns, textures, or materials and that are selectable to initiate a visual search of digital content having a similar pattern, texture, or material. The described aspects also include pattern-based authentication in which the system determines authenticity of an item in an image based on a similarity of its visual characteristics to visual characteristics of known authentic items.

Abstract: A computer program product includes machine readable instructions for providing enhanced video output by: receiving footage including likeness information in a plurality of modalities; demultiplexing the plurality of modalities to provide information for each modality; comparing information from at least two of the modalities for determining a correlation in the likeness information; using the correlation, obtaining semantic information for association with the likeness; and combining the semantic information with the likeness information for providing the enhanced video output. A system for implementing the computer program product includes resources for receiving the footage.

Abstract: A method of generating a positional representation of a document, including identifying each unique term in a document and positions in the document at which the unique term appears, and for the each unique term, storing positional information derived from the positions into a positional representation.

Abstract: A computer program product includes machine readable instructions for providing enhanced video output by: receiving footage including likeness information in a plurality of modalities; demultiplexing the plurality of modalities to provide information for each modality; comparing information from at least two of the modalities for determining a correlation in the likeness information; using the correlation, obtaining semantic information for association with the likeness; and combining the semantic information with the likeness information for providing the enhanced video output. A system for implementing the computer program product includes resources for receiving the footage.

Abstract: A method of generating a positional representation of a document, including identifying each unique term in a document and positions in the document at which the unique term appears, and for the each unique term, storing positional information derived from the positions into a positional representation.

Abstract: A computer program product includes machine readable instructions for providing enhanced video output by: receiving footage including likeness information in a plurality of modalities; demultiplexing the plurality of modalities to provide information for each modality; comparing information from at least two of the modalities for determining a correlation in the likeness information; using the correlation, obtaining semantic information for association with the likeness; and combining the semantic information with the likeness information for providing the enhanced video output. A system for implementing the computer program product includes resources for receiving the footage.

Abstract: The use of multiple complementary classes of measure to assess face and speech consistency in video. In an exemplary embodiment, both synchrony measures and plausibility measures are employed.

Abstract: Methods and arrangements for annotating digital input. Digital media input is accepted, with the input being arranged in frames, while in annotating at least one of the following are performed: the presentation of frames for annotation in non-linear fashion; and the employment of a cached annotation lexicon for applying labels to frames.

Abstract: A mechanism is provided for cross-linking information sources using multiple modalities. Text documents, images, audio sources, video, and other media are analyzed to determine media descriptors, which are metadata describing the content of the media sources. The media descriptors from all modalities are collated and cross-linked. A query processing and presentation module, which receives queries and presents results, may also be provided. A query may consist of textual keywords from user input. Alternatively, a query may derive from a media source, such as a text document, image, audio source, or video source.

Abstract: A manual annotation system of multi-modal characteristics in multimedia files. There is provided an arrangement for selection an observation modality of video with audio, video without audio, audio with video, or audio without video, to be used to annotate multimedia content. While annotating video or audio features in isolation results in less confidence in the identification of the features, observing both audio and video simultaneously and annotating that observation results in a higher confidence level.