Abstract: A search computing system receives an interaction history for one or more respective users describing interactions with one or more items and generates a co-interaction matrix, each value in the co-interaction matrix representing a number of common users determined to have had the interaction with an item of a corresponding row and an item of the corresponding column of the co-interaction matrix where the value is located. The search computing system generates an embeddings matrix comprising an item embedding value for each of the one or more items by applying matrix factorization to the co-interaction matrix and determines, in response to a search query of a particular user, a user embedding value for the searching user based on the interaction history for the searching user. The search computing system determines a similarity between each search result and user interaction history by comparing the user embedding value against each of the item embedding values.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides systems, methods, and computer program products for modifying documents for shared use, and for collaborative discussion of shared documents. Captured digital images of documents or portions associated with a user are received, along with modifications to the images. Documents are recognized from the captured digital images, and the modifications to the images are applied to the documents. Alternatively, captured digital images of documents are received, and the documents are recognized along with a hotspot in the document. The user is authenticated to a website associated with the hotspot, and is provided access to an associated multimedia repository.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The MMR system provides mechanisms for forming a media document that includes media of at least two types, such as printed paper as a first medium and a digital photograph, digital movie, digital audio file, or web link as a second medium. Furthermore, the MMR system facilitates business methods that take advantage of the combination of a portable electronic device, such as a cellular camera phone, and a paper document.

Abstract: An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a recognition result. Image tracking information also is provided for determining relative locations of images to each other. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The disclosure also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

Abstract: An MMR system includes an analytics module for monitoring and analyzing the creation and usage of visual content search. The analytics module is part of an MMR gateway and provides a mechanism for administrators to determine the usage patterns and other information about users of the MMR system. The analytics module comprises a monitoring module, a usage database, an access analysis module and a presentation module. The monitoring module collects information about the retrieval request, image queries, context information and recognition results. The usage database stores the information collected by the monitoring module.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides a system, method, and computer program product for annotating unindexed documents. Captured digital images of unknown paper documents or portions associated with a user are received with annotations. Features are extracted from the captured digital image and are stored, along with the captured digital image and the annotations.

Abstract: An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a distributor for distributing the image to corresponding content type specific index tables and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The techniques described herein also include a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

Abstract: Techniques for automatically comparing one or more objects determined from digital media content (e.g., an image, audio information, video information) to one or more objects specified by a machine readable identifier to determine if an object determined from the media content matches an object specified by the machine readable identifier. One or more actions may be initiated upon determining that an object determined from the media content matches an object specified by the machine readable identifier. Information identifying the action to be initiated may also be encapsulated by the machine readable identifier.

Abstract: The present invention uses invisible junctions which are a set of local features unique to every page of the electronic document to match the captured image to a part of an electronic document. The present invention includes: an image capture device, a feature extraction and recognition system and database. When an electronic document is printed, the feature extraction and recognition system captures an image of the document page. The features in the captured image are then extracted, indexed and stored in the database. Given a query image, the features in the query image are extracted and compared against those stored in the database to identify the query image. The feature extraction and recognition system of the present invention is integrated into a multifunction peripheral. This allows the feature extraction and recognition system to be used in conjunction with other modules to provide security and annotation applications.

Abstract: An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit, and an MMR publisher. A computer includes a document authoring application, plug-in, and printer driver. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a method for capturing symbolic information from documents and for capturing printer dll functions.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The MMR system provides mechanisms for forming a media document that includes media of at least two types, such as printed paper as a first medium and a digital photograph, digital movie, digital audio file, or web link as a second medium. Furthermore, the MMR system facilitates business methods that take advantage of the combination of a portable electronic device, such as a cellular camera phone, and a paper document.

Abstract: An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher integrated into a network with an advertiser, an ad broker, and an MMR service bureau. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR service bureau uses the result to retrieve advertising or other links associated with the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a differentially weighting links associated with an MMR document.

Abstract: An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a recognition result. Image tracking information also is provided for determining relative locations of images to each other. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The disclosure also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

Abstract: An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. Image tracking information also is provided for determining relative locations of images within a document page. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The present invention also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system includes an MMR user, a MMR computer, a user printer that produces a printed document, a networked media server, an office portal, a service provider server, an electronic display that is electrically connected to a set-top box, a document scanner, a network, a capture device, a cellular infrastructure, wireless fidelity (Wi-Fi) technology, Bluetooth® technology, infrared (IR) technology, wired technology, and a geo location mechanism.

Abstract: An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a distributor for distributing the image to corresponding content type specific index tables and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The techniques described herein also include a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

Abstract: The mobile device includes a client that has a number of modules, and the MMR Gateway and MMR matching unit are implemented as a server that has a number of modules. The implementation of the MMR system as a client and a server is advantageous because the modules may be distributed among the client and the server in a variety of configurations. The present invention includes a capture module, a preprocessing module, a feature extraction module, a retrieval module, a send message module, an action module, a prediction module, a feedback module, a sending module, an MMR database, a streaming module, an e-mail module, a voice recognition system and an audio database. These modules and systems are operational upon the client or the server.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides a system, method, and computer program product for creating an ad hoc association between users. Captured digital images of paper documents or portions associated with a user are received. Document recognition features are extracted from the captured digital image. An ad hoc association is created between the user and another user associated with an existing document recognition feature similar to the extracted document recognition features.

Abstract: An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit includes a segmenter for segmenting received images by content type, a distributor for distributing the images to corresponding content type index tables, and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The present invention also includes a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system includes a method, system, and computer program product for adding a hotspot to an imaged document. A source document is converted to an imaged document, from which features are extracted. Hotspots are added to the imaged document, and the imaged document, hotspot definitions, and the feature representation are stored.