Abstract: The present invention relies on the two-dimensional information in documents and encodes two-dimensional structures into a one-dimensional synthetic language such that two-dimensional documents can be searched at text search speed. The system comprises: an indexing module, a retrieval module, an encoder, a quantization module, a retrieval engine and a control module coupled by a bus. Electronic documents are first indexed by the indexing module and stored as a synthetic text library. The retrieval module then converts an input image to synthetic text and searches for matches to the synthetic text in the synthetic text library. The matches can be in turn used to retrieve the corresponding electronic documents. In one or more embodiments, the present invention includes a method for comparing the synthetic text to documents that have been converted to synthetic text for a match.

Abstract: An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit, and an MMR publisher. A computer includes a document authoring application, plug-in, and printer driver. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a method for capturing symbolic information from documents and for capturing printer dll functions.

Abstract: The present invention relies on the two-dimensional information in documents and encodes two-dimensional structures into a one-dimensional synthetic language such that two-dimensional documents can be searched at text search speed. The system comprises: an indexing module, a retrieval module, an encoder, a quantization module, a retrieval engine and a control module coupled by a bus. A number of electronic documents are first indexed by the indexing module and stored as a synthetic text library. The retrieval module then converts and input image to synthetic text and searches for matches to the synthetic text in the synthetic text library. The matches can be in turn used to retrieve the corresponding electronic documents. It should be noted that a plurality of matches and corresponding electronic documents may be retrieves ranked by order according the similarity of the synthetic text.

Abstract: A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides a system, method, and computer program product for modifying tie strength between members of an existing network using captured digital images of documents. Documents associated with the captured images are recognized and other members associated with the document are determined. Using this information, ties between members or the network are modified.

Abstract: Techniques for performing a task while preserving the privacy or confidentiality of information used as input for the task. In one embodiment the task is broken down into smaller tasks (called subtasks or microtasks), which are then outsourced. The input information for each microtask is based upon and is generally a subset of the input information received for the task. The determination of microtasks for the task is performed in such a manner that constraints associated with the task are satisfied. For example, microtasks may be determined for the task based upon risk (e.g., the risk associated with the privacy or confidentiality of the input information being compromised as a result of the outsourcing), quality constraints (e.g., desired quality of the work product resulting from performance of the task), cost constraints, and other constraints associated with the job.

Abstract: Techniques for pricing a task are provided. The pricing may be based on attributes of the input information and one or more rules determined for the input information. The rules may be based on one or more variables and customer-specified constraints.

Abstract: A MMR system that uses multiple classifiers for predicting, monitoring, and adjusting index tables for image recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit includes a plurality of recognition unit and index table pairs corresponding to classifiers to be applied to received image queries, as well as an image registration unit for storing and monitoring performance data for the classifiers. The MMR matching unit receives the image query and identifies, using a classifier set, a result including a document, the page, and the location on the page corresponding to the image query. The present invention also includes methods for monitoring online performance of a multiple classifier image recognition system, for classifier selection and comparison, and for offline classifier prediction.

Abstract: A system and method for indexing and retrieval of document images in an MMR system having repeated content is described. The system provides one or more hierarchical shared content indices that produce faster and/or more accurate search results. The system is also advantageous because the number and configuration of the hierarchical shared content indices is automated, scalable and efficient for processing documents with partially repeated content. In particular, the MMR matching unit includes a hierarchical shared content index (HSCI) and associated methods of use for processing images where the MMR system includes repeated content.

Abstract: An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher integrated into a network with an advertiser, an ad broker, and an MMR service bureau. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR service bureau uses the result to retrieve advertising or other links associated with the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a differentially weighting links associated with an MMR document.

Abstract: The mobile device includes a client that has a number of modules, and the MMR Gateway and MMR matching unit are implemented as a server that has a number of modules. The implementation of the MMR system as a client and a server is advantageous because the modules may be distributed among the client and the server in a variety of configurations. The present invention includes a capture module, a preprocessing module, a feature extraction module, a retrieval module, a send message module, an action module, a prediction module, a feedback module, a sending module, an MMR database, a streaming module, an e-mail module, a voice recognition system and an audio database. These modules and systems are operational upon the client or the server. In one embodiment, the client includes only the capture module with the remaining modules operational on the server. In a second embodiment, the server includes the action module with the remaining modules operational on the client.

Abstract: An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit, and an MMR publisher. A computer includes a document authoring application, plug-in, and printer driver. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a method for capturing symbolic information from documents and for capturing printer dll functions.

Abstract: An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. Image tracking information also is provided for determining relative locations of images within a document page. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The present invention also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

Abstract: An MMR system for processing image queries across index tables with unequal priority comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a dispatcher, a plurality of recognition units, and index tables, as well as an image registration unit. In one embodiment, the system includes an MMR matching plug-in installed on the mobile device. The present invention also includes methods for processing image queries across index tables of unequal priority and updating a high priority index based on received or projected image queries.

Abstract: A MMR system that uses multiple classifiers for predicting, monitoring, and adjusting index tables for image recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit includes a plurality of recognition unit and index table pairs corresponding to classifiers to be applied to received image queries, as well as an image registration unit for storing and monitoring performance data for the classifiers. The MMR matching unit receives the image query and identifies, using a classifier set, a result including a document, the page, and the location on the page corresponding to the image query. The present invention also includes methods for monitoring online performance of a multiple classifier image recognition system, for classifier selection and comparison, and for offline classifier prediction.

Abstract: A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The image registration unit includes an indexing unit for generating images adapted to the environment and capabilities of the image capture device. The indexing unit also automatically adapts the configuration of the plurality of recognition units and index tables based upon image queries applied to the plurality of recognition and index tables. The plurality of recognition units and index tables are configured based on content they reference, recognition algorithm used or other factors.

Abstract: A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit also includes a result combiner coupled to each of the recognition units to receive recognition results. The result combiner produces a list of most likely results and associated confidence scores. This list of results is sent by the result combiner back to the MMR gateway for presentation on the mobile device. The result combiner uses the quality predictor as an input in deciding which results are best. The present invention also includes a number of novel methods including a method for generating the list of best results.

Abstract: A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR system also includes a quality predictor as a plug-in installed on the mobile device to filter images before they are included as part of a retrieval request or as part of the MMR matching unit. The quality predictor comprises an input for receiving recognition algorithm information, a vector calculator, a score generator and a scoring module. The quality predictor receives as inputs an image query, context information and device parameters, and generates an outputs a recognizability score. The present invention also includes a method for generating robustness features.

Abstract: A MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The mobile devices send retrieval requests including image queries and other contextual information. The MMR gateway processes the retrieval request from the mobile devices and then generates an image query that is passed on to the MMR matching unit. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit includes an image registration unit that receives new content from the MMR publisher and updates the index table of the MMR matching unit. A method for automatically registering images and other data with the MMR matching unit, a method for dynamic load balancing and a method for image-feature-based queue ordering are also included.

Abstract: An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit includes a segmenter for segmenting received images by content type, a distributor for distributing the images to corresponding content type index tables, and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The present invention also includes a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

Abstract: The present invention relies on the two-dimensional information in documents and encodes two-dimensional structures into a one-dimensional synthetic language such that two-dimensional documents can be searched at text search speed. The system comprises: an indexing module, a retrieval module, an encoder, a quantization module, a retrieval engine and a control module coupled by a bus. A number of electronic documents are first indexed by the indexing module and stored as a synthetic text library. The retrieval module then converts and input image to synthetic text and searches for matches to the synthetic text in the synthetic text library. The matches can be in turn used to retrieve the corresponding electronic documents. It should be noted that a plurality of matches and corresponding electronic documents may be retrieves ranked by order according the similarity of the synthetic text.