METHODS AND APPARATUS FOR BOUNDED IMAGE DATA ANALYSIS AND NOTIFICATION MECHANISM

Abstract

The present invention relates to a method and apparatus for preliminary optical character recognition document imaging and user notification on an electronic device with image capture capabilities. More specifically, this disclosure relates to an electronic device capable of connecting to a network, wherein a document image may be transmitting and stored securely in an external server.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the full benefit of U.S. Provisional Patent Application Ser. No. 62/032,543, filed Aug. 2, 2014, and titled “METHODS AND APPARATUS FOR BOUNDED IMAGE DATA ANALYSIS AND NOTIFICATION MECHANISM”, the entire contents of which are incorporated herein by reference.

FIELD OF INVENTION

This invention relates to methods and apparatus for processing an image captured via an image capture device. In some embodiments, the image capture device may be a mobile device and a processor in the mobile device may perform a preliminary scan and notify the user of the scan result. More particularly, the present invention relates to performing a preliminary bounded image data analysis on a document prior to transmission of the document to a server for continued remote processing, wherein the server may perform a complete characterization of the document and/or store the imaging results within a virtual cloud based image storage.

BACKGROUND

With the increasing use of smart phones equipped with cameras and integrated email and text communication platforms, people have a widespread convenience of electronically sending a picture or document from almost anywhere. However, in many cases a captured image of a document may not be legible when reprinted or processed at the messages destination. A sender may not realize that the document is not legible until it is too late to easily resend. Traditional methods of optical character recognition (OCR) of an entire document before sending is often impractical due to the significant processing power required to locally OCR each document.

In such examples, the necessary information may be generally minimal with easy-to-read fonts, wherein the image may be small and only specific text may be necessary to complete the action. Current applications are generally not targeted to process large, complex documents.

What is needed therefore is an apparatus and associated methods for timely, conveniently and efficiently capturing and notifying a user immediately whether the document can be properly analyzed transmitting electronically the image or document for remote processing.

SUMMARY

Accordingly, the present invention provides apparatus and methods for preliminary bounded image data analysis and notification mechanism on an electronic device with image capture capabilities.

The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. The accompanying drawings that are incorporated in and constitute a part of this specification, illustrate several embodiments of the invention and, together with the description, serve to explain the principles of the invention: Other features, objects, and advantages of the invention will be apparent from the description, drawings and the claims herein.

A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes an image feature processing apparatus including: an image capture device for capturing a set of image data; a preliminary image recognition mechanism operative to identify and analyze potential characteristics included in the set of captured image data, where preliminary image recognition includes—automated identification of a feature based upon at least partial recognition of the feature; and a notification mechanism configured to notify a user of a result of the preliminary image recognition as compared to predefined parameters. A notification may include no artificial delay in order to provide a prompt process. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Implementations may include one or more of the following features. One feature may include an example with an image feature processing apparatus where a preliminary image recognition mechanism provides optical character recognition identifying potential text characters from the image data, where preliminary optical character recognition includes one or both of: partial character recognition of potential text characters and character recognition of a portion of at least one text character. The example may also include a notification mechanism which may notify a user of a preliminary optical character recognition based on predefined parameters such as, but not limited to minimal font size, percent success of optical character recognition (OCR) attempts, total time to attempt an OCR as specified by the user for a document type captured in the image data. Other examples may include an image feature processing apparatus which may be capable of connecting to cloud storage and processing server. The image feature processing apparatus may include a mobile device. The image feature processing apparatus may include a digital camera. The image feature processing apparatus may also be configured to identify potential text characters from a document image and analyze the potential text characters with optical character recognition to provide a degree of character recognition.

There may be methods which also include transmitting the captured document image to a cloud server for storage of the document image on the cloud server. In some methods a transmission may be based on a result of analyzing the transmitted captured document image where the result includes a success result based upon recognition of one or both of a partial character recognition or a recognition of a portion of at least one text character. The method may further include steps of: transmitting the document image to an external server, where the external server is configured to identify the potential text characters from the document image and analyze the potential text characters with optical character recognition. The method may further include steps of: categorizing the document image based on predefined content criteria of the recognized text. The method may further include steps of: prompting a user to accept or reject the document image. The method may further include steps of prompting the user to input authorization information, where the authorization information is capable of granting access to a secure data storage system.

Some methods may have exemplary steps where a notification is performed and the notifying includes at least one of an audible notification, a visual notification, or a tactile notification. Methods may further include steps of: identifying borders of the document image. The method may also include recognizing an orientation of the at least one potential text character. In some methods, the notification of a fail may include prompting the user to select or capture an alternate document image. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

One general aspect may include methods of preliminary optical character recognition and notification, the methods may include steps of: identifying at least one potential text character from a captured document image, where an electronic device with image capture capabilities is configured to store the document image. The methods may include analyzing the at least one potential text character with preliminary optical character recognition, where the analyzing creates results based on an ability to recognize at least one text character, and where the preliminary optical character recognition includes one or both a partial character recognition or a character recognition of a portion of the at least one text character. Some methods may include communicating the results to a notification device configured to notify a user of the results based on predefined parameters of a fail result or a success result. The method of notifying a user may include at least one of an audible notification, a visual notification, or a tactile notification. Other aspects and examples of these methods may include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates exemplary embodiments of engaging a Bounded Image Data Analysis (BIDA) on a local system, wherein the local system comprises at least an image-capturing mechanism and a processor, are illustrated for an exemplary portable computing device.

FIG. 1B illustrates exemplary embodiments of engaging a Bounded Image Data Analysis (BIDA) on a local system, wherein the local system comprises at least an image-capturing mechanism and a processor, are illustrated for an exemplary portable computing device.

FIG. 2A illustrates exemplary embodiments of the BIDA capture system utilizing the image capturing mechanism on a local system.

FIG. 2B illustrates exemplary embodiments of the BIDA capture system utilizing the image capturing mechanism on a local system.

FIG. 3A illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 3B illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 3C illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 3D illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 3E illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 3F illustrates an exemplary step for utilizing a local system with BIDA capabilities and uploading a captured document.

FIG. 4A illustrates an exemplary step for developing a document image with recognizable specified features through use of overlap differential.

FIG. 4B illustrates an exemplary step for developing a document image with recognizable specified features through use of overlap differential.

FIG. 4C illustrates an exemplary step for developing a document image with recognizable specified features through use of overlap differential.

FIG. 4D illustrates an exemplary step for developing a document image with recognizable specified features through use of overlap differential.

FIG. 4E illustrates an exemplary step for developing a document image with recognizable specified features through use of overlap differential.

FIG. 5A illustrates an exemplary step for manually selecting an area of focus utilizing a BIDA image capture system on a mobile device with image capturing capabilities.

FIG. 5B illustrates an exemplary step for manually selecting an area of focus utilizing a BIDA image capture system on a mobile device with image capturing capabilities.

FIG. 5C illustrates an exemplary step for manually selecting an area of focus utilizing a BIDA image capture system on a mobile device with image capturing capabilities.

FIG. 5D illustrates an exemplary step for manually selecting an area of focus utilizing a BIDA image capture system on a mobile device with image capturing capabilities.

FIG. 5E illustrates an exemplary step for manually selecting an area of focus utilizing a BIDA image capture system on a mobile device with image capturing capabilities.

FIG. 6 illustrates exemplary method steps for utilizing a BIDA image capture system on an electronic device with image capturing and network connection capabilities.

FIG. 7A illustrates exemplary method steps for performing a BIDA image capture process.

FIG. 7B illustrates exemplary method steps for performing a BIDA image capture process.

FIG. 8 illustrates additional aspects of apparatus that may be used in some implementations of the present invention.

FIG. 9 illustrates a block diagram of a mobile device that may be used in some implementations of the present invention.

DETAILED DESCRIPTION

The present invention relates generally to a preliminary image processing system that analyzes a set of image data and characterizes the quality of the set of image data. In some preferred embodiments, the preliminary image processing system is incorporated into a mobile device such as a smart phone and includes image capture hardware and software and is further capable to communicate via a distributed communications network to a storage device, such as, for example a cloud document type storage apparatus. According to the present invention a set of image data is captured and processed to determine a threshold of image quality (TOIQ) prior to transmission of the set of image data to the storage apparatus. A threshold of image quality may include, for example, user defined parameters for a percent of success of optical character recognition, object image recognition, a potential for test character threshold, biometric recognition or other standard.

Glossary

• As used herein the following terms will have the following associated meaning:
• Bounded Image Data Analysis (BIDA): as used herein refers to an analysis of a captured image based on a predefined set of criteria, wherein a local system processes the analysis. The system-defined analysis comprises a preliminary feature quality analysis that may assess whether feature recognition may be possible with an acceptable threshold of image quality during continued remote processing.
• Feature Recognition: as used herein refers to a capability to isolate and identify specified features on an image, wherein the specified feature may depend on a type or category of image data.
• Examples of Feature Recognition may include: physical object, alphanumeric text recognition, barcode recognition, hash code recognition, feature recognition, biometric recognition, or other input recognition that may be captured as a two dimensional (2D) image on a three dimensional (3D) image derived from one of many known techniques of imaging.
• Specified Feature: as used herein refers to a component of an image that when processed may identify and categorize an image. For example, for a contract document, the specified feature may comprise alphanumeric characters and symbols. For a photograph, the specified features may comprise objects, including, for example, an article, person, device, structure, or combinations thereof. For some images, such as a driver's license, the specified features may comprise a combination of characters and objects utilized in facial recognition.
• Local System: as used herein refers to a device or system of devices that may process the BIDA without requiring logical communication with a network.
• Overlap Differential: as used herein refers to an image created through an amalgamation of multiple captured images of the same document, wherein feature recognition may not be possible on an isolated captured image but may be possible on the amalgamated image.
• Authenticated: as used herein refers to a state where the image capture and remote access rights have been verified by a predefined verification mechanism, including, for example, time stamp, user login, unique ID of image capture device, such as a IMEI, voice recognition, geolocation, fingerprint, or face recognition.
• Continued Remote Process: as used herein refers to a process separate from the BIDA, wherein the separate process occurs after receipt of an image from the local system over a network.
• Field of View: as used herein refers to the viewable boundaries of an image capturing device.
• Area of Focus: as used herein refers to a subset of boundaries within the field of view where the image capturing device may be capable of focusing text or images within the subset.
• Document Image: as used herein refers to a captured view of a document, wherein the captured view may comprise at least one specified feature.
• BIDA Image Capturing System: as used herein refers to an image capturing device component of the local system, which may be configured to capture images that may be compatible with the BIDA. In some embodiments, the BIDA image capturing system may automatically frame and focus the image capturing mechanism of the electronic device. Alternatively, the BIDA image capture system may comprise a manual mode, wherein a user may control the framing and focusing of the image capturing mechanism.

Referring now to FIGS. 1A-1B, exemplary embodiments illustrate some implementations of the present invention for engaging a BIDA on a local system, wherein the local system comprises at least an image-capturing mechanism and a processor on a local system 100 which may be an exemplary portable computing device, such as, for example a mobile smart phone. FIGS. 1A and 1B illustrate an exemplary embodiment of the BIDA capture system where the system may access and review images stored within a photograph album or other such file storage on the local system 100. FIG. 1A illustrates an image where the BIDA capture system may recognize specified features such as a line of text 121 on a document image 120. The user interface for file-based BIDA may include such features as an icon 140 to designate the local storage of images captured in file-based BIDA.

An ability to review the images in the storage location such as icons for proceeding through the images in a first direction at 154 or a reverse direction at 155. The image may be chosen at 165 or rejected at 160 as non-limiting examples of a user interface. A particular document image 120 may be represented where specified features 121 may include text that may be located where the text may have been successfully scanned. On a successful detection of character text during BIDA, a notification 130 may be provided to the user, such as, for example, one or more of: a vibration, a tone, a verification number certifying acceptance of the image at a defined quality level, and acceptance icon, or other indicator. A verification may include, for example, a universally unique identification number (UUID) that may be stored on one or both of the mobile device and the cloud document mechanism.

FIG. 1B illustrates a document image 125 where the BIDA capture system may not recognize a specified feature 126. In some embodiments, there may still be a notification of some kind including one or more of audible, visual, or tactile indicator. In other embodiments, the absence of a notification may be the indicator that the document image does not comprise recognizable features and does not pass a BIDA capture test, such as meeting a threshold of image quality. In some embodiments, a visual notification 131 may highlight the specified feature 126 that may not be recognizable at a certain distance from the camera, which may allow a user to decide to move the camera orientation and/or adjust lighting before taking a new image capture of the document or to accept the document image despite one or more unrecognizable specified features.

Referring now to FIGS. 2A and 2B, exemplary embodiments of the BIDA capture system utilizing the image capturing mechanism on a local system 200 for different configuration setup. FIG. 2A may illustrate a local system 200 where the BIDA capture system may not recognize a specified feature 221 within a document 220. This may be due to various factors including an inadequate focusing condition or the lack of textual data capable of an efficient OCR in the image captured. For example, as illustrated, the document 220 may be within the field of view 210 of the capture system but may be beyond the area of focus 211. Accordingly, the BIDA capture system may not notify the user of a successful scan, wherein the scan meets defined criteria.

FIG. 2B may illustrate an image capture where the BIDA capture system on the local system 200 may recognize the specified feature 221 on the document 220 and provide a notification 230 to the user, such as one or more of a visual, tactile, or audible indication. For example, the notification 230 may comprise a vibration or audio laying of a .wav file, which may articulate or announce the condition of the captured image. In some examples, an alternate notification may occur if feature recognition fails during a BIDA of a captured image.

In some embodiments, the BIDA capture system may engage the targeted document once the local system 200 captures the image. In some embodiments, the BIDA capture system may perform a continuous scan process when the image capturing device is actively capturing images. In such embodiments, the BIDA capture system may control the capturing mechanism and trigger the capture when text is recognized according to a threshold of image quality where a PFQA is successful. In still further embodiments, the BIDA may occur multiple times. For example, the BIDA may occur while the camera is active and then again after the image has been captured. Such combinations may allow the user to consistently capture readable images combined with a secondary confirmation scan. In some embodiments, the image capture device may capture a series of images in quick succession or extract images of a document from a video capture. The specific settings may be based on the image capturing capabilities of the local and/or user preferences.

Referring now to FIGS. 3A-3F, exemplary steps for utilizing a local system with BIDA capabilities and uploading a captured document are illustrated. FIG. 3A illustrates application icons, including a BIDA capture system icon 370, and a camera icon 375 on a portable computing device 300, wherein a user 380 may select the BIDA capture system icon 370. In some embodiments, at FIG. 3B the BIDA capture system may prompt a user to select between two image selection icons; a photograph album icon 340, and a camera icon 345. The user 380 may click a camera icon 345, which may initiate the image capturing mechanism on the mobile device, or the photograph album icon 340, which may access document images stored on the portable computing device 300.

At FIG. 3C, the user 380 may select a captured image 320 to send to a secure cloud based image storage facility over a network for continued remote processing. In some exemplary embodiments, the user 380 may cycle through captured images in the folder by clicking directional arrows 354, 355. The user 380 may confirm selection of the captured image 320 by selecting the remote secure cloud based image storage 365 icon. In some embodiments, at FIG. 3D, a user 380 may select the folder within the remote secure cloud based image storage to direct the storage of the captured image 320. In other embodiments, not shown, the BIDA may be capable of limited content analysis, wherein the local system may suggest folders to place image files.

At FIG. 3E, the user 380 may be prompted to respond to an authentication step. In some exemplary embodiments, the authentication step may be prompt to connect a dongle or chip to the portable computing device 300, which may be used to confirm the identity of the user 380. In alternate embodiments, the image capturing mechanism may flip and capture a facial image of the user 380. In still further embodiments, the authentication step may comprise a voice recognition, and the portable computing device 300 may be programmed to recognize specific words spoken by the user 380. The authentication step may comprise a fingerprint recognition or other biometric as available on some smart mobile devices. At FIG. 3F, the local system may transmit the captured image 320 over a network 390 to a remote secure cloud based image storage 395 for continued remote processing.

Referring now to FIGS. 4A-4E, exemplary steps are illustrated for developing a document image with recognizable specified features through use of overlap differential. At FIG. 4A, a BIDA image capturing device 400 may capture an image of a document 420 comprising three specified feature segments 421, 422, 423. The document 420 may be placed within the field of view 410 of the BIDA image capturing device but out of range of the area of focus 411. This step may allow the local system to recognize the boundaries of the document 420 but may not allow for specified feature 421, 422, 423 recognition.

At steps 4B-4D, the BIDA image capturing device 400 may capture multiple images of the document 420, wherein the recognizable specified features 421, 422, 423 may be different at each step. The images captured at 4B-4D capture focused and recognizable features 421, 422, 423, respectively. At 4E, the local system may process the multiple images utilizing overlap differential, which may develop a single document image 424 comprising completely recognizable features 421, 422, 423. The BIDA image capturing device 400 may present the amalgamated document image 424 and notify 430 the user that the document image 424 comprises recognizable specified features. Embodiments may include notification of recognizable specified features across an entire document or a portion thereof.

In some exemplary embodiments, the images captured in FIGS. 4A-4D may be captured through an automated series of image capture. In some alternate embodiments, the images captured may be extracted from a video segment filming the document. In some embodiments, the user may select multiple images and confirm that the images comprise a single document. In some exemplary embodiments, the BIDA may determine whether multiple images comprise a single document or multiple documents. In some such embodiments, the BIDA may identify the borders of a document and separate multiple documents from a single captured image. This embodiment may be particularly useful when the BIDA may extract the documents from a video segment filming one or more documents.

In BIDA for more complex images, such as three-dimensional object recognition, known techniques, such as simultaneous localization and mapping (SLAM), may be implemented in part within the local system and completed through remote continued processing.

Referring now to FIGS. 5A-5E, exemplary steps for manually selecting an area of focus are illustrated. At 5A, a BIDA image capturing device 500 may capture an image of a document 520 comprising three specified feature segments 521, 522, 523. The document 520 may be placed within the field of view 510 of the BIDA image capturing device but out of range of the area of focus 515. This step may allow the local system to recognize the boundaries of the document 520 but may not allow for specified feature 521, 522, 523 recognition. At 5B, the BIDA image capturing device 500 may present the document image 524 to the user 580. In other embodiments, multiple documents may be located within the field of view 510, and the local system may be capable of identifying multiple borders and isolating the documents.

In some embodiments at 5B, the user 580 may select a particular specified feature 523 deemed significant to the document, wherein the document image may be adequate as long as the selected specified feature may be recognizable. For example, a receipt may comprise the amounts and items of the sale, but it may also comprise an intricate logo for the store. A user may determine that the logo may not be a crucial piece of information for that receipt. Accordingly, the user may specify that the necessary specified features comprise the bottom text 523.

At step 5C, the BIDA image capturing device 500 may capture an image or be capable of capturing an image wherein unselected specified features 521 may be recognizable but the selected specified features 522 may be out of the field of view 510. At step 5D, the BIDA image capturing device 500 may capture or be in a position to capture a document image, wherein the document image may comprise recognizable specified features 522 that the user 580 may have selected at step 5B. In some embodiments, the feature recognition may trigger a notification 530 from the image capturing device. At step 5E, the document image 525 may be presented to the user 580, who may select the document image 525 for transmission to a secure cloud based image storage for continued remote processing.

Referring now to FIG. 6, an exemplary process 600 for utilizing a BIDA capture system may be illustrated. At 605, a user may activate the BIDA capture system. In some embodiments, the function may occur on a mobile device. In some embodiments, a user may manually activate the BIDA capture system, such as illustrated in FIG. 1. In some alternative embodiments, a user may activate the BIDA image capture system indirectly by operating a mobile device comprising the BIDA image capture system. For example, a digital camera may automatically initiate the BIDA image capture system when a user is taking photographs or otherwise capturing image data.

Steps 610, 620 illustrate method steps for utilizing the BIDA image capture system on existing images stored on the mobile device or on a memory card. At 610, the user may select an image file folder, and at 620 may select an image. Steps 615, 625 illustrate method steps for utilizing the BIDA image capture system with an image capturing function. At 615, the user may select the image capture function, and at 620, the user may capture an image. At 630, the user may wait for a confirmation indication from the BIDA image capture system. As described in reference to FIG. 1A and FIG. 1B, the BIDA image capture system may scan a document before the user captures its image at 625.

At 635, a user may receive no response or a negative response from the BIDA image capture system, which may prompt the user to restart the image selection process at 610, 620 or image capturing process 615, 625. Alternatively, at 640, a user may receive a confirmation indication. In some embodiments, a successfully recognized document may be automatically stored without requiring further input from a user. In other embodiments, the user may accept and store the document image at 645 or may reject the document at 650. When a document is stored at 645, the application may communicate with a storage system such as a server and a control application on the storage system, at 655, may verify the identification of the user associated with the image capture device or the application communicating the image data. In the process of verifying the image, further processing of the document image data may occur on the storage system for appropriate storage as a fully processed digital document. At 660, the user may exit the application directly or indirectly.

Referring now to FIG. 7A, exemplary method steps of a BIDA image capture process 700A are illustrated. At 701, a BIDA image capture system may capture a document image. In some embodiments, a user may select a previously captured document image from a file folder, such as a photograph album, stored within the electronic device. Some embodiments, such as illustrated in reference to FIGS. 1A and 1B, may allow a user to choose to capture a new document image or select from a previously captured document image.

At 702, the BIDA image capture system may identify the borders using one or more of many well known techniques such as contrast edge detection of the documents within the view, and at 703 the BIDA image capture system may identify potential characters. The steps at 702, 703 may provide framing and orientation cues, which may allow for more reliable BIDA capture. As described in reference to FIG. 1A and FIG. 1B, in some embodiments, the BIDA image capture system may control the image capturing mechanism of the electronic device, and such framing and focusing may be automatically performed, without requiring further input from the user. At 704, the BIDA image capture system may analyze at least a portion of the potential characters identified at 703.

At 705, the results of the analysis may be transmitted to a notification mechanism, wherein the notification mechanism for example an OCR processing algorithm which, may discern between and fail result and a success result based on predefined parameters. In some embodiments, a success result may comprise a complete text character recognition of at least a portion of the identified potential text characters, which may comprise a portion of the document image text characters. For example, the BIDA image capture system may identify specific text characters, such as those considered more difficult to recognize or more likely to allow full word recognition.

In other embodiments, a success result may comprise a partial text character recognition of at least a portion of the identified potential text characters, wherein predefined portions of text characters may be recognized. For example, analyzing a select portion of a string of text characters may be sufficient to establish legibility of a document image. In still further embodiments, the predefined parameters may be based on a combination of recognition factors.

In some embodiments, a preliminary optical character recognition system may transmit a fail result at 706, the notification mechanism may optionally notify the user at 707. At 708, a fail result may trigger a prompt to the user to select or capture an alternate document. Alternatively, in some embodiments the preliminary optical character recognition system may transmit a success result 709, and the notification mechanism may notify the user at 710. As discussed above, a notification may include one or more of audible, visual, or tactile notifiers, such as a vibration. In some embodiments, the notification may only occur with a success result at 710.

Where the preliminary optical character recognition system may transmit a success result at 709, the preliminary optical character recognition system may optionally prompt the user to accept or reject the document at 711, which may confirm that the user intended to select and/or capture the particularly document image. At 712, the preliminary optical character recognition system may verify access authorization of the electronic device, and at 713, the preliminary optical character recognition system may optionally prompt the user to provide responses authentication or security inquiries. Such security measures may be significant where the user intends to transmit the document image to a secure external server storage, such as a cloud based image storage system.

In some embodiments, at 714, a preliminary optical character recognition system may be capable of categorizing the document image. For example, based on a predefined classification system and recognition parameters, the preliminary optical character recognition system may be able to analyze identified characters and identify key words, phrases, logos, patterns of words, proximity of words to each other, or other criteria. The analysis may be used to discern between a purchased item, a page of text, a face, an object, an official government document, a receipt, a deed, a birth certificate, and a tax document. In some embodiments, the preliminary optical character recognition system may prompt the user to confirm or reject the categorization, allowing the user to specify the destination file folder or classification of the document image. At 715, the preliminary optical character recognition system may transmit the document image file and related OCR results to an external server, wherein the external server may be capable of a complete optical character recognition analysis. In some embodiments, such as a mobile phone or tablet, the electronic device may be capable of wirelessly connecting to the external server and transmitting the data. Alternately, in other embodiments such as with a digital camera, the electronic device may require a hard connection to a secondary device, such as a laptop computer, capable of connecting to a network system.

Referring now to FIG. 7B, similar to the above example relating to an object containing text, exemplary method steps of a BIDA object image capture process 700B are illustrated. At 721, a BIDA image capture system may capture an object image. In some embodiments, a user may select a previously captured object image from a file folder, such as a photograph album, stored within the electronic device. Some embodiments, such as illustrated in FIG. 1A and FIG. 1B, may allow a user to choose to capture a new object image or select from a previously captured object image.

At 722, the BIDA image capture system may identify the borders of objects within the view, and at 723 the BIDA image capture system may identify potential types of objects using one or more of many well known techniques such as contrast edge detection of the documents within the view. The steps at 722, 723 may provide framing and orientation cues, or contrast which may allow for more reliable BIDA capture. As described in reference to FIG. 1A and FIG. 1B, in some embodiments, the BIDA image capture system may control the image capturing mechanism of the electronic device, and such framing and focusing may be automatically performed, without requiring further input from the user. At 724, the BIDA image capture system may analyze at least a portion of the potential objects identified at 723.

At 725, the results of the analysis may be transmitted to a notification mechanism, wherein the notification mechanism may discern between and fail result and a success result based on predefined parameters. In some embodiments, a success result may comprise a complete object recognition of at least a portion of the identified potential object, which may comprise a portion of the object image. For example, the BIDA image capture system may identify specific object characteristics, such as those considered more difficult to recognize or more likely to allow full object recognition.

In other embodiments, a success result may comprise a partial object recognition of at least a portion of the identified potential object, wherein predefined portions of object may be recognized. For example, analyzing a select portion of a string of objects may be sufficient to establish recognition of an object within an image. In still further embodiments, the predefined parameters may be based on a combination of recognition factors.

In some embodiments, a preliminary object recognition system may transmit a fail result at 726, the notification mechanism may optionally notify the user at 727. At 728, a fail result may trigger a prompt to the user to select or capture an alternate object. Alternatively, in some embodiments the preliminary object recognition system may transmit a success result 729, and the notification mechanism may notify the user at 730. As discussed above, a notification may include one or more of audible, visual, or tactile notifiers, such as a vibration. In some embodiments, the notification may only occur with a success result at 730.

Where the preliminary object recognition system may transmit a success result at 729, the preliminary object recognition system may prompt the user to accept or reject the object at 731, which may confirm that the user intended to select and/or capture the particularly object image. At 732, the preliminary object recognition system may verify access authorization of the electronic device, and at 733, the preliminary object recognition system may prompt the user to provide responses to security inquiries. Such security measures may be significant where the user intends to transmit the object image to a secure external server storage, such as a cloud based image storage system.

In some embodiments, at 734, a preliminary object recognition system may be capable of categorizing the object image. For example, based on a predefined classification system and recognition parameters, the preliminary object recognition system may be able to analyze identified object characteristics and identify key words, phrases, logos, patterns, proximity of patterns to each other, or other criteria. At 735, the preliminary object recognition system may transmit the object image file and related recognition results to an external server, wherein the external server may be capable of a complete object recognition or character recognition analysis. In some embodiments, such as a mobile phone or tablet, the electronic device may be capable of wirelessly connecting to the external server and transmitting the data. Alternately, in other embodiments such as with a digital camera, the electronic device may require a hard connection to a secondary device, such as a laptop computer, capable of connecting to a network system.

Referring now to FIG. 8, additional aspects of controller hardware which may be included as computer hardware, useful for implementing the present invention may be illustrated as a block diagram that may include a controller 850 upon which an embodiment of the invention may be implemented. Controller 850 may include a bus 852 or other communication mechanism for communicating information, and a processor 854 coupled with bus 852 for processing information.

Controller 850 may also include a main memory 856, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 852 for storing information and instructions to be executed by processor 854. Main memory 856 may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 854. Controller 850 may further include a read only memory (ROM) 858 or other static storage device 860.

Controller 850 may be coupled via bus 852 to a display 862, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 864, including alphanumeric and other keys, or modes of input, such as, for example, a microphone and a radio frequency device such as Bluetooth, may be coupled to bus 852 for communicating information and command selections to processor 854. Another type of user input device may be a cursor control 866, such as a mouse, a trackball, a touchpad, touchscreen, or cursor direction keys for communicating direction information and command selections to processor 854 and for controlling cursor movement on display 862. This input device may typically have two or three degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane or stereo cameras that process and provide a third axis of input.

Some embodiments of the invention may be related to the use of controller 850 for setting operational parameters. According to one embodiment of the invention, control parameters may be defined and managed by controller 850 in response to processor 854 executing one or more sequences of one or more instructions contained in main memory 856. Such instructions may be read into main memory 856 from another computer-readable medium, such as storage device 860. Execution of the sequences of instructions contained in main memory 856 causes processor 854 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

The term “computer-readable medium” as used herein may refer to any medium that participates in providing instructions to processor 854 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, solid state devices (SSD) or magnetic disks, such as storage device 860. Volatile media may include dynamic memory, such as main memory 856. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 852. Transmission media may also take the form of infrared and radio frequency transmissions, acoustic or light waves, such as those generated during radio wave and infrared data communications.

Common forms of computer-readable media may include, for example, a memory stick, hard disk or any other magnetic medium, a CD-ROM, any other optical medium, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 854 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a distributed network such as the Internet. A communication device may receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector may receive the data carried in the infrared signal and appropriate circuitry can place the data on bus 852. Bus 852 may carry the data, or otherwise be in logical communication to the main memory 856, from which processor 854 retrieves and executes the instructions. The instructions received by main memory 856 may optionally be stored on storage device 860 either before or after execution by processor 854.

Controller 850 may also include a communication interface 869 coupled to bus 852. Communication interface 869 provides a two-way data communication coupling to a network link 870 that may be connected to a local network 872. For example, communication interface 869 may operate according to the internet protocol. As another example, communication interface 869 may be a local area network (LAN) card a data communication connection to a compatible LAN.

Network link 870 may typically provide data communication through one or more networks to other data devices. For example, network link 870 may provide a connection through local network 872 to a host computer 874 or to data equipment operated by an Internet Service Provider (ISP) 876. A wireless links may also be implemented. ISP 876 in turn may provide data communication services through the worldwide packet data communication network now commonly referred to as the “Internet” 879. Local network 872 and Internet 879 may both use electrical, electromagnetic or optical signals that carry digital data streams. The signals may be transmitted through the various networks and the signals on the network link 870 and through communication interface 869, which carry the digital data to and from controller 850 are exemplary forms of carrier waves transporting the information.

In some embodiments, Controller 850 may send messages and receive data, including program code, through the network(s), network link 870 and communication interface 869. In the Internet example, a server 890 might transmit a requested code for an application program through Internet 879, ISP 876, local network 872 and communication interface 869.

Processor 854 may execute the received code as it is received, and/or stored in storage device 860, or other non-volatile storage for later execution. Some exemplary controllers 850 may include a personal digital assistant, a mobile phone, a smart phone, a tablet, a netbook, a notebook computer, a laptop computer, a terminal, a kiosk or other type of automated apparatus. Additional exemplary devices may include any device with a processor executing programmable commands to accomplish the steps described herein.

FIG. 9 is a block diagram of some embodiments of a network access device that may include a mobile device 902. The mobile device 902 comprises an optical capture device 908 to capture an image and convert it to machine-compatible data, and an optical path 906, typically a lens, an aperture or an image conduit to convey the image from the rendered document to the optical capture device 908. The optical capture device 908 may incorporate a Charge-Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS) imaging device, or an optical sensor of another type.

A microphone 910 and associated circuitry may convert the sound of the environment, including spoken words, into machine-compatible signals. Input facilities 914 exist in the form of buttons, scroll-wheels or other tactile sensors such as touch-pads. In some embodiments, input facilities 914 may include a touchscreen display.

Visual feedback-to the user is possible through a visual display, touchscreen display, or indicator lights. Audible feedback 93.4 may come from a loudspeaker or other audio transducer. Tactile feedback may come from a vibrate module 936.

A motion sensor 938 and associated circuity convert the motion of the mobile device 902 into machine-compatible signals. The motion sensor 938 may comprise an accelerometer, which may be used to sense measurable physical acceleration, orientation, vibration, and other movements. In some embodiments the motion sensor 938 may include a gyroscope or other device to sense different motions.

A location sensor 940 and associated circuitry may be used to determine the location of the device. The location sensor 940 may detect Global Position System (GPS) radio signals from satellites or may also use assisted GPS where the mobile device may use a cellular network to decrease the time necessary to determine location. In some embodiments, the location sensor 940 may use radio waves to determine the distance from known radio sources such as cellular towers to determine the location of the mobile device 902. In some embodiments these radio signals may be used in addition to GPS.

The mobile device 902 comprises logic 926 to interact with the various other components, possibly processing the received signals into different formats and/or interpretations. Logic 926 may be operable to read and write data and program instructions stored in associated storage 930 such as RAM, ROM, flash, or other suitable memory. It may read a time signal from the clock unit 928. In some embodiments, the mobile device 902 may have an on-board power supply 932. In other embodiments, the mobile device 902 may be powered from a tethered connection to another device, such as a Universal Serial Bus (USB) connection.

The mobile device 902 also includes a network interface 916 to communicate data to a network and/or an associated computing device. Network interface 916 may provide two-way data communication. For example, network interface 916 may operate according to the internet protocol. As another example, network interface 916 may be a local area network (LAN) card allowing a data communication connection to a compatible LAN. As another example, network interface 916 may be a cellular antennae and associated circuitry which may allow the mobile device to communicate over standard wireless data communication networks. In some implementations, network interface 916 may include a Universal Serial Bus (USB) to supply power or transmit data. In some embodiments other wireless links may also be implemented.

As an example of one use of mobile device 902, a reader may scan some text from a newspaper article with mobile device 902. The text is scanned as a bit-mapped image via the optical capture device 908. Logic 926 causes the bit-mapped image to be stored in memory 930 with an associated time-stamp read from the clock unit 928. Logic 926 may also perform optical character recognition (OCR) or other post-scan processing on the bit-mapped image to convert it to text. Logic 926 may optionally extract a signature from the image, for example by performing a convolution-like process to locate repeating occurrences of characters, symbols or objects, and determine the distance or number of other characters, symbols, or objects between these repeated elements. The reader may then upload the bit-mapped image (or text or other signature, if post-scan processing has been performed by logic 926) to an associated computer via network interface 916.

As an example of another use of mobile device 902, a reader may capture some text from an article as an audio file by using microphone 99 as an acoustic capture port. Logic 926 causes audio file to be stored in memory 928. Logic 926 may also perform voice recognition or other post-scan processing on the audio file to convert it to text. As above, the reader may then upload the audio file (or text produced by post-scan processing performed by logic 926) to an associated computer via network interface 916.

A controller may include one or more of: personal computers, laptops, pad devices, mobile phone devices and workstations located locally or at remote locations, but in communication with the controller. System apparatus may include digital electronic circuitry included within computer hardware, firmware, software, or in combinations thereof. Additionally, aspects of the invention may be implemented manually.

Apparatus of the invention may be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a programmable processor and method actions can be performed by a programmable processor executing a program of instructions to perform functions of the invention by operating on input data and generating output. The present invention may be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. Each computer program may be implemented in a high-level procedural or object oriented programming language, or in assembly or machine language if desired, and in any case, the language can be a compiled or interpreted language. Suitable processors may include, by way of example, both general and special purpose microprocessors.

Generally, a processor may receive instructions and data from a read-only memory and/or a random access memory. Generally, a computer may include one or more mass storage devices for storing data files; such devices include Solid State Disk (SSD), magnetic disks, such as internal hard disks and removable disks magneto-optical disks and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as, internal hard disks and removable disks; magneto-optical disks; and CD ROM disks may be included. Any of the foregoing may be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

In some embodiments, implementation of the features of the present invention may be accomplished via digital computer utilizing uniquely defined controlling logic, wherein the controller includes an integrated network between and among the various participants in Process Instruments.

The specific hardware configuration used may not be particularly critical, as long as the processing power is adequate in terms of memory, information updating, order execution, redemption and issuance. Any number of commercially available database engines may allow for substantial account coverage and expansion. The controlling logic may use a language and compiler consistent with that on a CPU included in the controller. These selections may be set according to per se well-known conventions in the software community.

The present invention is described herein with reference to block diagrams and functional illustrations of methods and apparatus to implement various aspects of the present invention. It is understood that each block of the block diagrams or operational illustration or function represented, and combinations of blocks in the block diagrams or operational or functional illustrations, may be implemented by automated apparatus, such as analog or digital hardware and computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, cellular device, smart device, ASIC, or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implements the functions/acts specified in the block diagrams or operational block or blocks. In some implementations, the functions or method steps described in relation to the blocks or functional representations may occur in an order other than the order noted or described herein, For example, blocks or functional representations shown in a succession may be executed substantially concurrently or the blocks in an alternate order, depending upon a specific implementation of the present invention. It is therefore understood that unless otherwise specifically noted and thereby limited, the discussion here is presented in an order to facilitate enablement and understanding and is not meant to limit the invention disclosed.

CONCLUSION

A number of embodiments of the present invention have been described. While this specification contains many specific implementation details, there should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of the present invention.

Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in combination in multiple embodiments separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous.

Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order show, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the claimed invention.

Claims

1. An image feature processing apparatus comprising:

an image capture device for capturing a set of image data;

a preliminary image recognition mechanism operative to identify and analyze potential characteristics included in the set of captured image data, wherein preliminary image recognition comprises automated identification of a feature based upon at least partial recognition of the feature; and

a notification mechanism configured to notify a user, without artificial delay, of a result of the preliminary image recognition as compared to predefined parameters.

2. The image feature processing apparatus of claim 1 wherein the preliminary image recognition mechanism provides optical character recognition identifying potential text characters from the image data, wherein preliminary optical character recognition comprises one or both of: partial character recognition of potential text characters and character recognition of a portion of at least one text character; and

the notification mechanism notifies a user of a preliminary optical character recognition based on predefined parameters as specified by the user for a document type captured in the image data.

3. The image feature processing apparatus of claim 1, wherein the image capture device is capable of connecting to cloud storage and processing server.

4. The image feature processing apparatus of claim 3, wherein the image capture device comprises a mobile device.

5. The image feature processing apparatus of claim 3, wherein the image capture device comprises a digital camera.

6. The image feature processing apparatus of claim 3, wherein the preliminary image recognition mechanism is configured to identify potential text characters from a document image and analyze the potential text characters with optical character recognition to provide a degree of character recognition.

7. A method of preliminary optical character recognition and notification, the method comprising the method steps of:

identifying at least one potential text character from a captured document image, wherein an electronic device with image capture capabilities is configured to store the document image;

analyzing the at least one potential text character with preliminary optical character recognition, wherein the analyzing creates results based on an ability to recognize at least one text character, and wherein the preliminary optical character recognition comprises one or both a partial character recognition or a character recognition of a portion of the at least one text character; and

transmitting the results to a notification device configured to notify a user of the results based on predefined parameters of a fail result or a success result.

8. The method of claim 7 further comprising the method steps of transmitting the captured document image to a cloud server for storage of the document image on the cloud server.

9. The method of claim 8, wherein a transmission is based a result of analyzing wherein the result comprises a success result based upon recognition of one or both the partial character recognition or the recognition of the portion of the at least one text character.

10. The method of claim 9, further comprising the method steps of:

transmitting the document image to an external server, wherein the external server is configured to identify the potential text characters from the document image and analyze the potential text characters with optical character recognition.

11. The method of claim 7, wherein the notifying comprises at least one of an audible notification, a visual notification, or a tactile notification.

12. The method of claim 7 further comprising the method steps of:

identifying borders of the document image; and

recognizing an orientation of the at least one potential text character.

13. The method of claim 10 further comprising the method steps of:

categorizing the document image based on predefined content criteria of the recognized text.

14. The method of claim 10 further comprising the method steps of:

prompting the user to accept or reject the document image.

15. The method of claim 10 further comprising the method steps of prompting the user to input authorization information, wherein the authorization information is capable of granting access to a secure data storage system.

16. The method of claim 7 wherein the notification of a fail results comprises prompting the user to select or capture an alternate document image.