SYSTEMS AND TECHNIQUES FOR IMPROVING CLASSIFICATION OF IMAGE DATA USING LOCATION INFORMATION
In one embodiment, a method includes receiving, at a mobile device, an image depicting a document; attempting to classify, using a processor of the mobile device, the document depicted in the image to one of a plurality of predetermined document classes, wherein attempting to classify the document results in an ambiguous classification result; determining, using the mobile device, location information identifying a geographic location of the mobile device at a particular time; and disambiguating, using the processor of the mobile device, the ambiguous classification result based on the location information. Exemplary systems and computer program products are also described.
This application is a continuation of U.S. patent application Ser. No. 14/177,136, filed Feb. 10, 2014, which claims priority to Provisional U.S. Patent Application No. 61/815,210, filed Apr. 23, 2013, which are herein incorporated by reference in their entirety.
RELATED APPLICATIONSThis application is related to copending U.S. patent application Ser. No. 13/740,123, filed Jan. 11, 2013; U.S. Pat. No. 6,370,277, granted Apr. 9, 2002 (U.S. patent application Ser. No. 09/206,753, filed Dec. 7, 1998); Ser. No. 13/802,226, filed Mar. 13, 2013 and Provisional U.S. Patent Application No. 61/780,747, filed Mar. 13, 2013, each of which are also herein incorporated by reference in its entirety.
FIELD OF INVENTIONThe present invention relates to software development, and more particularly to software development platforms for generating and/or modifying applications for use on a mobile device.
BACKGROUND OF THE INVENTIONMobile technology is rapidly developing, fueling even more rapid development of software capable of exploiting the new and expanded functionality offered by mobile devices. As a result, entire new development communities have arisen to contribute to the expanding pool of software tools available to mobile device users. Large producers of mobile hardware and software have even released software development platforms to the general public and/or provided access to application distribution services to select developers, e.g. via a registration process. To users' great benefit, a wide array of mobile applications designed to perform myriad activities using mobile devices are now readily available for quick download, installation and implementation via mobile communication networks.
The currently available mobile software applications and development platforms provide diverse and powerful functionality to both end-users and developers capable of leveraging the various capabilities of existing mobile devices. However, there is currently no tool available for dynamically developing and employing smart mobile applications capable of adapting to user behavior and/or requirements.
One field that has emerged with the ascendance of mobile technology is the so-called “location-based services” industry. These services typically leverage a conventional location determination technique (such as cellular location techniques (e.g. techniques estimating location based on identifying a cellular tower to which the device is connected/with which the device is in communication) or a global positioning system (GPS) to identify a user's location and in response to determining the user is present at a particular location, providing location-specific information to the user.
The typical example of location-based services is the use of location information to present customized advertising to a user based on the user's location. For example, a conventional location-based advertising service displays advertisements to a user based on the user's location being proximate to one or more places of commerce, as indicated by the GPS or cellular tower-based location techniques. Based on the user's location being proximate to the retail entity's address, the user's smartphone displays advertising information for the retailer (e.g. via a prompt such as a “push” notification, an email, etc. or by invoking a web page in a browser or application interface).
However, location information has been utilized in a limited fashion, merely utilizing the GPS and/or cellular tower-based location information as a stimulus to evoke a responsive advertisement, for example displaying the aforementioned prompt in response to determining a user's proximity to a particular location. These applications are simplistic in scope, and neither appreciate nor leverage location information to investigate, influence, and/or define the evolution of a business process, particularly as embodied by dynamic workflows.
In addition, conventional location based services have failed to leverage the interconnected nature of mobile devices to determine location information using techniques other than the conventional GPS and/or cellular tower-based approaches.
Accordingly, it would be beneficial to provide techniques, systems and computer program products capable of leveraging location data ubiquitously available to modern mobile devices in a manner that comprehensively addresses the complexity of various nuances of initiating, supervising, reviewing and/or executing operations and/or complete business workflows.
BRIEF SUMMARYIn one embodiment, a method includes receiving, at a mobile device, an image depicting a document; attempting to classify, using a processor of the mobile device, the document depicted in the image to one of a plurality of predetermined document classes, wherein attempting to classify the document results in an ambiguous classification result; determining, using the mobile device, location information identifying a geographic location of the mobile device at a particular time; and disambiguating, using the processor of the mobile device, the ambiguous classification result based on the location information.
In another embodiment, a computer program product includes a computer readable medium having embodied therein computer readable program instructions executable by a mobile device to cause the mobile device to perform a method. The method includes receiving, at a mobile device, an image depicting a document; attempting to classify, using a processor of the mobile device, the document depicted in the image to one of a plurality of predetermined document classes, wherein attempting to classify the document results in an ambiguous classification result; determining, using the mobile device, location information identifying a geographic location of the mobile device at a particular time; and disambiguating, using the processor of the mobile device, the ambiguous classification result based on the location information.
Other aspects and embodiments of the present invention will become apparent from the following detailed description, which, when taken in conjunction with the drawings, illustrate by way of example the principles of the invention.
The following description is made for the purpose of illustrating the general principles of the present invention and is not meant to limit the inventive concepts claimed herein. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations.
Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including meanings implied from the specification as well as meanings understood by those skilled in the art and/or as defined in dictionaries, treatises, etc.
It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless otherwise specified.
General Workflow Concepts
The present descriptions set forth novel and useful techniques and technologies configured to leverage the emerging advances in mobile, image capture, image analysis, and location-based services and technologies. These disclosures present exemplary and novel implementations from the perspective of users of mobile devices conducting various business processes. The business processes are wholly or partially embodied as workflows, executable and/or capable of being interfaced with via a mobile device. The workflows also uniquely leverage image capture/processing and location-based aspects of mobile technology to enhance the user experience with respect to generating, managing, and performing the workflow(s).
A user experience for mobile smart application development includes workflows, as well as any constituent operations forming the workflows (e.g. activities and rules), and associated systems, tools, or techniques relating to creation, performance, and/or management of the workflows. Preferably, the user experience, workflows (e.g. activities and/or rules), etc. are embodied as a mobile application, and may be based in whole or in part on techniques, technology, and/or concepts disclosed in related Provisional U.S. Application No. 61/815,210, filed May 30, 2013.
For example, in one embodiment mobile applications configured to initiate, facilitate, or conduct portions of and/or complete workflows in the context of the present application may be considered to encompass the following general scenario.
A user defines a workflow as a set of activities and rules. The workflow executes by moving from one activity to another in a fixed order or a by a dynamic order as determined by stimuli. Rules are applied at fixed points within the sequence or in response to stimuli. The user also designs UIs independently, with assistance from the development platform, or UIs are rendered automatically by development platform tools to associate with activities that require human interaction.
The workflow, via activities, rules and UI definitions, defines a mobile user experience which provides both the mobile UI and also the application behavior. The process definition can describe the application behavior because the mobile development platform exposes a federated view of native mobile services and server services. The process executes and transparently orchestrates the execution of native code directly on the device and remote code that resides on a server.
In one embodiment, a user launches a mobile application. The application initiates the process, takes the first activity and renders the defined UI. The user interacts with the UI and completes the activity or provides stimuli, such as “clicking” a UI button. At this point a rule may be executed or the next activity may be taken/performed. In either case local native services may be accessed, such as the device location being retrieved from the OS or a server service, such as a database lookup, may be used. This provision of native and/or remote services is transparent to the user.
The mobile application may be downloaded in whole or in part and/or run in real-time on the mobile device. For example, the application may be maintained in an online repository. An instance of the mobile application may be transferred to the mobile device automatically, in response to a user request, in response to a new release of the mobile application becoming available in the online repository, etc. In a preferred embodiment, transferring new instances of the mobile application to mobile devices and instantiating those new instances is a process that occurs transparently to the user and without requiring any interaction or instruction from end-users operating the mobile application on mobile devices.
Of course, other equivalent forms of the presently described workflows and implementations thereof on mobile devices are also to be understood as falling generally within the scope of the present descriptions according to the understanding that would be achieved by one having ordinary skill in the art based on reviewing the instant application.
Mobile Image Capture and Processing
As discussed herein, “image processing” (and particularly image processing using a mobile device) should be understood to optionally include any of the techniques and/or technology disclosed in related U.S. patent application Ser. No. 13/740,123 filed Jan. 11, 2013.
These techniques include but are not limited to capturing image data using a mobile device, image processing algorithms configured to improve image quality, and particular with respect to images of documents. For example, “image processing” in various embodiments may include one or more of page detection, rectangularization, skew correction, color conversion (e.g. from 24-bit RGB color to 8-bit grayscale, 1-bit bitonal, or other color depth representation as understood by those having ordinary skill in the art), resolution estimation, illumination correction, blur detection, etc. as disclosed in related U.S. patent application Ser. No. 13/740,123, and/or as would be understood by one having ordinary skill in the art upon reading the present descriptions.
While the present descriptions are offered primarily with reference to exemplary embodiments utilizing image data captured and/or in the form of still images, e.g. digital photographs, the skilled artisan reading the present descriptions will also understand that these principles apply equally to the use of digital video data. Of particular interest are techniques and/or technologies configured to capture and/or process video data using a mobile device. Further information describing the capture and processing of video data as a source of information may be reviewed in the disclosures presented in related Provisional U.S. Patent Application No. 61/819,463, filed May 3, 2013, which is herein incorporated by reference.
In addition, and according to various embodiments, the presently disclosed methods, systems and/or computer program products may utilize and/or include any of the classification and/or data extraction functionalities disclosed in related U.S. patent application Ser. No. 13/802,226, filed Mar. 13, 2013 and Provisional U.S. Patent Application No. 61/780,747, filed Mar. 13, 2013.
With the foregoing relation to associated inventive concepts in the field of mobile technology, mobile image processing, and workflow management, exemplary inventive principles of the presently disclosed workflows utilizing location-based services, location information, etc. may be understood to include the following general embodiments.
General Embodiments of Location-Based Workflows
In one general embodiment, a method includes initiating a workflow; performing at least one operation within the workflow using a processor of a mobile device; receiving location information pertaining to the workflow; and influencing at least a portion of the workflow based on the location information.
In another general embodiment, a computer program product includes a computer readable storage medium having embodied thereon computer readable program instructions configured to cause a processor to: initiate a workflow; perform at least one operation within the workflow using a processor of a mobile device; receive location information pertaining to the workflow; and influence at least a portion of the workflow based on the location information, wherein the workflow is configured to facilitate a business process.
In yet another general embodiment, a system includes a processor and logic in and/or executable by the processor. The logic is configured to cause the processor to: initiate a workflow; perform at least one operation within the workflow using a processor of a mobile device; receive location information pertaining to the workflow; and influence at least a portion of the workflow based on the location information, wherein the workflow is configured to facilitate a business process.
In still yet another general embodiment, a method includes initiating a workflow; and performing at least one operation within the workflow using a processor of a mobile device. The workflow operations include: detecting a location trigger condition; and at least partially in response to detecting the location trigger condition: determining location information corresponding to the mobile device; prompting a user to capture an image of a document; capturing the image via the mobile device; associating the location information with the captured image as location metadata; and storing the location metadata and the captured image to a memory of the mobile device. The method further includes facilitating a business process relating to the workflow, the facilitating being based on the location metadata.
In various approaches, the presently disclosed systems, methods and/or computer program products may be advantageously applied to one or more of the use methodologies and/or scenarios disclosed in the aforementioned related patent applications, among others that would be appreciated by one having ordinary skill in the art upon reading these descriptions.
It will further be appreciated that embodiments presented herein may be provided in the form of a service deployed on behalf of a customer to offer service on demand.
The description herein is presented to enable any person skilled in the art to make and use the invention and is provided in the context of particular applications of the invention and their requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Computer Implementation and Network Communications
In particular, various embodiments of the invention discussed herein are implemented using the Internet as a means of communicating among a plurality of computer systems. One skilled in the art will recognize that the present invention is not limited to the use of the Internet as a communication medium and that alternative methods of the invention may accommodate the use of a private intranet, a Local Area Network (LAN), a Wide Area Network (WAN) or other means of communication. In addition, various combinations of wired, wireless (e.g., radio frequency) and optical communication links may be utilized.
The program environment in which one embodiment of the invention may be executed illustratively incorporates one or more general-purpose computers or special-purpose devices such hand-held computers. Details of such devices (e.g., processor, memory, data storage, input and output devices) are well known and are omitted for the sake of clarity.
It should also be understood that the techniques of the present invention might be implemented using a variety of technologies. For example, the methods described herein may be implemented in software running on a computer system, or implemented in hardware utilizing one or more processors and logic (hardware and/or software) for performing operations of the method, application specific integrated circuits, programmable logic devices such as Field Programmable Gate Arrays (FPGAs), and/or various combinations thereof. In one illustrative approach, methods described herein may be implemented by a series of computer-executable instructions residing on a storage medium such as a physical (e.g., non-transitory) computer-readable medium. In addition, although specific embodiments of the invention may employ object-oriented software programming concepts, the invention is not so limited and is easily adapted to employ other forms of directing the operation of a computer.
The invention can also be provided in the form of a computer program product comprising a computer readable storage or signal medium having computer code thereon, which may be executed by a computing device (e.g., a processor) and/or system. A computer readable storage medium can include any medium capable of storing computer code thereon for use by a computing device or system, including optical media such as read only and writeable CD and DVD, magnetic memory or medium (e.g., hard disk drive, tape), semiconductor memory (e.g., FLASH memory and other portable memory cards, etc.), firmware encoded in a chip, etc.
A computer readable signal medium is one that does not fit within the aforementioned class of computer readable storage media. For example, illustrative computer readable signal media communicate or otherwise transfer transitory signals within a system, between systems e.g., via a physical or virtual network, etc.
In use, the gateway 101 serves as an entrance point from the remote networks 102 to the proximate network 108. As such, the gateway 101 may function as a router, which is capable of directing a given packet of data that arrives at the gateway 101, and a switch, which furnishes the actual path in and out of the gateway 101 for a given packet.
Further included is at least one data server 114 coupled to the proximate network 108, and which is accessible from the remote networks 102 via the gateway 101. It should be noted that the data server(s) 114 may include any type of computing device/groupware. Coupled to each data server 114 is a plurality of user devices 116. Such user devices 116 may include a desktop computer, lap-top computer, hand-held computer, printer or any other type of logic. It should be noted that a user device 111 may also be directly coupled to any of the networks, in one embodiment.
A peripheral 120 or series of peripherals 120, e.g., facsimile machines, printers, networked and/or local storage units or systems, etc., may be coupled to one or more of the networks 104, 106, 108. It should be noted that databases and/or additional components may be utilized with, or integrated into, any type of network element coupled to the networks 104, 106, 108. In the context of the present description, a network element may refer to any component of a network.
According to some approaches, methods and systems described herein may be implemented with and/or on virtual systems and/or systems which emulate one or more other systems, such as a UNIX system which emulates a MAC OS environment, a UNIX system which virtually hosts a MICROSOFT WINDOWS environment, a MICROSOFT WINDOWS system which emulates a MAC OS environment, etc. This virtualization and/or emulation may be enhanced through the use of VMWARE software, in some embodiments.
In more approaches, one or more networks 104, 106, 108, may represent a cluster of systems commonly referred to as a “cloud.” In cloud computing, shared resources, such as processing power, peripherals, software, data processing and/or storage, servers, etc., are provided to any system in the cloud, preferably in an on-demand relationship, thereby allowing access and distribution of services across many computing systems. Cloud computing typically involves an Internet or other high speed connection (e.g., 4G LTE, fiber optic, etc.) between the systems operating in the cloud, but other techniques of connecting the systems may also be used.
The workstation shown in
The workstation may have resident thereon an operating system such as the Microsoft Windows® Operating System (OS), a MAC OS, a UNIX OS, etc. It will be appreciated that a preferred embodiment may also be implemented on platforms and operating systems other than those mentioned. A preferred embodiment may be written using JAVA, XML, C, and/or C++ language, or other programming languages, along with an object oriented programming methodology. Object oriented programming (OOP), which has become increasingly used to develop complex applications, may be used.
Location Information
As utilized herein, the term “location information” should be understood to encompass any known form, format, or expression of physical location of a person or object. Location information, in at least some embodiments, should also be understood to encompass data from which a physical location may be determined, either in whole or in part based on the data alone or in combination with additional data including but not limited to additional location information data and/or metadata associated with location information data.
In one embodiment, location information includes one or more of discrete location information and continuous location information.
Discrete location information may include any type of location information that relates a precise physical location (e.g. an address, a set of GPS coordinates, etc.) to a precise time (e.g. a time of day in hours, minutes, and optionally seconds, such as “2:40 PM” “14:40”, “2:40:32 PM,” or any other definite time point (as opposed, for example, to a span of time ranging from several minutes to several hours). Conversely, continuous location information should be understood to include real-time or near real-time tracking data (e.g. telemetry data) from which location information over a duration of time may be described. Continuous location information may, in some approaches, comprise a plurality of discrete location information data.
For example, in preferred implementations, discrete location information may include a “snapshot” of a person's location at a given time. The snapshot preferably follows or is associated with a request to capture or gather location information or other information in response to taking a particular action within or relating to a workflow.
Similarly, continuous location information preferably comprises a series of measurements relating to a user's physical location over a period of time. Continuous location information may preferably be represented by a vector of motion or a plurality of vectors describing movement throughout a physical environment from a “start” point to an “end” point over a particular duration of time.
Of course, it should be understood that the present descriptions are inclusive of any type of “location information” that would be comprehended by a skilled artisan reading this disclosure. The exemplary forms of “location information” provided herein are offered for illustrative purposes only and should not be considered limiting on the scope of inventive concepts now disclosed.
Moreover, the instant descriptions are made primarily with reference to location information as a discrete point in physical space, such as a physical address or measurement of physical distance with respect to a reference point having a known location (e.g. physical proximity to a particular cell tower or network hub, a router, another mobile device, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions). However, it should be understood that “location information” includes any information suitable to convey a physical location with sufficient specificity to conduct one or more workflow(s) and/or corresponding operations, in a multitude of approaches.
Accordingly, in various embodiments location information may be expressed with varying levels of specificity. For example, precise location information may be provided in the form of GPS coordinates, and the GPS coordinates may be specified within a desired level of precision, such as within one ten-thousandth of a degree, or +0.0001 degrees latitude/longitude, etc. Additionally and/or alternatively, precise location information may include part or all of an address (e.g. a street address, a room or suite number within a particular street address, etc.).
In alternative embodiments, precise location information may also identify a specific device to which a primary user's mobile device (or primary mobile device) is connected, e.g. a router, wireless “hotspot” (which may include other mobile devices), etc.; a specific cellular tower; or any other suitable expression of location that would be understood by one having ordinary skill in the art upon reading the present descriptions.
Similarly, and in contrast to the “precise” location information defined above, relatively “imprecise” location information may be utilized in additional and/or alternative approaches. For example, “less precise” location information may include high-level address information such as identifying a state or territory where a user is located, identifying a city, town or other municipality within which the user is located, a zip code; a plurality or set of candidate addresses (e.g. a particular block of a city street, an intersection of two nearby streets); etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions). “Less precise” location information may additionally and/or alternatively include identification of an Internet or network service provider (ISP), preferably in combination with one or more forms of high-level address information such as described above.
In more approaches, a workflow that relies upon a user's presence in a particular state may only require relatively less precise location information than the specific street address, GPS coordinates, cell tower or network to which the user's device is connected. To maximize the efficiency of data processing and communication while minimizing the workflow's use and consumption of device resources, a location-based service implemented as a workflow may accordingly seek or utilize only a requisite level of specificity location information, even where more specific information than necessarily required is available.
For example, in one embodiment a user's location may be determined with sufficient specificity based on knowledge of less precise location information, such as a state, county, township, zip code, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Location information may also include data collected from one or more components of the mobile device. The collected data may include the location information, or information from which a location may be determined. For example, in one embodiment the data are collected from a GPS antenna of the mobile device, and comprise location information in the form of GPS coordinates.
Accordingly, in the context of the presently disclosed inventive concepts, location information may be obtained and/or determined using any suitable technique that would be appreciated by one having ordinary skill in the art upon reading the present descriptions. Several exemplary techniques and approaches to acquiring and managing location information are described below, and should be understood as merely providing illustrative approaches that do not limit the scope of the present disclosures.
Acquiring and Determining Location Information
Those having ordinary skill in the art will appreciate upon reading the present descriptions that location information as described above may be leveraged to facilitate, enable, and/or conduct a wide variety of operations relating to business workflows. As detailed further below, location information may be acquired, determined, and/or managed according to a diverse number of possible approaches.
As understood herein, location information should be understood to encompass any expression of physical location, whether “acquired” in a state and/or format directly useful to an associated workflow or underlying business process or “determined” using source data not in a state/format directly useful to the workflow or business process. While location information may be acquired or determined, it should be understood that the presently disclosed inventive concepts apply equally to all sources of location information, as well as techniques practiced using any type of location information.
Location information may be determined and/or acquired using any suitable or known technique, service, technology, or combination thereof that would be understood by one having ordinary skill in the art of location services.
Similarly, in the course of a workflow as described herein manipulating/managing location information may include determining a location corresponding to the provided location information. For example, while location information may be provided in one form, this form may not be acceptable or intelligible to the workflow or user, and so the location information may be converted to the acceptable format. The determination is preferably based on either the acquired location information or a combination of acquired location information and additional data such vector of motion data as described above.
In one illustrative embodiment a user's mobile device includes functional components and/or instructions configured to determine a location of the mobile device. The location determination may be accomplished using any known mechanism, technique, or combination thereof as would be understood by one having ordinary skill in the art. Exemplary location determination approaches include global system for mobile communication (GSM) based approaches (for example cellular tower-based location techniques), global positioning system (GPS) based approaches, two-dimensional triangulation (also known as “control plane location” e.g. emergency 911 or “E-911” radio signal delay-based location techniques), and/or network-based approaches, in various embodiments.
As noted above, location determination may include one or more network-based location approaches, which should be understood to generally include techniques that associate a physical location with a network identification. For example, in one embodiment a network-based location approach includes associating a physical address where a network router is present with one or more unique identifiers for client and/or host devices connected to the network (e.g. IP address, MAC address, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions). The approach utilizes the network identification in a manner sufficient to report a device location, e.g. the physical address where a router is installed/operating.
Location determination may additionally and/or alternatively include soliciting and/or utilizing human-reported location information. Human-reported location information may be obtained from any source of human input or reporting such as described above.
In one embodiment, location information may be acquired automatically, or semi-automatically, from one or more data sources and/or mobile device components, e.g. in response to detecting a trigger condition being satisfied. For example, automated or semi-automated location information acquisition may include, preferably in the course of conducting a workflow, querying one or more components of the mobile device for location information. Additionally and/or alternatively location information may be obtained from additional data source(s), such as from an image of a document, from a remote database, from another mobile device in communication with the mobile device being located, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
If conducted in the course of a workflow, the acquisition may optionally occur in response to and/or in connection with performing a capture operation wherein image data are captured using the mobile device. The workflow may additionally and/or alternatively query a memory of the mobile device to retrieve previously determined/obtained location information stored therein, in more approaches.
In more embodiments, location information may be obtained from or in connection with user input. Generally, human-reported location information is to be understood as including any form of location information generated a human.
For example, in one embodiment human reporting of location information may correspond to a user inputting location information to the mobile device or another device, which may or may not itself be considered a “mobile” device. The location information may be input presently, e.g. in response to a displayed prompt, or may have been previously input and stored for future use, e.g. as metadata in association with a captured image.
Location information may be input by the user in any suitable manner, including by providing tactile, auditory or visual input such as a “typed” address, a spoken name of an institution or place of business, or image data depicting a physical location, respectively. Image data depicting a physical location preferably include one or more landmarks capable of serving as a reference point in determining location.
For example, in one approach a user is visiting a museum (e.g. an art gallery, a history museum, a scientific museum, zoo, aquarium, etc.), historical site (e.g. national sites such as the White House, Capitol Building, a battlefield, harbor, monument, etc.) library, or any other site characterized by an organized locational structure preferably having one or more unique landmarks distributed throughout wishes to receive directions from the user's current location to a location of interest, e.g. a room where an exhibit of interest is currently displayed. The user, preferably via a workflow, captures an image of a unique landmark proximate to the user's current physical location, and submits the image to the workflow to determine a current position in the Of course, the user may, in alternative approaches, utilize any other technique as described herein for obtaining/determining location information.
In another similar scenario, a plurality of users are participating in an art exhibition/auction hosted at a local gallery or museum. To facilitate receiving bids, the auctioneers provide a workflow allowing prospective purchasers to place bids on available pieces, and review competing bids in near or real-time. The workflow also facilitates prospective purchasers capability to inspect and review the items offered at auction, for example by providing a user directions from one exhibit to another within the auction site.
The user provides input indicative of the user's current location to the workflow. The user input may take any suitable form, in various approaches. For example, the user may capture an image of an exhibit proximate to the user's current physical location, select the exhibit name or an image of the exhibit from a set of predetermined exhibits in a list, indicate a room number or name in which the exhibit is displayed, or otherwise provide an of the user's current physical location. The user also provides a similar type of input indicative of the user's desired destination.
Based on the user input, the workflow determines a suitable route from the user's current physical location to the desired destination, and displays navigation instructions to the user. For example, where the user captures an image of the exhibit, the workflow may process the image and determine location information corresponding to the imaged exhibit. The location information may be determined in any suitable form or using any suitable technique, and in preferred embodiments the location information is determined by performing a lookup operation leveraging the captured and processed image data as part or all of a query from which to determine corresponding location information.
In various embodiments, the location determination may comprise comparing the captured and processed image to one or more reference images, each reference image corresponding to a unique location, e.g. one of the displayed exhibits. Based in whole or in part on the comparison, the workflow may identify the exhibit depicted in the captured/processed image, and based on the identified exhibit may determine corresponding location information, e.g. using a relational database linking exhibit image data to corresponding location information. Using the location information, the user may be provided directions from the current location to any desired destination capable of being determined using the user's mobile device, and preferably to any of the exhibits or other unique locations represented in the aforementioned database. It should be understood that the present example is not limited to determining location information from relational databases based on image comparison, but rather may utilize the comparison to determine location information from any number of data sources as described herein.
Continuing with the auction example, the user may also be prompted for a bid. Bid prompts may be provided via an interface from which the user may place a competing bid, withdraw a bid, or place a temporary “hold” on the bidding process, e.g. by initiating a countdown timer having a duration sufficient to allow one or more bidders to travel to and/or inspect the piece in question.
In various embodiments, the user may provide input partially or solely in response to the mobile device providing an indication to the user that location information is required.
For example, a user conducting a workflow via a mobile device may be presented with a prompt requesting user-defined location information. The prompt is optionally presented in response to the user triggering a predetermined condition, such as by taking a predetermined action in connection with the workflow. Exemplary predetermined conditions and/or corresponding user actions suitable as a trigger may include capturing an image, attempting or initiating a financial transaction, moving toward or away from a predetermined location, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
In response to the displayed prompt, preferably the user provides input comprising the location information. The user input may take any suitable form that would be appreciated by one having ordinary skill in the art upon reading the instant disclosures. For example, in various embodiments user input may include a string of alphanumeric characters and/or symbols, such as may be “typed” on a display of the mobile device.
In more embodiments user input may include a selection of one or more from among a plurality of predetermined locations. For example, a user may select a predetermined location from a drop-down list having entries corresponding to one or more locations previously input by the user.
The entries may additionally and/or alternatively include location information previously determined using the mobile device. For example, location information may have been previously determined in connection with the workflow instance currently engaged by the user; in connection with a previous operation or aspect of the present workflow; in connection with a previous instance of the workflow; in connection with another workflow, or even in connection with one or more unrelated process(es) conducted using the mobile device.
Additionally and/or alternatively, a user may be prompted to select from among a plurality of user-preferred sources of location information (e.g. Wi-Fi, GPS, GMS, etc.).
In several instances, a background process may detect requests for location information, e.g. requests submitted to the device from other applications or processes. The requests may be in any suitable form, for example as system calls to native mobile device components or capabilities (e.g. modules in a software library). The location information request may additionally and/or alternatively be detected via detecting invocation of one or more components of the mobile device utilized to acquire and/or determine location information (e.g. a GPS antenna, Wi-Fi antenna, Bluetooth antenna, processor, memory, etc.).
For example, location information may be received, determined, and/or compiled based at least in part on data and/or input received from one or more of: a user other than a primary owner/user of the mobile device (e.g. a friend or family member of the primary owner/user, a colleague of the primary owner/user); a second mobile device (which may optionally be in communication with the primary owner/user's mobile device); from a remote data source (e.g. a data storage center, repository, database, etc.); from a third party service or application (which includes any service or application that allows a user to designate the user's past, current, or future location, e.g. a GPS navigation application, a “Maps” application, a social networking application, etc.) or any other source of human reported location information that would be recognized by one having ordinary skill in the art upon reading the present descriptions.
Human location reporting may be accomplished, in various embodiments, via a “check-in” or other similar feature such as is currently common to many known social networking platforms or services, such as FACEBOOK, GOOGLE PLUS, YELP, etc. The “check-in” feature allows users to designate a physical location at which that individual may be located at a particular time, within a particular time window, and/or for a particular duration of time, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Similarly, any number of peripheral devices and/or services may be leveraged to report location information as required or advantageous to the underlying operations and/or associated workflows. As understood herein, peripheral devices and/or services may include any device, component, or service that may be utilized to obtain and/or determine location information and which is not provided as an integrated part of the primary mobile device.
For example, in various embodiments peripheral devices may include a plurality of systems, such as a network of one or more cellular towers; one or more satellites (especially with respect to GPS location information); network-connected devices other than the mobile device, e.g. one or more Wi-Fi routers, modems, hubs, client devices such as desktop workstations, laptops, additional mobile devices; and one or more radio-frequency identification (RFID) devices, such as passive/active RFID tags and/or readers; etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Where input is received from another user, the input may be received either with or without regard to the identity of the user providing the input. In one approach, input received from another user may be associated with the information identifying the user providing the input, for example associated with user account information corresponding to the user providing the input. By associating identifying information, such as a user account, with a corresponding source of input, it is possible to leverage crowdsourcing approaches to determining location information, including determining a precise location without input from the user and/or device for which the location information is being determined.
In one implementation of this “crowdsourcing” approach, a plurality of individuals are each in possession of a mobile device. Each mobile device has associated therewith unique user account information for the individual user in possession of the mobile device. The individuals may be engaged in a cooperative activity, workflow, operation, etc. that relies on location of one or more of the individuals in the course of the activity (workflow, etc.).
While the activity relies primarily on each individual to input location information, the activity also includes secondary protocols to obtain requisite location information, for example by prompting one or more additional individuals (which may or may not be engaged in the cooperative activity) to supply missing location information concerning one of the individuals engaged in the cooperative activity.
An exemplary implementation of crowdsourcing location information would be in emergency response, where first responders may assist in locating a missing individual. For example, a responder to a flood event falls into a crevice and fails to check in during a regular reporting event. The remaining responders notice this person's absence and are able to narrow a search area for the missing responder by reporting “last-known” location information. The “last-known” location information may be requested from the remaining responders, e.g. via a prompt issued across an entire emergency response network in response to detecting the missing responder's failure to check-in at the predetermined reporting event time and/or location. Similar operative principles apply to scenarios not involving an emergency, but rather any cooperative activity such as a construction project, promotional marketing or sales campaign (e.g. door-to-door sales), etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Location Information Management
Location information may also be managed. For example location information may be manipulated, e.g. by converting the location information from one format to another, associating the location information with other data, and/or storing the location information to a memory, preferably a memory of the mobile device.
Similarly, upon acquisition or determination, location information may be utilized in the course of a workflow. Location information may be used in any suitable manner, and these uses may encompass management of the data, e.g. format conversion.
Location information manipulation may include converting the location information from one format to another (e.g. converting between a street address, GPS coordinates, network address/connection information, etc.), as well as associating the location information with additional information, including but not limited to metadata. Metadata may include any form of data associated with the location information, and preferably the metadata comprises information relating to the workflow(s) in which the location information was requested and/or in which the location information is utilized. Associating location information with additional information relating to one or more workflow(s) advantageously allows facile and efficient retrieval and utilization of the location information in subsequent iterations of the workflow and/or in connection with other workflow(s), in various embodiments.
In some embodiments, location information may be stored, e.g. to a local memory of the mobile device, upon being obtained, determined and/or utilized in the various manners disclosed presently. Preferably, location information may be stored as metadata in association with other data, e.g. image data captured using the mobile device (especially as part of or in connection with conducting a workflow) and most preferably the metadata and/or other data are relevant to the workflow, operation(s) thereof, and/or a business process relating to the workflow, such as a financial transaction being facilitated via the workflow).
Location-Based Workflows
In the context of the present disclosures, location information may be acquired, determined, managed and/or utilized in the course of various workflows, as well as constituent activities and/or rules of workflows. Location information may be used in any suitable manner, and these uses may encompass management of the data as described above, e.g. format conversion, in some approaches.
In general, the presently described location-based workflows may comply with the following illustrative processes, although it should be understood that any of the various aspects discussed herein may be combined in any suitable manner as would be appreciated by a skilled artisan as having utility in any number of applications.
In one approach, a location-based workflow may substantially follow a process as outlined in method 300, shown in
As shown in
Moreover, and with continuing reference to
In operation 306, method 300 includes receiving location information pertaining to the workflow. The location information may include any type or types of location information disclosed herein, and may be received from any suitable source of location information, including both on-device hardware (such as the various antennae disclosed herein), software, and/or remote sources of location information (such as remotely located compute resources, GPS satellite(s), radio frequency transmitters such as cellular towers, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
In more embodiments, method 300 includes operation 308 in which at least one portion of the workflow, e.g. an activity, a rule, etc., is influenced based at least in part on the location information. As understood herein, the influence may take a number of forms, and generally includes actions that either facilitate conducting an activity, interfere with conducting an activity, prevent conducting an activity, suspend or delay conducting an activity, as well as actions that create, modify, or remove one or more rule(s) of a workflow.
In general, according to the presently disclosed systems, techniques, and computer program products, various workflow operations, particularly operations relating to one or more of authorization and access paths to at least one of a workflow, data relating to the workflow and a user interface (UI) configured to facilitate the workflow may be determined in whole or part based on location information corresponding to an individual user requesting the access or authorization.
Similarly, in more approaches one or more workflow decisions can be determined in whole or part based on location information corresponding to the user reviewing, approving, denying, authorizing, or otherwise acting upon or in response to the aforementioned request. In one illustrative scenario, for example, a user may be granted or denied access to a workflow, data relating to a workflow, a UI, etc. based on user-submitted authentication data and location data collected in association with submitting the authentication data.
For example, while a user's authentication data (such as username and password) may be sufficient to authenticate the user via conventional security protocols, the presently disclosed concepts may additionally and/or alternatively leverage location information to act as an additional level of security or authentication. In one instance, if an individual requesting access to data provides authentication information corresponding to an authorized account or authorized user but location data collected in association with the submitted authentication information indicates the requesting individual is attempting to access the secured data from a new, potentially unsecure location or via a potentially unsecure connection.
In a related approach, the location information collected in association with the submitted authentication data may be compared against one or more predetermined locations corresponding to authorized access points, such as the individual's work address, residential address, satellite office or location, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions. Of course, additional variations on the aforementioned location-based authentication may be performed, including any equivalent to the techniques described herein such as would be appreciated by one having ordinary skill in the art upon reading these disclosures.
Several exemplary forms of influence as applied to a plurality of illustrative scenarios are described below and should be considered non-limiting on the scope of the instant disclosures.
In more embodiments, the presently disclosed techniques may include performing some or all of a method 400, such as depicted in
In one approach, method 400 includes operation 402 where a workflow is initiated. The workflow may be initiated in any suitable manner, such as described above with reference to
Moreover, and with continuing reference to
Further, and also at least partially in response to detecting the trigger condition, operation 404 of method 400 includes prompting a user to capture an image of a document relating to the workflow using the mobile device. The user is preferably directed to a capture interface such as provided via the resident operating system (OS) or an application installed on the mobile device (including potentially an application hosting or encompassing the workflow, in one embodiment). The user captures the image of the document using the mobile device, and, in a manner transparent to the user, location information are simultaneously gathered and associated with the captured image as location metadata.
With continuing reference to operation 404, the captured image and associated location metadata are stored to a memory of the mobile device. The stored data may be maintained in a buffer, or retrieved for subsequent use in the current workflow instance. Additionally and/or alternatively, the stored data may be retrieved for use in relation to a subsequent instance of the same workflow, in another workflow related to the same business process, in a different workflow, which may or may not itself be relevant to the to the same business process, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
In operation 406, method 400 includes facilitating the business process to which the workflow relates, based at least in part on the location information reflected in the location metadata.
As noted above, the location information are preferably stored and/or associated with the captured image in the form of location metadata. In one embodiment, location metadata may include a plurality of key/value-type data describing features of the data with which the metadata are associated. The metadata may be embedded in the data with which the metadata are associated, or stored separately, e.g. in a metadata file or pagefile including a reference to the data with which the metadata are associated.
For example, in one approach metadata may include a “LOCATION” field as a key and location information, such as GPS coordinates or a street address, as the corresponding value, e.g. “LOCATION=GPS(194.2334;202.3256)” or “LOC=GPS(194.2334,202.3256)” as two exemplary forms of GPS coordinate-based location metadata, “LOC=ZIP_21702” or “LOC=STATE_MD” for zip code-based and state-based location metadata, respectively. Of course, any alternative or equivalent form of expressing location information, either as metadata or otherwise, are also to be considered within the scope of the present disclosures.
The presently disclosed techniques advantageously leverage the aforementioned location information and/or location determination techniques to enhance a user's experience creating, performing administering and/or otherwise engaging in workflows. Upon reading the instant descriptions, those having ordinary skill in the art will appreciate the particular applicability and advantages of employing the generally-described location-based techniques and/or technologies in workflows, and particularly in business-oriented workflows.
Accordingly, the presently disclosed location-based workflows all relate to a business process, and may rely on one or more business activities and/or business rules in the course thereof. A workflow may be considered to “relate” to a business process whenever that workflow is designed, configured, or enabled to assist a human user in accomplishing one or more business-oriented tasks using the workflow.
Examples of business processes according to various embodiments as described herein include conducting a financial transaction such as a purchase, sale, rental, loan, investment, or any type of financial accounting transaction.
Exemplary business processes may also include processing a claim, such as an insurance claim, a benefits claim, an unemployment claim, a legal claim, or any other type of claim that would be appreciated by a skilled artisan reading the instant disclosures.
Other illustrative business processes include tracking of goods and/or individuals, e.g. for delivery of the goods or services, in some approaches.
In still more approaches, the illustrative business processes contemplated herein may include processes relating to image capture and processing, especially the capture and processing of images depicting business documents. Accordingly, the business-process related workflows described herein may, in some embodiments, include using the location information to facilitate document classification, data extraction, etc.
Business processes may additionally and/or alternatively relate to personnel management, e.g. in applications such as supervisory review/approval of a work product or process, or opportunistic scheduling, in more embodiments. Similarly, the business processes described herein encompass data management and data processing operations, such as timely completion of processing jobs, accessibility of required data to one or more individuals in need, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Business-oriented workflows are to be understood as including, but not being limited to, workflows configured to direct, enable or facilitate performance of various tasks relating to business operation, and especially tasks relating to and/or involving capture and/or use of image data in the course of conducting the business-oriented workflow(s). Additional examples of business-oriented workflows include workflows configured to facilitate or conduct financial transactions, data management, human resource management, location tracking with relation to goods, personnel and/or services, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
While the descriptions herein may be offered with respect to one or more illustrative scenarios, contexts, or use-cases, these techniques and technologies should be understood to be broadly applicable to a plethora of workflow types and/or iterations in a wide variety of contexts relevant to and/or relying upon a diverse selection of data.
Disambiguating Classification
For example, in one embodiment a business-oriented workflow is configured to facilitate providing services, such as credit services, insurance services, etc. A user wishing to apply for a loan or line of credit may submit an application via the workflow, which provides a user-friendly and convenient mechanism for entering requisite information. The exemplary application requires personal identification information, which a user may optionally submit via either directly inputting requested information or by submitting an image of one of a predetermined set of identifying documents, such as a birth certificate, social security card, driver's license, passport, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
In one scenario, the user submits an image of a driver's license to the workflow for the purpose of providing requisite identification information. The workflow processes the image and attempts to classify the driver's license for purposes of image analysis and content identification/extraction. For example, the workflow may utilize custom image processing settings or an extraction template corresponding to driver's licenses issued by a particular authority (and therefore conforming to a uniform layout and/or formatting paradigm). The workflow, for reasons beyond the scope of these disclosures, may identify a plurality of potential classes to which the driver's license in the submitted image may belong. In order to disambiguate among the plurality of potential classes, the workflow may request location information corresponding to one or more of a time when the application was submitted via the workflow, a time when the workflow was instantiated, a time when the image was captured, etc. Using the location information, the workflow may determine that the submitted driver's license belongs to one among the plurality of potential classes due to that class corresponding to a same or similar issuing authority as the issuing authority having jurisdiction over the location where the image was captured.
In a simpler expression of this concept, a Maryland resident initiates a credit application via a workflow. The workflow directs the user to provide requisite identifying information by submitting an image of their driver's license. The workflow invokes a capture interface and directs the user to capture an image of their driver's license. Transparent to the user, the workflow simultaneously acquires or determines location information for the device's current location, and associates the location information with the acquired image as metadata. The workflow attempts to classify the imaged driver's license, and determines the license is equally likely to be a Maryland or a Florida driver's license. Using the location information metadata, the workflow determines the driver's license is a Maryland driver's license based on the location information indicating the image was captured in the state of Maryland.
Conversely, location information may be leveraged to determine a document corresponds to a classification not associated with a particular location.
For example, in one embodiment, a delivery person delivers a package to a destination address. In the course of performing a workflow configured to verify delivery and provide proof-of-delivery data, the delivery person captures an image of the package at the recipient's doorstep. Preferably in the same image, but potentially in an additional image, the delivery person captures image data depicting the packing slip, receipt, invoice, or other documentation serving as a record of the purchase, delivery, etc., and the captured image is associated with location information corresponding to the user's location at the time the image was captured. The delivery person submits the image of the package and packing slip to the workflow, where the image is processed and a classification algorithm is applied thereto to classify the depicted document according to type (e.g. as a receipt, packing slip, address label, inventory sheet, chain-of-custody document, etc.). The classification algorithm may be incapable of definitively classifying the imaged document to only one of the potential classes, and may utilize the location information gathered in conjunction with the image data to disambiguate the document class.
In one approach, the workflow may determine that the location information corresponding to the captured image does not correspond to one or more predetermined locations (e.g. a location corresponding to a production facility, an inspection facility, a distribution facility, a retail location, etc.). The classification may accordingly disqualify otherwise eligible document classifications based at least in part on the location information corresponding to the captured image, especially any classification associated uniquely with the location(s) to which the captured image data's location information do not correspond. The classification decision may optionally be based on one or more business rules (e.g. a standard practice of attaching specific document types to packages at specific stages in a product cycle, such as production/distribution/sale/delivery, etc.).
To be more specific, in one instance a delivery person captures an image of a delivered package and the attached address label. GPS coordinates are collected in connection with the image capture operation, and the image and location information are submitted to the workflow. The workflow attempts to classify the address label, but cannot definitively determine between the document being a packing slip or an address label. Based on the comparison, the workflow determines the location data collected when the delivery person captured the image does not correspond to an address for the carrier's production facility or distribution facility. The workflow may utilize this mismatch singly or in unison with other information, such as predefined business rules, to disambiguate the classification result.
For example, assume a business rule requires packing slips be placed inside the corresponding package, prior to a delivery person receiving the package for delivery. Based on this knowledge, especially in combination with the mismatch referenced above, the workflow may conclude that the imaged address label is not a packing slip because packing slips are sealed and placed interior to the package before the delivery person receives the package for delivery. Accordingly, while the classification algorithm may not be able to affirmatively determine the imaged document is an address label, the workflow may nonetheless achieve determination the correct classification via a process of eliminating the least likely classifications using location information.
In various approaches described herein, as well as any functional equivalents thereof that would be recognized and/or appreciated by one having ordinary skill in the art upon reading the present descriptions, workflows may leverage location information to provide context-dependent, and particularly location-context-dependent functionality to one or more operations of the workflow.
For example, in some approaches location-context-dependent functionality may include invoking a workflow based at least in part on determining location information. For example, a workflow relating to travel may be invoked upon determining the user has arrived at a departure location, such as initiating a check-in process upon determining a user has arrived at a port or an airport, train, or bus terminal.
Secure Data Management
Other location-context-dependent functionality may include interfering with, suspending, or terminating a workflow, or one or more operations in a workflow, based at least in part on determining location information. In one illustrative approach, a workflow utilizes sensitive data, such as personal identification information, financial information, medical information, etc. and the sensitive data are subject to privacy protection or other security or data management protocols that restrict access to and use of the sensitive data.
For example, data access may be granted only in response to one or more prerequisite conditions being satisfied, such as a user providing appropriate authentication information (e.g. a username and password, etc.) and/or requesting access to the sensitive data from a device connected to a particular network to which communications regarding the sensitive data are restricted. Put another way, in this scenario the sensitive data may only be accessed by a user present at a particular location (e.g. a campus serving the network to which communications are restricted).
In order to verify that the request for access to sensitive data is an authorized request satisfying all the prerequisite criteria, the workflow may query the user's mobile device for location information. In some approaches the location information may also serve as network identification information (e.g. an IP address, or identity of a router to which the mobile device is connected). Accordingly, the workflow may determine a user's location, and whether that location corresponds to an authorized “access point” with respect to the sensitive data. Based in whole or in part on whether the location information matches the corresponding information as expected for an authorized “access point,” the workflow may be granted or denied access to the sensitive data.
Additionally and/or alternatively, in other embodiments the location information may serve as an independent means of verifying satisfaction of the geographically restrictive prerequisite criteria. For example, in one embodiment an unauthorized attempt to access the sensitive data may be performed by a user capable of bypassing network authentication security measures (e.g. by falsifying an IP address or other identifying data corresponding to the network upon which access to the sensitive data is restricted). In this embodiment, the user attempting to gain unauthorized access to the sensitive data may be prevented from doing so despite the user's network authentication information indicating a proper request for authorized access. More specifically, based on the fact that location information indicates the user is not located near an authorized access point (e.g. not on the secured campus), the request for access may be denied despite being submitted with appropriate authentication information (e.g. username, password, IP address, etc.).
This capability would allow the user to create a dynamic, adaptive mobile user experience that leverages both the local device capabilities and remote location services.
The application may be installed on the mobile device, e.g., stored in a nonvolatile memory of the device. In one approach, the application includes instructions to perform processing of an image on the mobile device. In another approach, the application includes instructions to send the image to a remote server such as a network server. In yet another approach, the application may include instructions to decide whether to perform some or all processing on the mobile device and/or send the image to the remote site to perform some or all of the processing operations.
Location-Based Entity Identification
In other approaches, the present concepts may be leveraged to identify an entity of interest according to location information.
For example, a user wishes to know which retailer(s) have a branch near the user's current location. The user may engage a workflow, and using location information, identify at least one entity associated with a corresponding location in proximity to the physical location indicated by the location information. For example, a workflow may advantageously identify a commercial entity based on location information, where commercial entities may include one or more of a vendor, a customer, a distributor, a supplier, a carrier, a retailer, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Higher levels of granularity are advantageous in certain applications because less specific information may be necessary to accomplish the desired resolution of location information. Accordingly, in such embodiments, processing and/or communication resource usage may be reduced by utilizing relatively less precise location information rather than precisely determining the physical location. For example, in one instance location information may be utilized to identify a geographic region, e.g. a township, city, state, country, sales territory, distribution territory, service coverage area, intended route/path, etc. This frees up the corresponding processing and/or communication resources for other tasks in or relating to the workflow and/or other unrelated operations.
In preferred embodiments, a workflow may access one or more data sources, each data source comprising location information and associated entity identification information The data source(s) may take any suitable form, including but not limited to one or more databases (including both traditional relational databases such as MySQL, nontraditional databases such as NoSQL and/or “fuzzy” databases, etc. The data sources may additionally and/or alternatively include one or more lookup tables, a structured document comprising data depicted/formatted according to one of a plurality of predetermined conventions, such as hypertext markup language (HTML), extended markup language (XML), comma separated value (CSV), a document depicting entity identification information and location information (e.g. a document such as a bill depicting an entity name and street address), one or more web pages, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Real-Time Tracking and Telemetry
The presently disclosed concepts may also leverage location information to facilitate tracking of data, goods, and/or individuals participating in or otherwise relevant to conducting a workflow. Preferably, the tracking occurs in real-time or near real-time and is based on telemetry data as the preferred form of location information, although discrete location information and non-real-time tracking should also be considered within the scope of these inventive concepts, in various embodiments.
In one approach, tracking may include validating a physical location of a user at a predetermined time based at least in part on the location information. For example, it may be useful in certain workflows to verify that a certain individual, such as a sales representative, service technician, delivery person, public representative, employee, customer, client, etc. is present at a particular location at a particular time, such as a time and location scheduled previously for delivery/rendering of services, goods, etc.
By leveraging a user's mobile device and tracking location in real-time over a particular duration (e.g. over the course of an eight-hour shift, a two-hour service call, a two-week period, etc.) and/or location (e.g. employer campus, sales territory, service region, etc.). In this manner, workflows may facilitate business processes by verifying the presence of various individuals throughout the course of conducting the workflow.
For example, to facilitate customer relations and customer service, an enterprise offering services may leverage a workflow to schedule an appointment window for a client. The client may provide information identifying their mobile device, preferably via an application that allows the client to interface with the workflow, e.g. via an application downloaded to the client's mobile device that allows submission of help requests, scheduling of service appointments, and tendering payment for services rendered (and optional feedback on service quality). A service provider similarly utilizes a mobile device having an application installed thereon and configured to facilitate similar activities from the perspective of the service provider (e.g. dispatch, technician, supervisor, etc.).
Location information may be acquired, either via requests for location information submitted to the mobile device by each user's respective application, or gathered in connection with other actions taking place on each user's respective mobile device, such as described above. Using the location information, preferably in combination with timing information (e.g. as gathered from an on-device clock or network resource), the location of the service technician may be tracked in real-time, and the technician's presence at the client's indicated address within the scheduled appointment window may be verified. Similarly, the client's presence may also be verified at the corresponding location and time. In this manner, it is possible to provide real-time verification of the location of various individuals, goods, data, etc. throughout the course of a workflow. This improves the ability to provide reliable service and resolve disputes arising in the course of conducting various workflows as presented herein and would be appreciated by one having ordinary skill in the art upon reading the present descriptions.
In some embodiments, location verification may utilize the location information in conjunction with an image acquired at the physical location corresponding to the location information, thereby providing multiple levels of verification. In a preferred embodiment, validating a physical location may be utilized to effectuate one or more business processes such as a proof-of-delivery, a proof-of-service (e.g. reading a meter for a utility, service of process, etc.) a proof of presence (e.g. pursuant to a service appointment, a check-in process relating to a transportation service provider, employer, law enforcement authority, etc.); and other equivalent applications as would be comprehended by a skilled artisan reading these disclosures.
Financial Transactions
Location information may also be leveraged in several approaches to facilitate financial transactions, such as submitting a payment, purchasing services, etc.
In one such example, a user is engaged in a workflow to process a payment, e.g. to a utility company. The user images a remittance slip and submits the image as part of the payment processing. Upon receiving the user-submitted image, the workflow invokes image processing functions to identify important information from the remittance slip, e.g. a customer name, customer account number, utility company name, utility company address, account number, amount due, etc. The workflow may leverage location information gathered in connection with capturing the user-submitted image to disambiguate from a plurality of candidate utility companies, e.g. by selecting a utility company in closest physical proximity to a location where the user-submitted image was captured.
In other embodiments, a financial transaction may be routed for processing based at least in part on location information. For example, a financial institution may determine location for routing of requests, etc. to various servers based on location information, especially network-based location information such as an IP address. The financial institution may route the requests in a manner designed to provide maximum quality and speed of service for the center processing the request.
In more approaches, financial transactions may include a financial institution presenting relevant branding and/or advertising material. For example, a bank might integrate location-based services into a financial service workflow to offer various products or brands based at least in part on location information. In more approaches, the offer may be based additionally or alternatively on one or more of: historical location information (e.g. as may be indicative of a likelihood of future travel and therefore applicability of offers pertaining to one or more likely travel destination(s)), financial risk profile, financial investment profile, financial assets, obligations, or any other relevant financial data as would be understood by one having ordinary skill in the art upon reading the present descriptions.
For example, in one approach a user's travel history indicates likelihood of future travel to a foreign country, e.g. for a user that travels frequently to Europe. A workflow leveraging such travel history may also, based on profile information, determine that the user is not a citizen of the travel destination country, and accordingly determine that particular services may be of interest to that user. For example, the workflow may display a notification, prompt, banner, etc. for financial services, such as traveler's checks or a credit card with travel-related benefits, etc. as would be appreciated by a skilled artisan cognizant of these disclosures.
As will be appreciated by one having ordinary skill in the art upon reading the present descriptions, the aforementioned techniques may be applied to a variety of financial transactions, involving a diverse array of document(s) and/or transaction types. For example, the financial transactions described herein may involve financial documents and/or transactions relating to mortgages, insurance information, medical information, law enforcement and/or emergency service reports, delivery, tracking, and/or logistics applications. As will be explained in further detail below, standard forms frequently associated with such transactions are also contemplated as useful applications for location based services and workflows as described herein.
Claim and Form Processing
The presently disclosed techniques are also advantageous when applied to processing forms submitted in connection with any number of business processes, such as medical claims, benefits claims, insurance claims, loan applications, new account openings, rental applications, tax filings, or any other equivalent business process relying on data collection and/or processing via one or more forms, such as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Generally, claim processing and/or form processing may comprise a data gathering portion of a workflow, and a user may submit one or more images of a form, or images of one or more other documents (whether or not fitting a standard “form”) such as identifying documents, financial documents, medical documents, etc. The workflow preferably processes the image(s) and utilizes data from the imaged document(s) to populate fields on the form.
In one approach, a plurality of potentially applicable forms may exist, and a workflow may include a disambiguation operation relying at least partially on location information to resolve among plural potentially applicable forms. For example, although individuals across a diverse geographical region may engage in substantially similar and/or common business processes, the common processes may have associated therewith a wide variety of different forms, each form potentially being “standard” within the respective locality but having little similarity in one or more aspects when compared to corresponding forms in other localities.
In one specific example, a plurality of forms exist for a single or highly similar purpose across a plurality of localities. Each form therefore depicts substantially similar types of information (e.g. identifying information including name, date of birth, address, social security number, etc.), but may be highly divergent from one another with respect to the location and/or format of this information on each form. For example, a plurality of driver's license application forms all require the applicant's name, date of birth, and current address, but express this information in different formats, and on different portions of the corresponding form employed in each locality. Accordingly, a workflow in which data obtained from documents is utilized to populate the forms (or other similar operations as would be appreciated by a skilled artisan) will have variable performance, based largely on whether the workflow identifies the appropriate form to populate with the obtained data.
In one embodiment a first form includes dates in a “MM/DD/YY” format while a second form includes full dates, such as “Jan. 1, 2012”. Moreover, the first form includes name, date of birth and address fields next to a portrait image of the applicant's face, all located in an upper-right region of a portrait-oriented form document. Meanwhile, the second form is in “landscape” orientation, omits the portrait image, and includes name, date of birth, and address fields on a lower right portion of the landscape-oriented document. (In this example, the differences between the several forms are exaggerated for illustrative purposes. It should be understood that practical implementations also include scenarios where forms are substantially identical with exception of a single or relatively small number (e.g. three to five) of definitive differences with relation to corresponding forms utilized in other localities.
Workflows may determine an appropriate form from among a plurality of potential forms using location information, in some approaches. For example, in one embodiment a form processing workflow includes a user capturing an image of one or more source documents and the form. The user submits the images, which are preferably associated with location information corresponding to the location the document images were imaged. Based in whole or in part on the location information, the workflow may determine from among a plurality of potentially appropriate forms with which to populate the data.
For example, the workflow may compare the location information with information depicted on one or more of the source documents. If a location depicted on a source document matches a locality corresponding to the location information (such as a Maryland driver's license and location information comprising GPS coordinates corresponding to Germantown, Md.), then the workflow may select a form corresponding to the matching locality from among the plurality of forms. Additionally and/or alternatively, the location information alone may be leveraged to select the appropriate form.
Location information may also be utilized to populate forms with data based in whole or in part on location information associated with the form, location information determined or provided by a user in connection with the workflow, location information associated with an image captured and/or analyzed in connection with a workflow, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
For instance, in one approach a claims processing workflow may include filling out the location where an automobile accident occurred based on the location information associated with the image of the damaged vehicle.
In additional and/or alternative approaches, a claims processing workflow includes filling out appropriate fields of a form that corresponds to a particular location using data gathered from another document corresponding to the same location or a similar location. For example, one workflow populates a state accident report form with information gathered from/in connection with an image of a driver's license. The selection of data from the driver's license, as well as the placement of said data on the form may be based at least in part on determining that the driver's license was issued by the same state/authority in which the accident occurred. The determination of similar location is preferably indicated by similarity of location information associated with the image of the driver's license and location information associated with an image or document involved in the workflow, e.g. an image of the damaged vehicle(s) taken in connection with the accident.
In a particularly preferred embodiment, the image of the damaged vehicle(s) was taken in connection with the accident occurrence, and has associated therewith metadata comprising GPS location information. Similarly, the image of the driver's license has associated therewith metadata comprising location information. The location metadata may be in any form, and may include imprecise or “less precise” location information. Indeed, in some approaches the location metadata may simply be a two-letter state abbreviation—such as “MD” for the state of Maryland). The workflow compares the location metadata, and without needing to perform complex image analysis, data extraction, etc. on an image of a document captured in connection with the accident, e.g. a driver's statement taken at the scene of the accident, the workflow may populate the accident report form with pertinent information relating to the accident. Data population may involve any combination of locating appropriate field(s) on the form with which to populate the data, providing the data to populate the appropriate field(s) on the form, validating data currently populating the form, and replacing data currently populating the form, in multiple variations.
The workflow determines the propriety of such an automatic data population operation based on similarity of the location information associated with each image, and may optionally prompt a user for confirmation that the automatic data population would be proper in the context of the presently engaged workflow. Upon determining the population is proper, the workflow populates the form with pertinent information previously determined/acquired. The pertinent information may have been determined and/or acquired, for example, from the image of the driver(s) license(s), from a central data source, etc., in various embodiments. Preferably, the pertinent information was retrieved using the image of the driver(s) license(s), such as by retrieving information from a remote data source, e.g. retrieving a record of a database maintaining driver license data based on an individual's driver license number and last name. Even more preferably, the pertinent information were acquired and/or determined singly or in combination either directly from the image of the driver's license; or in the course of performing another workflow utilizing the image of the driver's license.
Data Processing and Management
The presently disclosed inventive concepts also have utility in data management and data processing applications.
In one embodiment, for example, continuous location information may be utilized to prioritize one or more tasks relating to the workflow. These examples also include embodiments where, a mobile device's location information may be utilized to facilitate data management. For example, in one approach a user engages a business process via a workflow that relies on particular source data. The source data may require some preprocessing before being suitable for intended use in the workflow, for example the data may require specific formatting, quality analysis, etc.
In another illustrative approach, source data comprising an image of a document are preferably processed using one or more image processing operations such as disclosed in the related patent applications referenced above (including particularly U.S. patent application Ser. No. 13/740,123, filed and/or Ser. No. 13/802,226, filed Mar. 13, 2013, as well as Provisional U.S. Patent Application Nos. 61/819,463, filed May 3, 2013 and 61/780,747, filed Mar. 13, 2013. In other embodiments, a workflow requires the source data be formatted according to one among a plurality of predetermined data formats.
Regardless of the particular processing to be performed, in various embodiments workflows within the scope of the present disclosures may leverage location information, e.g. gathered via the user's the mobile device, to suggest and/or automatically conduct data processing according to the user's business process activity and/or needs.
For example, in one approach an automobile insurance agent is engaged in a workflow configured to facilitate processing an insurance claim relating to an accident involving a client. The agent receives notification that the client was involved in an automobile accident (e.g. via a telephone call, email, or automatic notification sent to the insurer). The notification may include, or be accompanied by additional data relating to the event, such as image data depicting the automobile(s) involved in the accident, image data depicting a driver's license and/or other identifying documents for the client, etc.
In order to facilitate processing the claim (e.g. by providing a quote), the insurance agent requires (or if not required, would benefit from) certain information, at least some of which may be obtained from the image data and/or additional data. Furthermore, the agent may require access to this information at a predetermined time, such as upon arrival at the scene of the accident. Location information may be leveraged in this case to facilitate the workflow by ensuring all requisite data are available at the moment the agent requires or is otherwise prepared to receive such data.
For example, in one approach a workflow detects satisfaction of a trigger condition, such as a request for a quote or notification of an accident occurring, according to the previously described scenario. In response to detecting the trigger condition, the workflow may initiate processing of the data relating to the event, manage a processing priority of a job handling the data relating to the event, etc. The trigger condition may take any suitable form, however, and may involve a plurality of conditions. In a particularly preferred approach according to the insurance quote example, the trigger condition may comprise a combination of receiving notification of a client's involvement in an accident, as well as determining that the agent is en route to the accident site.
To determine an agent is en route to an accident site, the workflow preferably utilizes location information. While any location information is suitable, the location information is preferably in the form of telemetry data. The location information indicate the agent is headed on a trajectory leading toward the accident location, and will arrive at an estimated time of arrival (ETA). Based on the agent's ETA, as well as the processing requirements (e.g. minimum processing resources, time), the workflow may influence data management and/or processing. For example, the workflow may initiate a processing job, modify an existing processing job (e.g. by dedicating more/less or different resources to performing the job), and/or terminate processing jobs in a manner suitable to ensure that any requisite data are ready for the agent's use upon arrival at the accident site.
Information that may be useful in a given scenario will depend upon the nature of the scenario. Generally, useful information should be considered to include but not be limited to any information that may be acquired from an image and/or a, such as identifying information, quality information, content information, location information, etc. In the context of the automobile accident, useful information includes but is not limited to: information relating to the identity of the client, such as name, date of birth, address, etc.; information relating to the identity of the vehicle, such as VIN, license plate number, make/model, year, etc. in various approaches.
In preferred embodiments, automatic notifications may also be based in whole or in part on location data. For example, in the context of the present automobile accident scenario, an automatic notification could be sent to the insurer in response to determining a user is traveling in an automobile (as may be indicated according to telemetry data corresponding to travel at a rate of speed typically accomplished via automobile, and GPS data corresponding to travel along a major interstate or other known automotive traffic route), and that the automobile is likely to have been involved in an accident (e.g. as indicated by a particular rate of deceleration being greater than a threshold “safe” deceleration rate, and/or divergence from the known traffic route upon which the automobile was traveling being greater than a predetermined distance).
As will be appreciated by skilled artisans reading these descriptions, these indicia are equally applicable to other scenarios, such as emergency response workflows, traffic reporting workflows, etc. In addition, other indicia applicable to other scenarios described herein may also be leveraged to provide automatic notifications in a similar manner, according to myriad approaches.
In at least some approaches, the processing priority and/or execution of jobs may be invoked and/or managed at least partially in response to initiating a process in the workflow, detecting motion for a predetermined period of time (e.g. as may be indicative of an individual embarking on a journey from a first location to a destination location) arriving within a predetermined proximity of a physical location, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
Supervisory Approval/Review
In still other embodiments, the instant location-based approaches are advantageously applicable to supervision and approval of various business processes. For example, in one embodiment an action relating to or part of a workflow might require approval by an authorizing entity. In the course of the workflow, when approval from the authorizing entity is required, the workflow may leverage location information to locate a closest available authorizing entity, an alternate authorizing entity, etc.
This is particularly advantageous in situations where a primary authorizing entity is unavailable for any variety of reasons. By providing real-time or near-real-time alternative authorizing entities, a user may facilitate expeditious processing of the workflow and underlying data in a manner that may otherwise have taken significantly more time and/or effort due to the unavailability of the primary authorizing entity.
The authorizing entities, in various embodiments, may include one or more human and/or non-human entities. For example, human authorizing entities may include individuals occupying a supervisory role, individuals responsible for inspection or quality assurance, or simply peers/colleagues in the case of a peer-review process.
Non-human authorizing entities, similarly, may include hardware and/or software configured to perform similar supervisory, quality assurance and/or inspection tasks (e.g. a data processing system including one or more computer readable storage media and processor(s) configured to perform authorization.
In one embodiment, the system may include software configured to perform authorization tasks may, in one approach be configured to ensure data are submitted in a predetermined format (e.g. for downstream processing). The software may optionally be configured to convert submitted data from a plurality of formats to the predetermined format in response to determining submitted data are not in compliance with the predetermined format.
The system may, according to additionally and/or alternative embodiments, include hardware configured to perform authorization tasks. For example, in one approach such hardware may include one or more sensors, such as optical sensors, magnetic sensors, electrical sensors, temperature sensors, or any other type of sensor or device as would be understood by one having ordinary skill in the art as being useful to measure one or more physical properties of an article. Another example of useful hardware include scanning and/or wireless communication devices, such as barcode readers, RFID tags and/or readers, cameras, etc.
The hardware may be utilized, under the control of software and/or a human operator, to inspect physical properties of goods (such as weight, size, physical appearance, color, etc.) and/or information relating to such goods (e.g. in a barcode label, a memory coupled to the goods, etc.) to ensure compliance with one or more predetermined quality control criteria.
In one particular instance, a workflow utilizes hardware and software to facilitate quality control. The hardware is configured, under the control of the software, to automatically capture an image of goods scheduled for shipment. The goods may be of any type and in any suitable form, including but not limited to physical articles of manufacture, documents, data, e.g. data stored to a computer readable storage medium, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
The goods may have identifying information associated therewith, for example as a document attached to the goods such as an invoice, shipping manifest, barcode, etc.; as a label affixed to the goods and encoding the identifying information in a barcode; as data stored in a computer readable storage medium (e.g. a non-volatile memory) affixed to the goods or packaging, etc. as would be understood by one having ordinary skill in the art upon reading the present descriptions.
The captured image may be processed. The processing may optionally include performing image processing to improve the quality of the image, e.g. for improving subsequent processing operations using the image data, such as a document classification operation, a content extraction operation, etc. Upon processing the captured and/or analyzed image, in at least some embodiments the workflow may encounter an authorization requirement, in response to which the workflow may seek eligible authorizing entities and submit the image and/or analyzed data to one or more available authorizing entities for authorization to proceed in the workflow. Authorizing entities may be identified, at least in part, based on location information relating to one or more of the goods and/or the supervising entity. For example, authorizing entities may be identified based on physical proximity to goods, especially in cases where human inspection of goods is requisite to providing the authorization.
Similarly, in additional embodiments a preferred authorizing entity may not be available to perform the required inspection and/or authorization of the goods. In such scenarios, the workflow may seek one or more alternative authorizing entities, and identify such alternatives to the workflow, and/or to a user. In various approaches, the alternative authorizing entities may be sought either in addition to the preferred authorizing entity, or in response to determining that the preferred authorizing entity is incapable of performing the required inspection and/or authorization (e.g. due to physical unavailability, due to insufficient resources, due to business rules, due to security criteria, etc.).
While the foregoing authorization concepts have been described from the perspective of inspecting goods, it should be appreciated that the principles disclosed herein are equally applicable to a host of applications and scenarios beyond tracking and/or inspection of goods. For example, many forms of data processing may be similarly facilitated using a general approach wherein a workflow encounters an authorization requirement, and in response seeks eligible authorized entities based at least in part on location information relating to the authorized entity and/or the data requiring authorization. The central feature conferring advantage to these applications of location information is to remove bottlenecks in business processes caused by physical unavailability of required entities at a given location, and/or to improve the manner in which data are processed to ensure efficient, timely handling of jobs in a queue as required to maintain, for example, a production schedule or otherwise meet regular business goals (e.g. daily, monthly, quarterly, annual output, data processing volume, etc.).
Opportunistic Scheduling
Similarly, location information may be leveraged across a plurality of devices to influence and/or conduct fortuitous context-dependent actions. For example, in one embodiment a plurality of workflow participants, e.g. volunteers at an event, employees at a company, etc. utilize mobile devices to conduct one or more operations of a workflow. The workflow may require or benefit from several of the participants, and/or other individuals, being in close physical proximity to one another in certain circumstances. For example, in one instance a consensus vote may be required to proceed with a workflow, instructions from a supervisor may need to be distributed to subordinates, input relating to a particular data point may be required etc. in order for the workflow to proceed. In some approaches, particularly where the workflow requires or benefits from participation from a threshold number of participants, it may be advantageous for location information to be shared among the plurality of workflow participants, in order to opportunistically schedule a mutually convenient meeting time and/or location for the various required participants.
Business Resource Management
In a similar vein, location information may be useful to control a facility. For example, in one embodiment it may be advantageous to conserve power during non-peak hours by reducing or disabling one or more systems, e.g. lighting, heating/cooling, security, etc. Based on location information obtained from one or more users' devices, it may be possible to activate/deactivate appropriate systems based on satisfaction of certain conditions, such as a threshold number of individuals being physically present at the facility, environmental conditions satisfying one or more threshold criteria (such as a minimum/maximum temperature, humidity, gas composition, etc.).
The presently disclosed inventive concepts may also be embodied as a system. For example, in one such embodiment a system includes a processor and logic. The logic is in and/or executable by the processor, and configured to cause the processor to perform one or more operations in response to invocation and/or execution thereof
The inventive concepts disclosed herein have been presented by way of example to illustrate the myriad features thereof in a plurality of illustrative scenarios, embodiments, and/or implementations. It should be appreciated that the concepts generally disclosed are to be considered as modular, and may be implemented in any combination, permutation, or synthesis thereof. In addition, any modification, alteration, or equivalent of the presently disclosed features, functions, and concepts that would be appreciated by a person having ordinary skill in the art upon reading the instant descriptions should also be considered within the scope of this disclosure.
While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of an embodiment of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
Claims
1. A computer-implemented method, comprising
- receiving, at a mobile device, an image depicting a document;
- attempting to classify, using a processor of the mobile device, the document depicted in the image to one of a plurality of predetermined document classes, wherein attempting to classify the document results in an ambiguous classification result;
- determining, using the mobile device, location information identifying a geographic location of the mobile device at a particular time; and
- disambiguating, using the processor of the mobile device, the ambiguous classification result based on the location information.
2. The method as recited in claim 1, further comprising:
- building an extraction model based at least in part on the disambiguated classification result, the location information, or both; and
- extracting content from the document using the extraction model.
3. The method as recited in claim 1, wherein the particular time is a time at which the image was captured using a capture device.
4. The method as recited in claim 1, wherein the particular time is a time of attempting to classify the document.
5. The method as recited in claim 1, wherein disambiguating the ambiguous classification result comprises affirmatively matching the location information to a predetermined location.
6. The method as recited in claim 1, wherein disambiguating the ambiguous classification result comprises determining the document corresponds to a same authority as an authority having jurisdiction over the location of the mobile device at the particular time.
7. The method as recited in claim 6, wherein disambiguating the ambiguous classification result further comprises assigning the document to the one of the plurality of document classes corresponding to the authority having jurisdiction over the location of the mobile device at the particular time.
8. The method as recited in claim 1, wherein disambiguating the ambiguous classification result comprises disqualifying at least some of the plurality of predetermined document classes based at least in part on the location information.
9. The method as recited in claim 8, wherein disambiguating the ambiguous classification result comprises a process of eliminating all but one of the plurality of predetermined document classes.
10. The method as recited in claim 1, wherein disambiguating the ambiguous classification result comprises comparing the location information to predetermined location information corresponding to one or more predetermined locations.
11. The method as recited in claim 10, wherein disambiguating the ambiguous classification result comprises determining that the location information does not match or otherwise correspond to the predetermined location information corresponding to the one or more predetermined locations.
12. The method as recited in claim 1, wherein the document is a packing document.
13. The method as recited in claim 1, wherein the document is an identifying document.
14. The method as recited in claim 1, further comprising converting the received location information to match an expected location information format.
15. A computer program product, comprising a computer readable medium having embodied therein computer readable program instructions executable by a mobile device to cause the mobile device to perform a method comprising:
- receiving, at a mobile device, an image depicting a document;
- attempting to classify, using a processor of the mobile device, the document depicted in the image to one of a plurality of predetermined document classes, wherein attempting to classify the document results in an ambiguous classification result;
- determining, using the mobile device, location information identifying a geographic location of the mobile device at a particular time; and
- disambiguating, using the processor of the mobile device, the ambiguous classification result based on the location information.
16. The computer program product as recited in claim 15, wherein disambiguating the ambiguous classification result comprises affirmatively matching the location information to a predetermined location.
17. The computer program product as recited in claim 15, wherein disambiguating the ambiguous classification result comprises determining the document corresponds to a same authority as an authority having jurisdiction over the location of the mobile device at the particular time.
18. The computer program product as recited in claim 15, wherein disambiguating the ambiguous classification result comprises disqualifying at least some of the plurality of predetermined document classes based at least in part on the location information.
19. The computer program product as recited in claim 15, wherein disambiguating the ambiguous classification result comprises comparing the location information to predetermined location information corresponding to one or more predetermined locations.
20. The computer program product as recited in claim 15, further comprising:
- building an extraction model based at least in part on the disambiguated classification result, the location information, or both; and
- extracting content from the document using the extraction model.
Type: Application
Filed: Feb 3, 2017
Publication Date: May 25, 2017
Inventors: Steven Kilby (Rancho Santa Margarita, CA), Anthony Macciola (Irvine, CA), Jan W. Amtrup (Silver Spring, MD), Bruce Orcutt (Newport Beach, CA)
Application Number: 15/424,756
