ELECTRONIC FORM GENERATION FROM ELECTRONIC DOCUMENTS
In some aspects, a method for converting legacy electronic forms and static electronic documents to web-fillable electronic forms includes receiving data corresponding to an electronic document. The electronic document has an input file format. The method includes detecting one or more elements of the electronic document from the data based on a predetermined element interpretation rule set. The method includes generating an intermediate file representing an electronic form based on the data. The intermediate file includes metadata indicating the one or more elements and has an intermediate file format. The method includes displaying a user interface configured to enable user selection of a target output file format. The method also includes converting the intermediate file to an output file having the target output file format. The output file represents the electronic form configured to support user entry of information. The method further includes outputting the output file.
The present application is a continuation of U.S. patent application Ser. No. 16/922,971 filed Jul. 7, 2020 and entitled “ELECTRONIC FORM GENERATION FROM ELECTRONIC DOCUMENTS,” which is related to U.S. patent application Ser. No. 17/714,079 filed Apr. 5, 2022 and entitled “INTERACTIVE TOOL FOR MODIFYING AN AUTOMATICALLY GENERATED ELECTRONIC FORM,” which is a continuation of U.S. patent application Ser. No. 16/922,974 filed Jul. 7, 2020 and entitled “INTERACTIVE TOOL FOR MODIFYING AN AUTOMATICALLY GENERATED ELECTRONIC FORM,” the contents of which are expressly incorporated herein by reference in their entirety.
TECHNICAL FIELDThe present subject matter is directed generally to applying layers of recognition (e.g., computer vision, text, shape, and form) to transform legacy electronic forms and static electronic documents to intelligent web-fillable electronic forms.
BACKGROUNDForms are a common type of paper document. Typically, forms include multiple fields for a person to fill in with various information, such as a name, a date, an answer to a question, a description, or other information. If an enterprise wishes to have employees fill out a paper form, a copy of the form must be made for and provided to each employee, which can take significant amounts of time and resources, such as printer ink, paper, and the like. Additionally, if the form requests entry of sensitive information, the completed paper forms must be stored or destroyed (after obtaining the information contained therein) in a secure manner.
As data processing and electronic documents have evolved, many enterprises have transitioned from using paper forms to using electronic forms, also known as “smart forms.” An electronic form or smart form refers to an electronic document that supports features such as electronic completion, dynamic sections, database calls, and electronic submission, among other features. For example, a user may enter information electronically into an electronic form using a computer, and by submitting the electronic form, the user may cause the entries in the electronic form to be provided to a database, server, or other network location for storage and/or additional processing.
Electronic forms may significantly reduce the amount of time and resources spent extracting information as compared to using paper forms. However, creating an electronic form is not an insignificant task. For example, a user may access an electronic document creation application to create an electronic form by entering text, creating and positioning data entry fields, creating and positioning user-configurable options such as buttons, checkboxes, date entries, and the like, designing tables for data entry, linking elements to data sources or databases, and configuring electronic submission protocols. As such, the electronic form creation process may be time consuming and pose additional costs to the enterprise. These costs may seem particularly burdensome when paper forms already exist, which may cause some enterprises to resist transitioning paper forms to electronic forms.
SUMMARYThe present disclosure provides systems, methods, and computer-readable media for generating a digital document that supports electronic completion (e.g., web-fillable completion), such as an electronic form, particularly from digital documents that do not already have metadata indicating elements of the document. For example, an electronic document may be a portable document format (pdf) file generated by scanning a print document, such as a paper form, or a document generated using a basic word processing or document creation application. To illustrate, the systems and methods described herein may detect one or more elements of the electronic document, such as text blocks, labels, fields, buttons, and/or tables, as non-limiting examples. Detection of the one or more elements may be based on a predetermined element interpretation rule set. For example, the predetermined element interpretation rule set may include rules for interpreting visual indicators (e.g., graphical features representing buttons, checkboxes, or other interactive elements), rules for interpreting dimensions or sizes of elements (e.g., rules that indicate a correspondence between a particular size or dimensions and a type of element), label threshold rules (e.g., rules for applying labels to fields based on distances), rules for detecting tables or repeating tables, rules for detecting data entry formats, text recognition rules (e.g., optical character recognition (OCR) rules), other types of rules, or a combination thereof. In this manner, the systems and methods described herein may be able to identify various types and other characteristics of elements in an electronic document that lacks metadata defining the elements. Alternatively, if the electronic document does include such metadata, the systems and methods described herein may extract the information related to the elements from the metadata, with reduced time and processing resources, as compared to automatically detecting the elements.
After detecting the elements, the systems and methods described herein may generate an intermediate file representing an electronic form based on the electronic document. For example, the intermediate file may include text extracted from the electronic document in addition to the detected elements. One or more of the detected elements may be converted or otherwise modified to an electronically-interactive format. For example, text corresponding to a selection between options, such as “yes/no,” may be converted into a button or checkbox that is configurable electronically. As another example, a straight line indicating a location to write information may be converted into a field, such as a text box, that is user-fillable. Additional examples are described further within. The intermediate file may include metadata that indicates the detected elements, or information associated with the detected elements, such as labels, formats, dimensions, element types, and/or the like. The intermediate file has an intermediate file format. For example, the intermediate file format may be a “universal” proprietary file format that enables display and editing of the electronic form without restricting the electronic form to a particular file format.
After generating the intermediate file, the systems and methods described herein may display a user interface (UI) configured to enable user selection of a target output file format. For example, the UI may enable selection between a hypertext markup language (HTML) file, a proprietary output file format, or one or more output file formats supported by existing third-party electronic form applications. The intermediate file may be converted to an output file having the selected output file format to generate an output file representing the electronic form. For example, images or text of the electronic form may be formatted in accordance with rules associated with the output file format, the metadata may be modified or formatted in accordance with the rules, the elements may be adjusted or formatted in accordance with the rules, and/or data entry and electronic submission parameters may be set in accordance with the rules, as non-limiting examples. The output file may be output for electronic distribution to one or more users, such as via a network, an intranet, the Internet, etc. Thus, the techniques described herein enable generation of an electronic form from an electronic document, such as a pdf file of a scanned print document or another format of electronic document, that does not include metadata that indicates elements of the electronic document. The generation of the electronic form is performed automatically by the system, thereby reducing (or eliminating) manual input during the process.
In one particular aspect, a method for converting legacy electronic forms and static electronic documents to web-fillable electronic forms includes receiving data corresponding to an electronic document. The electronic document has an input file format. The method also includes detecting one or more elements of the electronic document from the data based on a predetermined element interpretation rule set. The method includes generating an intermediate file representing an electronic form based on the data. The intermediate file includes metadata indicating the one or more elements. The intermediate file has an intermediate file format. The method also includes displaying a user interface configured to enable user selection of a target output file format. The method includes converting the intermediate file to an output file having the target output file format. The output file represents the electronic form configured to support user entry of information. The method further includes outputting the output file.
In another aspect, an apparatus for converting legacy electronic forms and static electronic documents to web-fillable electronic forms may be provided. The apparatus includes a processor and a memory coupled to the processor. The memory stores instructions that are executable by the processor to cause the processor to receive data corresponding to an electronic document. The electronic document has an input file format. The instructions also cause the processor to detect one or more elements of the electronic document from the data based on a predetermined element interpretation rule set. The instructions cause the processor to generate an intermediate file representing an electronic form based on the data. The intermediate file includes metadata indicating the one or more elements. The intermediate file has an intermediate file format. The instructions also cause the processor to initiate display of a user interface configured to enable user selection of a target output file format. The instructions cause the processor to convert the intermediate file to an output file having the target output file format. The output file represents the electronic form configured to support user entry of information. The instructions further cause the processor to output the output file.
In yet another aspect, non-transitory computer-readable medium may be provided. The non-transitory computer-readable medium may store instructions that, when executed by a processor, cause processor to perform operations that include receiving data corresponding to an electronic document. The electronic document has an input file format. The operations also include detecting one or more elements of the electronic document from the data based on a predetermined element interpretation rule set. The operations include generating an intermediate file representing an electronic form based on the data. The intermediate file includes metadata indicating the one or more elements. The intermediate file has an intermediate file format. The operations also include initiating display of a user interface configured to enable user selection of a target output file format. The operations include converting the intermediate file to an output file having the target output file format. The output file represents the electronic form configured to support user entry of information. The operations further include outputting the output file.
The foregoing broadly outlines the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.
For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
Various features and advantageous details are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known starting materials, processing techniques, components, and equipment are omitted so as not to unnecessarily obscure the invention in detail. It should be understood, however, that the detailed description and the specific examples, while indicating embodiments of the invention, are given by way of illustration only, and not by way of limitation. Various substitutions, modifications, additions, and/or rearrangements within the spirit and/or scope of the underlying inventive concept will become apparent to those skilled in the art from this disclosure.
It is noted that the functional blocks, and components thereof, of system 100 of implementations of the present invention may be implemented using processors, electronics devices, hardware devices, electronics components, logical circuits, memories, software codes, firmware codes, etc., or any combination thereof. For example, one or more functional blocks, or some portion thereof, may be implemented as discrete gate or transistor logic, discrete hardware components, or combinations thereof configured to provide logic for performing the functions described herein. Additionally or alternatively, when implemented in software, one or more of the functional blocks, or some portion thereof, may comprise code segments operable upon a processor to provide logic for preforming the functions described herein.
It is also noted that various components of system 100 are illustrated as single and separate components. However, it will be appreciated that each of the various illustrated components may be implemented as a single component (e.g., a single application, server module, etc.), may be functional components of a single component, or the functionality of these various components may be distributed over multiple devices/components. In such aspects, the functionality of each respective component may be aggregated from the functionality of multiple modules residing in a single, or in multiple devices.
In some aspects, server 110, user terminal 160, output terminal 162, and data sources 170 may be communicatively coupled via network 180. Network 180 may include a wired network, a wireless communication network, a cellular network, a cable transmission system, a Local Area Network (LAN), a Wireless LAN (WLAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), the Internet, the Public Switched Telephone Network (PSTN), etc., that may be configured to facilitate communications between user terminal 160 and server 110 and/or between server 110 and output terminal 162.
User terminal 160 may be implemented as a mobile device, a smartphone, a tablet computing device, a personal computing device, a laptop computing device, a desktop computing device, a computer system of a vehicle, a personal digital assistant (PDA), a smart watch, another type of wired and/or wireless computing device, or any part thereof. User terminal 160 may be configured to provide a graphical user interface (GUI) via which a user may be provided with information related to data and information received from server 110. For example, user terminal 160 may display one or more GUIs from server 110. The one or more GUIs may facilitate user selection of an input file type, user selection of one or more parameters associated with detecting elements in an electronic document, editing of the electronic form, user selection of an output file type, or user selection of one or more parameters of a file conversion process, as illustrative, non-limiting examples. The selections or other user actions may be provided to server 110 from user terminal 160 as an input. Additionally or alternatively, the one or more GUIs may enable a user to view results of generating the electronic form (e.g., the intermediate file), a graphical representation of the electronic form, or results of converting the intermediate file to the target output file format, as non-limiting examples.
Output terminal 162 may be implemented as a mobile device, a smartphone, a tablet computing device, a personal computing device, a laptop computing device, a desktop computing device, a computer system of a vehicle, a PDA, a smart watch, another type of wired and/or wireless computing device, or any part thereof. Output terminal 162 may be configured to receive an output file from server 110 and to provide a GUI via which a user may enter information or otherwise interact with an electronic form represented by the output file. Additionally or alternatively, output terminal 162 may include or correspond to (or be replaced with) a network device, such as another server or a database, that is configured to store output files (e.g., electronic forms) for distribution to other terminal devices, such as via a private network, an intranet, the Internet, network 180, or any other type of network connection. In such implementations, output terminal 162 may additionally be configured to store input data received when users interact with the electronic form.
Data sources 170 may comprise at least one source of textual data, image data, or both, corresponding to one or more electronic documents. For example, the data source(s) may include a database or other electronic storage of electronic documents, such as forms, that have been scanned from physical documents or generated by a word processing or document creation application. In some implementations, data sources 170 may be owned or operated by an individual or an enterprise, and may be integrated within server 110 or accessible to server 110 via network 180, such as a network, WLAN, or an intranet, as non-limiting examples. Additionally or alternatively, data sources 170 may be owned or operated by a third party, and may be accessible via the Internet (or other network 180). For example, data sources 170 may include an online forms data source, a business data source, a legal compliance data source, a streaming data source, a database, a social media feed, a data room, another data source, the like, or a combination thereof. In some implementations, the data from data source 170 may include or correspond to one or more documents designed to be at least partially completed by a user, such as a form. In some implementations, the data from data source 170 does not include metadata that indicates elements of the electronic document, such as an electronic document that has been scanned from a physical document or created without such metadata by an application. In some other implementations, the data from data source 170 may include metadata that indicates elements of one or more electronic documents.
Server 110 may be configured to receive data from data sources 170, to apply customized text processing algorithms, image processing algorithms, rules-based analysis, machine learning algorithms, and/or other processing to generate an electronic form based on an electronic document. This functionality of server 110 may be provided by the cooperative operation of various components of server 110, as will be described in more detail below. Although
As shown in
In some aspects, processor 111 may comprise a processor, a microprocessor, a controller, a microcontroller, a plurality of microprocessors, an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), or any combination thereof, and may be configured to execute instructions to perform operations in accordance with the disclosure herein. In some aspects, implementations of processor 111 may comprise code segments (e.g., software, firmware, and/or hardware logic) executable in hardware, such as a processor, to perform the tasks and functions described herein. In yet other aspects, processor 111 may be implemented as a combination of hardware and software. Processor 111 may be communicatively coupled to memory 112.
Memory 112 may comprise read only memory (ROM) devices, random access memory (RAM) devices, one or more hard disk drives (HDDs), flash memory devices, solid state drives (SSDs), other devices configured to store data in a persistent or non-persistent state, network memory, cloud memory, local memory, or a combination of different memory devices. Memory 112 may store instructions that, when executed by processor 111, cause processor 111 to perform operations in accordance with the present disclosure. In aspects, memory 112 may also be configured to facilitate storage operations. For example, memory 112 may comprise database 113 for storing one or more electronic documents, extracted text data, one or more predetermined element interpretation rule sets, detected element data, one or more intermediate files, an editing application, file type information, one or more conversion algorithms, one or more output files, input (e.g., from user terminal 160), other information, etc., which system 100 may use to provide the features discussed herein. Database 113 may be integrated into memory 112, or may be provided as a separate module. In some aspects, database 113 may be a single database, or may be a distributed database implemented over a plurality of database modules. In some embodiments, database 113 may be provided as a module external to server 110. Additionally, or alternatively, server 110 may include an interface configured to enable communication with data source 170, user terminal 160 (e.g., an electronic device), output terminal 162 (e.g., an electronic device), or a combination thereof.
Element detector 120 may be configured to detect one or more elements from the data corresponding to the electronic document. The data corresponding to the electronic document may be image data, such as a portable document format (pdf) file or other file format. The data may be generated by scanning a print document using a scanner, a camera, or another image capturing means, or the document may be electronically created (e.g., such as by a user operating a document creation and/or management application). In some implementations, the data does not include metadata, or any other type of information, indicating elements of the document, a structure of the document, markers in the document, or the like.
The detectable elements may include any type of element or component of a document, such as one or more text blocks, one or more labels, one or more fields (e.g., text input fields), one or more tables, one or more buttons, one or more check boxes, one or more particularly formatted fields (e.g., date fields, time fields, currency fields, signature fields, etc.), other elements, or a combination thereof. For example, a text block near another element may be a label, an empty line or box may be a text input field, a formation of text and lines in rows and columns may be a table, etc.
Element detector 120 may detect the elements in the electronic document based on a predetermined element interpretation rule set 121. For example, predetermined element interpretation rule set 121 may include rules for interpreting various detected objects, such as text, lines, shapes, etc., as elements based on characteristics such as position, distance relative to other objects, dimensions, object type, color, formatting, other characteristics, or a combination thereof. In some implementations, predetermined element interpretation rule set 121 includes text recognition rules, visual indicator detection rules, element dimension or size rules, label threshold rules, table detection rules, configurable element detection rules, signature detection rules, other rules, or a combination thereof. To illustrate, the text recognition rules may include one or more rules for performing optical character recognition (OCR) on text within the electronic document to generate text data (e.g., to “extract” the text data), such as character detection rules, space detection rules, word formation rules, lexical rules, sentence detection rules, etc. Alternatively, element detector 120 may perform OCR on the electronic document prior to performing element detection. As another example, the visual indicator detection rules may include rules for interpreting visual indicators as belonging to a particular element type, such as rules for interpreting a horizontal line with a particular amount of blank space above as an input field, rules for interpreting particular colors as corresponding to particular element types, or rules for interpreting particular shadings or pixel densities as corresponding to particular element types. As another example, the element dimension or size rules may include rules for interpreting shapes having particular dimensions or sizes, such as boxes or rectangles, as particular element types, such as input fields or tables, based on the dimensions or sizes. As another example, the label threshold rules may include rules indicating a threshold distance and/or direction between an indicator of an input field (or another element type) and a text block for use in interpreting the text block as a label of the input field (or other element type). The table detection rules may include rules for interpreting one or more boxes or rectangles or multiple parallel lines or multiple sets of perpendicular lines as a table based on positions relative to other boxes or rectangles or distances between parallel lines or alignment of intersections of perpendicular lines, rules for detecting a pattern of a table, and rules for interpreting a table as a repeating table based on empty entries (e.g., cells) within a table or multiple entries (e.g., rows or columns) that contain matching text. As another example, the configurable element detection rules may include rules for interpreting input fields within a threshold distance, input fields associated with labels indicating a selectable option (e.g., “Mark X or leave blank”), combinations of particular words (e.g., “Yes/No”), or other indicators as configurable elements, such as buttons or checkboxes. As another example, the signature detection rules may include rules for interpreting an input field as a signature field based on a position of the input field, dimensions of the input field, a label of the input field (or lack thereof), or other information. The above-described rules are illustrative and not to be considered limiting. In other implementations, predetermined element interpretation rule set 121 may include other types of rules. Performing element detection based on predetermined element interpretation rule set 121 is further described herein with reference to
In some implementations, predetermined element interpretation rule set 121 is static. For example, predetermined element interpretation rule set 121 may be preprogrammed at server 110. Alternatively, one or more rules included in predetermined element interpretation rule set 121 may be dynamically selected from a group of stored element interpretation rules. For example, the one or more rules may be selected based on user selected parameters received responsive to display of a GUI, as further described herein. In this manner, predetermined element interpretation rule set 121 may enable server 110 (e.g., element detector 120) to detect elements in an electronic document that would otherwise require associated metadata or user-input indicating the elements to identify.
In some other implementations, the data corresponding to the electronic document includes metadata indicating elements included in the electronic document. For example, the metadata may include position information, element types, dimensions, formatting information, expected input information, links to external resources (e.g., databases), etc. In response to detecting the metadata, element detector 120 may detect the elements in the electronic document based on the metadata. Using the metadata to detect the elements may be faster and use less processing resources than detecting elements in the electronic document based on predetermined element interpretation rule set 121. In some implementations, element detector 120 may be configured to perform a pass on the electronic document based on predetermined element interpretation rule set 121 even if metadata is detected, to ensure that no elements are missed.
Intermediate electronic form generator 122 may be configured to generate an intermediate file representing an electronic form based on the data. For example, the smart form may be an electronic document that supports enhanced features, such as user completion (e.g., web-fillable completion), linkages to databases or other data sources, electronic submission, or a combination thereof. The electronic form may include the text, graphics, and other elements included in the received electronic document. However, in the electronic form, one or more elements may be converted into elements capable of user-manipulation or otherwise interactive elements. For example, a blank line for receiving text in the electronic document may be converted to a text box in the electronic form that is configured to display text based on received user input and to optionally re-size based on the amount of text. As another example, a field that indicated selection between two options (e.g., as signified by “Circle one: yes/no”) may be replaced with a checkbox or a button. As yet another example, a field designated for receiving a date may be replaced by a calendar indicator that, when selected by a user, causes a pop-up window that includes a calendar with user-selectable dates.
The intermediate file has an intermediate file format. The intermediate file format may be a “uniform” file format that supports the features of an electronic form without being tied to any specific file format or electronic form application. In some implementations, the intermediate file format is a proprietary file format. The intermediate file format may include sufficient information (e.g., related to the elements of the electronic form) such that the intermediate file may be efficiently converted to one or more of multiple different output file types, as further described herein. Additionally, the electronic form represented by the intermediate file may be viewable and editable by an application, such as a proprietary electronic form manager application.
After generation of the intermediate file, the intermediate file may be provided to electronic form editor 123. Electronic form editor 123 may be configured to enable display of a graphical representation of the electronic form represented by the intermediate file. Additionally, electronic form editor 123 may be configured to enable the electronic form represented by the intermediate file to be edited based on user input. For example, a user (such as using user terminal 160), may be able to move elements, modify characteristics or properties of elements, add elements, delete elements, or make other edits to the electronic form. In some implementations, electronic form editor 123 may support such functionality in combination with one or more graphical user interfaces (GUIs) generated by UI manager 125.
File converter 124 may be configured to receive the intermediate file and to convert (e.g., publish) the intermediate file to an output file having a particular output file format. In some implementations, UI manager 125 may be configured to generate a GUI that enables user selection of a target output file format. In some other implementations, the particular output file format may be preset or preprogrammed, or selected based on a user that initiated the electronic form generation process or based on a destination for the output file. The particular output file format may be selected from one or more open source file formats, such as HTML, a proprietary electronic form file format (also referred to herein as “Vines” or “Kudzu Vines”), and/or one or more third-party supported electronic form file formats, such as K2 smart forms or SharePoint, as non-limiting examples. File converter 124 may be configured to convert the intermediate file into the output file using one or more conversion algorithms. For example, file converter 124 may be configured to format text, images, elements, metadata, links to external data, or other information from the intermediate file in accordance with formats, rules, or protocols associated with the output file format.
After generating the output file, file converter 124 may be configured to output the output file. For example, the output file may be output for storage, such as at database 113, an external database, data sources 170, or another storage location accessible via network 180. In some implementations, the output file may be made available for downloading to other devices, such as via network 180, the Internet, a private network of an enterprise, etc. Additionally or alternatively, the output file may be distributed to one or more other devices, such as output terminal 162, via network 180 for completion by one or more users. In some implementations, a customer (e.g., a user of user terminal 160) may contract to access server 110 to facilitate creation of electronic forms based on the customer's electronic documents (or publicly available electronic documents), such as from data sources 170. In some such implementations, the customer may provide user input, via user terminal 160, to select options of the electronic form generation process at server 110, to edit the electronic form represented by the intermediate file, to select options for the output format, or a combination thereof. In some other implementation, a customer may contract with a third party to generate electronic forms based on electronic documents, such as from data sources 170, for the customer. For example, an employee of the third party may use user terminal 160 to control one or more aspects of the electronic form generation process at server 110, and the output file may be provided to output terminal 162, which may be owned or operated by the customer.
UI manager 125 may be configured to generate one or more UIs, such as one or more GUIs, to display information and enable user selection of one or more parameters of the electronic form generation process. For example, UI manager 125 may be coupled to element detector 120, intermediate electronic form generator 122, electronic form editor 123, and file converter 124 to enable generation of UIs corresponding to the operations performed by each of the components 120-124. As an example, UI manager 125 may be configured to generate a UI configured to enable selection of a target output file format. As another example, UI manager 125 may be configured to generate a UI configured to enable user selection of one or more parameters associated with detection of the one or more elements by element detector 120. The one or more parameters may include one or more dimensions of input fields, one or more label thresholds, a type of indicator associated with the input fields, or a combination thereof. As another example, UI manager 125 may be configured to generate a UI configured to enable user configuration of the electronic form represented by the intermediate file. The UI may be part of a display initiated by electronic form editor 123 and may include an add option configured to add a field, an element, a control, or a combination thereof, to the electronic form, an edit option configured to edit one or more properties associated with one or more fields, one or more elements, one or more controls, or a combination thereof, included in the electronic form, a delete option configured to delete one or more fields, one or more elements, one or more controls, or a combination thereof, included in the electronic form, or a combination thereof. As another example, UI manager 125 may be configured to generate a UI configured to display measured results associated with generation of the intermediate file. The measured results may include a number of fields detected in the electronic document, a number of views detected in the electronic document, a percentage of detected views included in the electronic form, a percentage of detected fields included in the electronic form, a percentage of rules applied to generate the electronic form, a percentage of data sources linked to the electronic form, or a combination thereof. As another example, UI manager 125 may be configured to generate a UI configured to display one or more errors associated with generation of the intermediate file, one or more warnings associated with generation of the intermediate file, one or more to-do lists associated with generation of the intermediate file, or a combination thereof. Additional details regarding the UIs generated by UI manager 125 are further described with reference to
The database 113 may be coupled to element detector 120, intermediate electronic form generator 122, electronic form editor 123, file converter 124, UI manager 125, or a combination thereof. In some implementations, database 113 is configured to store one or more electronic documents (e.g., input files), extracted text data, one or more predetermined element interpretation rule sets, detected element data, one or more intermediate files, an editing application, file type information, one or more conversion algorithms, one or more output files (e.g., prior to distribution to output terminal 162), input (e.g., from user terminal 160), other information, or a combination thereof.
The functionality of server 110 will now be discussed with respect to the block flow diagram illustrated in
At block 210, data is received (e.g., at a receiver). For example, the data may include or correspond to an electronic document (or multiple electronic documents) and may be received from data sources 170. As described above, the data may not include metadata, or any other information, that indicates or identifies elements within the electronic document. The data corresponding to the electronic document may be image data, such as a pdf file or an image file, or another type of data, such as a word processing document. The data may be generated by scanning a print document using a scanner, a camera, or another image capturing means, or the electronic document may be electronically created (e.g., such as by a user operating a document creation and/or management application).
At block 212, one or more elements within the electronic document are detected. In some implementations, the electronic document (e.g., the data) does not include metadata that indicates or identifies elements within the electronic document. In such implementations, one or more elements are detected based on predetermined element interpretation rule set 121. In some other implementations, the electronic document (e.g., the data) includes such metadata, and one or more elements are detected or identified based on the metadata. To illustrate, the metadata may include positions, dimensions, and formats of input fields, positions, labels, and selectable options associated with checkboxes or buttons, positions, dimensions, numbers of rows, number of columns, and cell formatting associated with tables, links to external resources, other element information, and/or the like.
In some implementations in which the element detection is performed based on predetermined element interpretation rule set 121, the element detection may include one or more of the operations described with reference to blocks 222-230, other detection operations, or a combination thereof. To illustrate, any of the operations described with reference to blocks 222-230 may be performed based on one or more corresponding rules included in predetermined element interpretation rule set 121.
At block 222, optical character recognition (OCR) is performed on the data. Performing OCR may extract text data from the electronic document by recognizing characters, words, sentences, punctuation marks, other characters, or a combination thereof, from the electronic document. In some implementations, performing the OCR may include performing one or more preprocessing operations, one or more post-processing operations, or a combination thereof, to improve the accuracy of the extracted text data.
At block 224, label detection is performed. A text block may be identified as a label based on a comparison of a distance between the text block being and an input field indicator (e.g., a straight line, an empty box, etc.) to a threshold distance. For example, if the distance between a text block and an input field is less than or equal to (or less than) a threshold distance, the text block may be identified as a label associated with the input field. Alternatively, if the distance between the text block and the input field is greater than (or greater than or equal to) the threshold distance, the text block may be identified as an individual text block (or other element). In some implementations, the detection of a label may be further based on a position of a text block relative to an input field indicator. For example, if a text block is positioned horizontally to the left, or vertically above, the input field indicator, the text block may be identified as a label of the input field. Alternatively, if the text block is positioned horizontally to the right, vertically below, or diagonally from, the input field indicator, the text block may be identified as an individual text block (or other element). Additionally or alternatively, the detection of a label may be based on characteristics of the text included in a text block. For example, a text block including text having a particular text format (e.g., styles, such as bold or italics), a particular font color, or a particular font size (e.g., satisfying a threshold font size) may be identified as a label. Additionally or alternatively, a detection of a label may be based on detection of one or more keywords included in a text block. For example, if a text block includes one or more keywords, such as “signature,” “date,” “cost,” “time,” “result,” “answer,” “address,” “phone number,” “e-mail,” etc., the text block may be identified as a label. The above-described examples are illustrative, in other implementations, labels may be detected using other techniques.
At block 226, field detection is performed. An input field may be identified based on detection of one or more particular graphical indicators. The graphical indicators may include shapes or objects detected in the electronic document, such as horizontal lines with blank space above, empty boxes or rectangles, or shaded regions, as non-limiting examples. In some implementations, characteristics, such as dimensions, of the detected object are compared to thresholds to determine whether the detected object is an input field. For example, if a length of a line satisfies a threshold and/or a vertical dimension of empty space above the line satisfies a threshold, the line may be identified as an input field. As another example, if a horizontal dimension of a box satisfies a threshold or exceeds a vertical dimension of the box, the box may be identified as an input field. Additionally or alternatively, an input field may be detected based on a detected object having an associated label. For example, if a horizontal line or box is associated with a detected label, the line or box may be identified as an input field. Additionally or alternatively, an input field may be detected based on a position of a detected object. For example, if a detected object has a particular position, such as within an empty cell of a table or at the bottom of a last page of the electronic document, the detected object may be identified as an input field. The above-described examples are illustrative, in other implementations, input fields may be detected using other techniques.
In some implementations, a type of input field may be detected for one or more identified input fields. The type of input field may be based on text included in a label associated with the input field. For example, if the label includes one or more keywords, such as “cost,” “currency,” “dollar amount,” “payment,” “$,” etc., the input field may be identified as a currency field. As another example, if the label includes one or more keywords, such as “date,” “day,” “month,” “year,” etc., the input field may be identified as a data field. As another example, if the label includes one or more keywords, such as “time,” “arrival,” “departure,” “clock-in,” “clock-out,” “check-in,” “check-out,” “second,” “minute,” “hour,” etc., the input field may be identified as a time field. As another example, if the label includes one or more keywords, such as “signature,” “name,” “sign here,” “endorse,” etc., the input field may be identified as a signature field. Additionally or alternatively, the type of input field may be identified based on text included in a trailing text block. For example, if a text block is within a threshold distance to the right of the input field, and the text block includes one or more keywords indicating a type of unit, the type of input field may be identified. As an example, if a trailing text block includes “dollars,” the input field may be identified as a currency field. As another example, if a trailing text block includes “AM,” “PM,” or “AM/PM,” the input field may be identified as a time field. Other types of input fields may be identified, such as name fields, address fields, phone-number fields, e-mail fields, or identification (ID) number fields, as non-limiting examples.
At block 228, buttons, or other user-configurable or other interactive elements, are detected. The user-configurable or interactive elements may be detected based on one or more graphical indicators. For example, a square having particular dimensions may be identified as a check box. As another example, two (or more lines) each having a length that is shorter than a typical input field that are separated by a threshold distance, or a threshold distance and the word “or” or the characters “ . . . ” may indicate a group of input fields that can be converted into a button or dropdown menu. Additionally or alternatively, the user-configurable or interactive elements may be detected based on detection of keywords in a text block. For example, if a text block comprises one or more keywords, such as “yes/no,” “agree/disagree,” “approve/reject,” etc., the text block may be converted into a check box or button. Additionally or alternatively, the user-configurable or interactive elements may be detected based on detection of keywords in a label or nearby text block. For example, if a label or nearby text block includes one or more keywords, such as “circle [NUMBER] of the following,” “choose [NUMBER] of the following,” “select [NUMBER] of the following,” “mark each that apply,” “pick one or more options,” etc., the text block may be converted into one or more buttons or a dropdown menu. Additionally or alternatively, the user-configurable or interactive elements may be detected based on detection of a particular input field type. For example, a time field or a date field may be converted into a pop-up clock or a pop-up calendar in the electronic form, as further described herein. Additionally or alternatively, the user-configurable or interactive elements may be detected based on detection of a link. For example, a link (e.g., a uniform resource locator (URL), a file link, etc.) in the electronic document may be converted to content pointed to by the link, such as an image, a video, an audio file, an animation, other information, etc. The link may be identified based on the text (e.g., an address delineated by /s), based on a web address (e.g., http:// followed by additional text), and/or a particular keyword or shape predefined to indicate a link. The above-described examples are illustrative, in other implementations, user-configurable or interactive elements may be detected using other techniques.
At block 230, tables are detected. A table may be detected based on detection of multiple graphical indicators in the electronic document. For example, a table may be detected by detecting one or more boxes based on one or more rectangles. For example, if multiple rectangles (or squares) each having at least one side in contact with at least one side of an adjacent rectangle are detected, then the multiple rectangles may be identified as a table. Alternatively, a table may be detected by detecting boxes based on multiple sets of perpendicular lines. For example, if multiple parallel lines are each separated by no more than a threshold distance, and each of the multiple parallel lines is perpendicular to two or more perpendicular lines, boxes forming a table may be detected. Detecting a table may also include determining a pattern associated with the boxes in a horizontal direction, a vertical direction, or both. For example, a pattern of boxes having the same vertical dimension and located horizontally adjacent to one another may be determined to be a row, and a pattern of boxes having the same horizontal dimension and located vertically adjacent to one another may be determined to be a column. A grid may be built to represent the table. For example, the grid may represent the rows and columns of cells of the table. One or more entries in the table may be determined based on text included in the boxes, position of content included in the boxes, controls included in the boxes, or a combination thereof. For example, text in cells may be detected as labels or headers, in a similar manner to as described with reference to block 224. Additionally, blank cells may be detected as input fields. In some implementations, a compound header may be generated based on a type of entries in a row of a table and descriptors included in entries of an adjacent row of the table. For example, if a first row includes a header (e.g., “Totals), a second row includes labels (e.g., “Jan.,” “Feb.,” “March,” and “April”), and a third row includes input fields, a compound header may be generated that replaces the first row and the second row and includes compound labels (e.g., Jan. Total,” “Feb. Total,” “March Total,” and “April Total”).
In some implementations, a table may be identified as a repeating table. A repeating table refers to a table that is designated to have additional rows (or columns) added based on the amount of information to be entered by a user. A repeating table may be detected based on one or more rows of empty entries or one or more rows having entries that include the same content. For example, multiple rows of empty cells without intervening labels may indicate a repeating table. As another example, multiple rows with cells having the same entries, such as “0.00” for a table of values or “name,” “address,” “phone number,” and “e-mail,” for a table of personal information. In some implementations, a child data source may be generated for a repeating table in the electronic form, such as a database entry or other type of data source, to store unknown amount of data to be entered into the repeating table.
After the elements in the electronic document are detected, at block 214, an intermediate electronic form is generated. For example, an intermediate file representing an electronic form may be generated based on the electronic document. The intermediate file has an intermediate file type. As described above, the intermediate file type may be a universal file type that supports electronic form functionality and is able to be efficiently converted to multiple different output file types.
The electronic form represented by the intermediate file includes electronic form elements corresponding to each of the detected elements in the electronic document. For example, for each text block in the electronic document, the electronic form may include extracted text data from the corresponding text block. As another example, for each input field detected in the electronic document, the electronic form may include an input field configured to receive user input. To illustrate, the input fields in the electronic form may be configured to display text received via user input. In some implementations, the input fields may be modified or enhanced to provide improved functionality. Such functionality may include displaying a default value that is replaced by user entered values. Such functionality may also include formatting entries in the input fields based on a type of the input field. To illustrate, a currency field may format entries as a $ sign, followed by one or more digits, followed by a decimal point, followed by two digits; a time field may format entries as one or two digits, followed by a colon, followed by two digits; an employee ID field may format entries as three digits, followed by a hyphen, followed by three digits, followed by a hyphen, followed by three digits, as non-limiting examples. As another example, text blocks and/or input fields may be converted into user-configurable or interactive elements, such as check boxes, buttons, and dropdown menus, as described above. Other interactive elements, such as a signature field that auto-populates with a digital signature associated with a user account used to access the electronic form, or other interactive elements, are also supported. As another example, input fields may be converted to include interactive elements to facilitate user input. For example, a date field may have an attached calendar icon that causes a calendar to pop-up and enable selection of a particular date, or a time field may have an attached clock icon that causes a clock to pop-up and enable selection of a particular time, as non-limiting examples. As another example, links may be replaced with the content pointed to by the links, such as images, text, video content, audio content, etc. As another example, the electronic form may include a table with one or more cells including input fields, or a repeating table, with one or more rows (or columns) of input fields, and a button to add additional rows (or columns) to the repeating table. The above-described examples are illustrative, in other implementations, other types of elements may be included in the electronic form.
The intermediate file may include metadata associated with each of the elements in the electronic form. The metadata may include position information, dimension information, type information, text data, interactivity information, formatting information, linking information, other information, or a combination thereof, associated with each element in the electronic form. The metadata may formatted in a manner that enables conversion to one or more output file types. In some implementations, the metadata may support additional functionality, such as navigation to different sections or elements of the electronic form, linking the electronic form to different data sources or documents, securing the electronic form, etc.
At block 216, editing of the electronic form may be enabled. For example, if a user selects an edit option, the electronic form may be displayed and the user may be able to edit or modify aspects of the electronic form, such as by using user terminal 160. For example, the user may be able to add additional elements to the electronic form or delete one or more elements from the electronic form. Additionally or alternatively, the user may be able to modify the position of elements within the electronic form. For example, the user may use a user input device to select an element and move the element to a new position within the electronic form. Additionally or alternatively, the user may able to edit parameters associated with the elements within the electronic form. For example, the user may modify text within a text block, label, input field, table, etc., the user may change visual properties associated with the elements (e.g., color, font, size, shading, rotation, alignment, etc.), the user may change a type of an element (e.g., change a check box to a button, a table to a repeating table, etc.), the user may modify properties of the elements (e.g., formatting, dimensions, input type, external links, etc.), or other modifications. The above-described examples are illustrative, in other implementations, the user may perform other types of document modifications.
At block 218, file conversion is performed. To illustrate, the intermediate file may be converted to an output file having an output file format. The output file format may be selected from one or more open source file formats, such as HTML, a proprietary electronic form file format, and/or one or more third-party supported electronic form file formats, such as K2 smart forms or SharePoint, as non-limiting examples. In some implementations, the output file format is user selected. For example, a UI may be displayed that enables a user to select the output file format from one or more supported file formats. Alternatively, the output file format may be preprogrammed or based on other information, such as a user ID of a user controlling the electronic form generation process or a distribution target of the output file, as non-limiting examples. Converting the intermediate file to the output file may include converting or formatting the metadata included in the intermediate file in accordance with one or more rules or protocols associated with the output file format. Additionally or alternatively, converting the intermediate file to the output file may include setting one or more submission parameters, such as a format of submitted data, a location for submitted data, etc., one or more security parameters, such as access credentials required to open or submit the electronic form, an encryption protocol used to encrypt the electronic form (or the submitted data), etc., or a combination thereof.
At block 220, the output file (e.g., representing the final version of the electronic form) is output. For example, the output file may be stored locally (e.g., at server 110, such as at database 113) or at a remote location via network 180. Additionally or alternatively, the output file may be distributed to one or more user devices, such as output terminal 162, or to a network location that enables downloading the output file to the one or more user devices.
As described with reference to
In the particular example shown in
During an electronic form generation process based on electronic document 300, one or more of the elements may be detected. For example, first input text box 304 may be detected based on identification of a rectangle with no text inside. First label 306 may be detected based on first label 306 being within a threshold distance to the left of first input text box 304. Additionally, second input text box 308 may be detected based on identification of another rectangle with no text inside. Second label 310 may be detected based on second label 310 being within a threshold distance above second input text box 308. Header 302 may be detected based on a position of header 302, a font size of the text of header 302, other characteristics, or a combination thereof. Other elements may be similarly detected, as further described with reference to
First UI 400 may also include electronic form display options 404. Electronic form display options 404 may include one or more selectable indicators that enable display (or hiding) of various electronic forms. In some implementations, electronic form display options 404 include a new option to enable display of new electronic forms, a wizard option to enable display of electronic forms available through a wizard, a prettify option to enable display of electronic forms that have been “prettified” (e.g., visually enhanced), and a show archived option to enable display of archived electronic forms. In other implementations, electronic form display options 404 may include fewer options than shown in
First UI 400 may also include a filter option window 406. Filter option window 406 may include a first button to apply selected filter(s) and a second button to reset the selected filter(s). Filter option window 406 may also include filter options that may be configurable to filter display of one or more stored electronic forms 402. In the example shown in
Selection of one of the one or more stored electronic forms 402 may cause one or more additional UIs to be displayed. The one or more additional UIs may display additional information associated with the selected electronic form or may enable one or more operations with respect to the selected electronic form, such as editing of the selected electronic form, data entry in the selected electronic form, distribution of the selected electronic form, other operations, or a combination thereof.
Second UI 500 includes multiple input file selection options associated with the new electronic form. In the particular example shown in
Second UI 500 also includes multiple settings that are configurable via selection of multiple tabs. For example,
Second UI 500 also includes a reset button 532 and one or more navigation buttons. Reset button 532, when selected, may cause the input file selection options and the additional settings to be reset to default selections. The one or more navigation buttons may include a previous button 534, a next button 536, and a finish button 538. Previous button 534, when selected, may cause display of a previous window corresponding to one of window indicators 502-508. Next button 536, when selected, may cause display of a next window corresponding to one of window indicators 502-508. Finish button 538, when selected, may finish the electronic form generation process.
To illustrate, the vision settings may include general recognition settings 540, input recognition settings 550, and label recognition settings 552. General recognition settings 540 may include one or more settings associated with computer vision and detection operations to be performed on the input file. For example, general recognition settings 540 may include an enablement option 542, an override option 544, a detect tables option 546, and a detect repeating tables option 548. Enablement option 542 may enable detection of one or more elements in the input file, such as text blocks, input fields, labels, etc. Override option 544 may override the default detection settings. Detect tables option 546 and detect repeating tables option 548 may enable detection of tables, and repeating tables, as described with reference to
To illustrate, third UI 600 may include electronic form generation results 602. Electronic form generation results 602 may include one or more results of generating the intermediate file (e.g., representing the electronic form). For example, electronic form generation results 602 may include a reader used to read the input file (e.g., corresponding to an input file format), a form name of the electronic form, a number of fields detected in the input file, a number of views (e.g., container(s) or groupings of various elements) detected in the input file, a status of the electronic form generation (e.g., success if the electronic form is generated or failure if the electronic form is unable to be generated), a preview option for the electronic form, an edit option for the electronic form, a duration of the electronic form generation process, a number of to-do items associated with the electronic form, a number of warnings associated with the electronic form, a number of errors associated with the electronic form, other results associated with the electronic form, or a combination thereof. In the example of
Third UI 600 may also include one or more graphical indicators configured to indicate results of generating the electronic form based on the input file. In some implementations, the graphical indicators include a first graphical indicator 604 (e.g., a “Views” indicator), a second graphical indicator 606 (e.g., a “Fields” indicator), a third graphical indicator 608 (e.g., a “Rules” indicator), and a fourth graphical indicator 610 (e.g., a “Data Sources” indicator). First graphical indicator 604 may display a percentage of views detected in the input file that are included in the electronic form. For example, first graphical indicator 604 may include a semi-circle corresponding to percentages from 0-100, and an arrow may indicate the percentage of the views that are included in the electronic form (e.g., 100% in the example of
Fourth UI 700 may include buttons 702-710, a display pane 712, a list of form fields 714, and form element properties 716. The buttons may include buttons that enable control of general settings or operations associated with fourth UI 700. For example, the buttons may include a form name button 702, a view button 704, a grid button 706, a save button 708, and a close button 710. Form name button 702 may display the form name of the electronic form and, via selection of form name button 702, may enable the electronic form to be renamed. View button 704 may enable selection of one or more different views for display pane 712, such as a current view, a full-screen view, one or more zoomed views, a split view, or another type of view. Grid button 706 may enable a grid to be displayed (or hidden) on display pane 712. Save button 708 may enable any modifications made to the electronic form to be saved (e.g., to update the intermediate file). Close button 710 may close the electronic form editor and return to the next window of the electronic form generation procedure.
Display pane 712 may be configured to display at least a portion of the electronic form represented by the intermediate file. The displayed electronic form may include electronic form elements, such as text blocks (e.g., a header (“Health and Safety in the Construction Sector Course”), instructions (“Please complete and submit this form”), additional information (“Duration: 10 hours” “Venue: 17, Edgar Ferro Street”), etc.), input fields (e.g., date fields, company name field, address field, VAT number field, telephone number field, contact field, etc.), labels associated with the input fields (“Date from:”, “Date to:”, “Agency/Department/Company”, “Address”, “VAT Number”, “Telephone”, “Contact Person”, “Position”, etc.), interactive elements (e.g., calendar icons included in the date fields), other types of electronic form elements, or a combination thereof, as further described with reference to
List of form fields 714 includes a list of identifiers of the various fields included in the electronic form. Selection of one of the field identifiers from list of form fields 714 may cause selection (e.g., highlighting) of the corresponding field in display pane 712. Although the field IDs are illustrated as numeric values, in other implementations, the field IDs may be any alphabetic, numeric, or alphanumeric value that identifies the various fields.
Form element properties 716 may indicate one or more properties of selected element 718 within display pane 712. In the example of
Fifth UI 800 may include a writer selection option 802. Writer selection option 802 may enable user selection of a writer (e.g., an output file format) for the output file. The output format may be one of HTML, Vines, K2, or SharePoint, as non-limiting examples. In some implementations, writer selection option 802 may include a dropdown menu that includes all the different output file formats that are supported.
Fifth UI 800 also includes multiple settings that are configurable via selection of one or more tabs. In some implementations, the tabs are specific to the selected writer (e.g., output file format). In some other implementations, at least one tab is common to all writers (e.g., output file formats). In the particular example of
Sixth UI 900 may include output file generation results 902. Output file generation results 902 may include one or more results of generating the output file (e.g., representing the electronic form and having the output file format). For example, output file generation results 902 may include a writer used to write (e.g., generate) the output file (e.g., corresponding to the output file format), a form name of the electronic form, a number of fields processed from the intermediate file, a number of views processed from the intermediate file, a status of the output file generation (e.g., success if the output file is generated or failure if the output file is unable to be generated), a duration of the output file generation process, a number of to-do items associated with the output file, a number of warnings associated with the output file, a number of errors associated with the output file, other results associated with the output file, or a combination thereof. In some implementations, the to-do items, warnings, or errors are described in a popup window or another UI, as further described with reference to
Sixth UI 900 may also include one or more graphical indicators configured to indicate results of generating the output file based on the intermediate file. In some implementations, the graphical indicators include a first graphical indicator 904 (e.g., a “Views” indicator), a second graphical indicator 906 (e.g., a “Fields” indicator), a third graphical indicator 908 (e.g., a “Rules” indicator), and a fourth graphical indicator 910 (e.g., a “Data Sources” indicator). First graphical indicator 904 may display a percentage of views from the intermediate file that are included in the output file. For example, first graphical indicator 904 may include a semi-circle corresponding to percentages from 0-100, and an arrow may indicate the percentage of the views that are included in the output file (e.g., 100% in the example of
To illustrate, seventh UI 1000 may include a list 1002 of information items, to-do items, warnings, and errors associated with generation of the output file. Each entry in list 1002 may include multiple details associated with the entry, such as a type of the entry (e.g., informational element, to-do item, warning, or error), a category associated with the entry, a timestamp indicating when the entry was created, a message included in the entry, other information, or a combination thereof. In the particular example of
Seventh UI 1000 may also include filter options 1008. Filter options 1008 may include one or more options for filtering entries displayed in list 1002. For example, filter options 1008 may include one or more options for displaying particular entries, one or more options for hiding particular entries, or both. In some implementations, filter options 1008 may include one or more type options and one or more category options. The type options may include an all option configured to enable display of all entries included in list 1002, an info option configured to enable display of the information elements included in list 1002, a warning option configured to enable display of the warnings included in list 1002, an error option configured to enable display of the errors included in list 1002, and a to-do option configured to enable display of the to-do elements included in list 1002. The category options may include options configured to enable display entries corresponding to each type of category, such as all categories, general, form, view, rule, datasource, fields, controls, layouts, style, and recognition. The above-described options are illustrative, and in other implementations, filter options 1008 may include fewer options than shown in
In the particular example shown in
One or more elements of electronic form 1100 may include enhanced functionality as compared to electronic document 300 of
As described with reference to
Method 1200 includes receiving data corresponding to an electronic document, at 1202. The electronic document may have an input file format. For example, server 110 may receive an electronic document from data sources 170 via network 180. The received electronic document may have an input file format. Method 1200 also includes detecting one or more elements of the electronic document from the data based on a predetermined element interpretation rule set, at 1204. For example, element detector 120 may detect one or more elements of the electronic document based on predetermined element interpretation rule set 121.
Method 1200 includes generating an intermediate file representing an electronic form based on the data, at 1206. The intermediate file may include metadata indicating the one or more elements, and the intermediate file may have an intermediate file format. For example, intermediate electronic form generator 122 may generate an intermediate file that represents an electronic form and that includes the one or more elements detected by element detector 120. The intermediate file may have an intermediate file format. Method 1200 also includes displaying a user interface configured to enable user selection of a target output file format, at 1208. For example, UI manager 125 may initiate display of fifth UI 800 of
Method 1200 includes converting the intermediate file to an output file having the target output file format, at 1210. The output file may represent the electronic form configured to support user entry of information. For example, file converter 124 may convert the intermediate file generated by intermediate electronic form generator 122 to an output file having the target output file format. The output file may represent the electronic form, configured in accordance with one or more rules or protocols associated with the target output file format. Method 1200 further includes outputting the output file, at 1212. For example, file converter 124 may output the output file for storage, either at server 110 or another device, or for distribution to one or more other devices.
In some implementations, the data does not include (e.g., omits) metadata indicating one or more predefined elements of the electronic document. For example, the electronic document may include or correspond to a scanned document or a document generated by a word processing or document management application that does not generate metadata indicating elements of the document. Additionally or alternatively, the one or more elements may include one or more labels, one or more fields, one or more text blocks, one or more tables, one or more buttons, one or more check boxes, one or more date fields, one or more time fields, one or more currency fields, one or more signature fields, or a combination thereof. For example, electronic document 300 may include labels, fields, text blocks, tables, buttons, check boxes, date fields, time fields, currency fields, signature fields, other elements, or a combination thereof. Additionally or alternatively, the predetermined element interpretation rule set may include visual indicator detection rules, element dimension or size rules, label threshold rules, table detection rules, repeating table detection rules, configurable element detection rules, signature detection rules, or a combination thereof. For example, predetermined element interpretation rule set 121 may include the rules described with reference to
In some implementations, method 1200 also includes, prior to detecting the one or more elements of the electronic document, displaying a second user interface configured to enable user selection of one or more parameters associated with detection of the one or more elements. For example, UI manager 125 may initiate display of second UI 500 of
In some implementations, method 1200 also includes, after generating the intermediate file, displaying a third user interface configured to enable user configuration of the electronic form represented by the intermediate file. For example, UI manager 125 may initiate display of fourth UI 700 of
In some implementations, detecting the one or more elements based on the predetermined element interpretation rule set may include detecting a particular type of input field based on one or more indicators in the electronic document corresponding to an input field and a label associated with the one or more indicators. The label may be located within a threshold distance of the one or more indicators in the electronic document. For example, element detector 120 may be configured to perform label detection, as described with reference to block 224, and field detection, as described with reference to block 226, of
In some implementations, the input file format includes a portable document format (PDF), and the target output file format includes an electronic form application file format. For example, server 110 may receive a PDF file (or other image data) corresponding to the electronic document, and file converter 124 may generate the output file having an open-source format, such as HTML, a proprietary format, such as Vines, or a third-party supported format, such as K2 or SharePoint, as non-limiting examples. Additionally or alternatively, the intermediate file format may include a uniform format that is convertible to multiple different output file formats. For example, the intermediate file generated by intermediate electronic form generator 122 may have a universal file format that is convertible to multiple different output file formats (e.g., electronic form application formats), such as based on one or more protocols, rulesets, or conversion algorithms associated with the output file formats.
In some implementations, method 1200 also includes initiating display of a fourth user interface configured to display measured results associated with generation of the intermediate file. The measured results may include a number of fields detected in the electronic document, a number of views detected in the electronic document, a percentage of detected views included in the electronic form, a percentage of detected fields included in the electronic form, a percentage of rules applied to generate the electronic form, a percentage of data sources linked to the electronic form, or a combination thereof. For example, UI manager 125 may initiate display of third UI 600 of
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. Skilled artisans will also readily recognize that the order or combination of components, methods, or interactions that are described herein are merely examples and that the components, methods, or interactions of the various aspects of the present disclosure may be combined or performed in ways other than those illustrated and described herein.
Functional blocks and modules in
The steps of a method or algorithm described in connection with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal, base station, a sensor, or any other communication device. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary designs, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Computer-readable storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, a connection may be properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL), then the coaxial cable, fiber optic cable, twisted pair, or DSL, are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods, and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
Claims
1. A method for converting legacy electronic forms and static electronic documents to web-fillable electronic forms, the method comprising:
- detecting one or more elements of an electronic document from based on a predetermined element interpretation rule set, the electronic document lacking metadata indicating any predefined elements of the electronic document, the predetermined element interpretation rule set including at least visual indicator detection rules for detection of graphical indicators of the one or more elements in the electronic document, the one or more elements including one or more labels, one or more fields, or one or more elements corresponding to user-configurable or interactive elements;
- generating an intermediate file representing an electronic form based on the electronic document, the intermediate file including metadata indicating the one or more elements, the intermediate file having an intermediate file format, wherein generating the intermediate file includes converting a first text element of the one or more elements of the electronic document to a first user-configurable or interactive element within the electronic form; and
- displaying a form editing interface configured to enable user configuration of the electronic form represented by the intermediate file, the form editing interface configured to enable editing of properties of content of the electronic form based on user input indicating selection or manipulation of the content within the form editing interface.
2. The method of claim 1, wherein the properties of the content include color, font size, shading, rotation, alignment, or a combination thereof.
3. The method of claim 1, wherein the properties of the content include formatting properties, dimensions of the content, an input type corresponding to the content, an external link associated with the content, or a combination thereof.
4. The method of claim 1, wherein the properties of the content include a type of the content selected from text, a check box, a button, a dropdown menu, a pop-up clock, a pop-up calendar, a digital signature field, a table, or a repeating table.
5. The method of claim 1, wherein converting the first text element to the first user-configurable or interactive element within the electronic form comprises converting the first text element to a checkbox, a button, a dropdown menu, a pop-up clock, a pop-up calendar, or a digital signature field.
6. The method of claim 1, wherein the one or more elements further include one or more text blocks, one or more date fields, one or more time fields, one or more currency fields, one or more signature fields, or a combination thereof.
7. The method of claim 1, wherein the predetermined element interpretation rule set further includes element dimension or size rules for interpreting dimensions or sizes as indicative of the one or more elements in the electronic document and label threshold rules for detecting labels and applying the labels to fields in the electronic document based on distances between the labels and the fields.
8. The method of claim 1, further comprising, prior to detecting the one or more elements of the electronic document, displaying a second user interface configured to enable user selection of one or more parameters associated with automatic detection of the graphical indicators of the one or more elements within the electronic document, wherein the one or more parameters include one or more label thresholds associated with detection of labels of input fields and types of graphical indicators associated with detection of the input fields.
9. The method of claim 1, wherein detecting the one or more elements based on the predetermined element interpretation rule set comprises detecting a particular type of input field based on one or more indicators in the electronic document corresponding to an input field and a label associated with the one or more indicators, the label located within a threshold distance of the one or more indicators in the electronic document.
10. The method of claim 1, wherein detecting the one or more elements in the electronic document comprises performing optical character recognition (OCR) on the electronic document and detecting one or more objects or shapes in electronic document.
11. The method of claim 1, further comprising:
- converting the intermediate file to an output file having a target output file format, the output file representing the electronic form configured to support user entry of information and including the first user-configurable or interactive element; and
- outputting the output file.
12. An apparatus for converting legacy electronic forms and static electronic documents to web-fillable electronic forms, the apparatus comprising:
- a processor; and
- a memory coupled to the processor and storing instructions executable by the processor to cause the processor to: detect one or more elements of an electronic document based on a predetermined element interpretation rule set, the electronic document lacking metadata indicating any predefined elements of the electronic document, the predetermined element interpretation rule set including at least visual indicator detection rules for detection of graphical indicators of the one or more elements in the electronic document, the one or more elements including one or more labels, one or more fields, or one or more elements corresponding to user-configurable or interactive elements; generate an intermediate file representing an electronic form based on the electronic document, the intermediate file including metadata indicating the one or more elements, the intermediate file having an intermediate file format, wherein the processor is configured to generate the intermediate file by converting a first text element of the one or more elements of the electronic document to a first user-configurable or interactive element within the electronic form; and initiate display of a form editing interface configured to enable user configuration of the electronic form represented by the intermediate file, the form editing interface configured to enable editing of properties of content of the electronic form based on user input indicating selection of manipulation of the content within the form editing interface.
13. The apparatus of claim 12, further comprising:
- a database coupled to the processor, the database configured to store intermediate files, output files, input files, the predetermined element interpretation rule set, or a combination thereof.
14. The apparatus of claim 12, further comprising:
- an interface coupled to the processor and configured to enable communication with a data source that stores the electronic document, an electronic device, or a combination thereof.
15. The apparatus of claim 12, wherein the instructions are further executable by the processor to cause the processor to:
- initiate display of a user interface configured to enable user selection of a target output file format;
- convert the intermediate file to an output file having the target output file format, the output file representing the electronic form configured to support user entry of information and including the first user-configurable or interactive element; and
- output the output file.
16. The apparatus of claim 15, wherein an input file format of the electronic document comprises an image file format or a word processing document file format, and wherein the target output file format comprises a smart form application file format that supports electronic completion of the electronic form, user interaction with the electronic form, and electronic submission functionality.
17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
- detecting one or more elements of an electronic document based on a predetermined element interpretation rule set, the electronic document lacking metadata indicating any predefined elements of the electronic document, the predetermined element interpretation rule set including at least visual indicator detection rules for detection of graphical indicators of the one or more elements in the electronic document, the one or more elements including one or more labels, one or more fields, or one or more elements corresponding to user-configurable or interactive elements;
- generating an intermediate file representing an electronic form based on the electronic document, the intermediate file including metadata indicating the one or more elements, the intermediate file having an intermediate file format, wherein generating the intermediate file includes converting a first text element of the one or more elements of the electronic document to a first user-configurable or interactive element within the electronic form; and
- initiating display of a form editing interface configured to enable user configuration of the electronic form represented by the intermediate file, the form editing interface configured to enable editing of properties of content of the electronic form based on user input indicating selection or manipulation of the content within the form editing interface.
18. The non-transitory computer readable medium of claim 17, wherein the operations further comprise:
- initiating display of a user interface configured to enable user selection of a target output file format;
- converting the intermediate file to an output file having the target output file format, the output file representing the electronic form configured to support user entry of information and including the first user-configurable or interactive element; and
- outputting the output file.
19. The non-transitory computer readable medium of claim 17, wherein the operations further comprise initiating display of a fourth user interface configured to display measured results associated with generation of the intermediate file, the measured results including a total number of detected fields in the electronic document that are successfully included in the electronic form, a percentage of rules applied to generate the electronic form from the electronic document, a percentage of data sources linked to the electronic form, or a combination thereof.
20. The non-transitory computer readable medium of claim 17, wherein the properties of the content include a data format, a requirement property, a read-only property, a visibility property, or a combination thereof.
Type: Application
Filed: Jun 13, 2022
Publication Date: Sep 29, 2022
Inventors: Robert Thomas D'Oria (Dexter, MI), Eric Eichler (Cumming, GA), Ron Douglass (Roswell, GA)
Application Number: 17/839,394
