User generated tag collection system and method
In some embodiments, an input document is received at a tagging platform, via a communication device, and associates it with a tag request. The tagging platform automatically selects at least one electronic record associated with a first user from a user data store containing electronic records associated with users (each record including at least a user identifier and a user communication address). The input document and tag request are transmitted to the communication address associated with the first user, and a document tag is received from the first user. The tagging platform may then store the document tag in a document mining result database by adding an entry to the database identifying the received document tag and transmit an indication associated with the document mining result database to a plurality of risk applications.
Latest HARTFORD FIRE INSURANCE COMPANY Patents:
An entity, such as an enterprise that analyzes risk information, may want to analyze large amounts of data, such as image data. For example, a risk enterprise might want to analyze tens of thousands of image files to look for patterns (e.g., a particular type of damage has occurred more frequently under particular circumstance). Note that an entity might analyze this data in connection with different types of risk-related applications, and, moreover, different applications may need to analyze the data differently. For example, a picture of a business or residence might have different meanings depending on the types of risk being evaluated. It can be difficult to identify patterns across such large amounts of data and different types of applications. In addition, manually managing the different needs and requirements (e.g., different business logic rules) associated with different applications can be a time consuming and error prone process.
Artificial Intelligence (“AI”) can accelerate the rate at which an enterprise can analyze data. AI capabilities, however, rely upon the manual curation of a “golden” or “ground truth” datasets (that typically involves a significant amount of time) and the strength of the AI is directly correlated to the accuracy and quality of the golden dataset. Note that creating a high-quality dataset to train machine learning and similar AI models can be a costly and time-consuming project.
As a result, it would be desirable to provide systems and methods to efficiently and accurately tag documents (including traditional document files, images, structured and unstructured text, audio and video files) in connection with risk-related and other applications.
SUMMARY OF THE INVENTIONAccording to some embodiments, systems, methods, apparatus, computer program code and means are provided for efficiently and accurately tagging document data. In some embodiments, an input document is received at a tagging platform, via a communication device, and associates it with a tag request. The tagging platform automatically selects at least one electronic record associated with a first user from a user data store containing electronic records associated with users (each record including at least a user identifier and a user communication address). The input document and tag request are transmitted to the communication address associated with the first user, and a document tag is received from the first user. The tagging platform may then store the document tag in a document mining result database by adding an entry to the database identifying the received document tag and transmit an indication associated with the document mining result database to a plurality of risk applications.
Some embodiments provide: means for receiving an input document at a tagging platform via a communication device; means for associating the input document with a tag request; means for automatically selecting at least one electronic record associated with a first user from a user data store containing electronic records associated with users, each electronic record including at least a user identifier and a user communication address; means for transmitting the input document and tag request to the communication address associated with the first user; means for receiving a document tag from the first user; means for storing the document tag in a document mining result database by adding an entry to the database identifying the received document tag; and means for transmitting an indication associated with the document mining result database to a plurality of risk applications
A technical effect of some embodiments of the invention is an improved and computerized way of tagging document information to provide improved results for risk-related and other applications. With these and other advantages and features that will become hereinafter apparent, a more complete understanding of the nature of the invention can be obtained by referring to the following detailed description and to the drawings appended hereto.
The present invention provides significant technical improvements to facilitate a monitoring, tagging, and/or processing of document-related data, predictive image and risk related data modeling, and dynamic data processing. The present invention is directed to more than merely a computer implementation of a routine or conventional activity previously known in the industry as it significantly advances the technical efficiency, access and/or accuracy of communications between devices by implementing a specific new method and system as defined herein. The present invention is a specific advancement in the areas of document monitoring, tagging, and/or processing by providing benefits in data accuracy, analysis speed, data availability, and data integrity, and such advances are not merely a longstanding commercial practice. The present invention provides improvement beyond a mere generic computer implementation as it involves the processing and conversion of significant amounts of data in a new beneficial manner as well as the interaction of a variety of specialized risk-related applications and/or third-party systems, networks and subsystems. For example, in the present invention document related risk information may be processed, tagged, forecast, and/or predicted via an analytics engine and results may then be analyzed efficiently to evaluate risk-related and other data, thus improving the overall performance of an enterprise system, including message storage requirements and/or bandwidth considerations (e.g., by reducing a number of messages that need to be transmitted via a network). Moreover, embodiments associated with predictive models might further improve the performance of claims processing applications, resource allocation decisions, reduce errors in templates, improve future risk estimates and document tags, etc.
Some embodiments described herein are associated with a web application (or similar program) that lets data scientists crowd source document labeling (“tagging”) work to users, such as employees (e.g., who volunteer or opt-in), to accelerate document tagging and bring costs down to enable underwriting and other use cases. For example, an image factory may use AI to let an underwriting process make better decisions, reduce repetitive or redundant workflow components, bring automation to new and renewal low risk business, and/or identify complex cases that may require further (manual) review. For example, in some embodiments, a web application may let employees label images using a link sent to their e-mail. The application may track users, images, labels, timing, and/or tagging campaign information as desired. In some embodiments, users and campaigns may be manually configured by a data science team.
The back-end application computer server 150 and/or the other elements of the system 100 might be, for example, associated with a Personal Computer (“PC”), laptop computer, smartphone, an enterprise server, a server farm, and/or a database or similar storage devices. According to some embodiments, an “automated” back-end application computer server 150 (and/or other elements of the system 100) may facilitate communications with remote user devices 160 and/or updates of electronic records in the document library 110 and user data store 112. As used herein, the term “automated” may refer to, for example, actions that can be performed with little (or no) intervention by a human.
As used herein, devices, including those associated with the back-end application computer server 150 and any other device described herein, may exchange information via any communication network which may be one or more of a Local Area Network (“LAN”), a Metropolitan Area Network (“MAN”), a Wide Area Network (“WAN”), a proprietary network, a Public Switched Telephone Network (“PSTN”), a Wireless Application Protocol (“WAP”) network, a Bluetooth network, a wireless LAN network, and/or an Internet Protocol (“IP”) network such as the Internet, an intranet, or an extranet. Note that any devices described herein may communicate via one or more such communication networks.
The back-end application computer server 150 may store information into and/or retrieve information from the document library 110, the user data store 112, and a document tagging database 170. The document tagging database 170 may store document tags supplied by users and be utilized to train AI models. The document library 110 and user data store 112 may also contain information about prior and current interactions with users, including those associated with the remote user devices 160 (e.g., user preference values associated with data formats, protocols, etc.). The document library 110 and user data store 112 may be locally stored or reside remote from the back-end application computer server 150. As will be described further below, the document library 110 and user data store 112 may be used by the back-end application computer server 150 in connection with an interactive user interface to gather information for the automated document tagging platform 155. Although a single back-end application computer server 150 is shown in
In this way, the system 100 may tag document information in an efficient and accurate manner. For example,
At 202, the system may receive an input “document” at a tagging platform via a communication device. As used herein, the term “document” may refer to an image (e.g., a satellite photograph, a drone picture, a street-view picture, etc.), a word processing document (e.g., a police or medical report), a video, an audio file, etc. Also note that a document may be newly “received” by an enterprise (e.g., it might arrive from an external source or a separate internal system via an electronic message). In some embodiments, a document may instead be “received” by accessing an existing document (e.g., an image file might be retrieved from an archive or warehouse that contains a collection of insurance-related images taken over the past 20 years). At 204, the system may associate the input document with a tag request. For example, the tag request might ask a user “is there visible damage to the roof in this picture?” As another example, the tag request might ask a user “does the customer's voice seem angry or annoyed in this audio clip?” As still another example, a campaign to identify boarded windows may ask users to select “yes” if boarded windows are present in an image or “no” if no boarded windows are present. Note that the tag request does not need to be a “yes” or “no” question. For example, a campaign to identify roof condition may ask users to select whether a roof is “below average,” “average,” or “above average.” Similarly, a tag request might ask for a numerical rating (e.g., from “1” through “10”).
At 206, at least one electronic record (associated with a first user) may be automatically selected from a user data store containing electronic records associated with users, each electronic record including at least a user identifier and a user communication address. By way of examples, the communication address might refer to a postal address, an email address, a smartphone telephone number, a link, a username and password, or any other communication link that might be automatically established by the system.
The input document and tag request may then be transmitted to the communication address associated with the first user at 208. At 210, the system may receive a document tag from the first user and store the document tag in a document mining result database (by adding an entry to the database identifying the received document tag) at 212. Note that, as used herein, the term “tag” might refer to any object describing a document. Examples of tags might include, an asphalt condition, an estimated building age, a building style, the presence of concrete, a building condition, siding materials, an angle of view, a fence, barbed wire, chipped paint, a dirt patch, manicured plants, a boarded window, a broken window, an overall image quality, etc.
At 214, the system may transmit an indication associated with the document mining result database to a plurality of risk applications (e.g., to train AI models to be used in connection with an underwriting process, a claims handling procedure, etc.).
A user interface 350 may then be used to collect tagging decisions from users. According to some embodiments, a quality control process 330 may inject a golden data set (tagged by experts) to automatically perform quality control on new tags submitted by novice/non-experts such as the employees 320 (e.g., to facilitate crowd sourcing/scaling-up the use case 300). The campaign data may then be saved into a tagging result database 370. The results might include, for example, a campaign identifier, a campaign start date, a campaign end date, image tag data (including image and user identifiers, user tag selections, an amount of time taken to tag the image), etc.
According to some embodiments, users or employees might receive a reward, such as an icon or badge, in exchange for participating in a campaign. For example, a badge might be displayed on an employee's computer screen after he or she tags 2,000 documents to broaden the AI capabilities of an enterprise by helping to train a model identify patterns from images, video/voice recordings, documents, etc. using large volumes of manually labeled data.
In some cases, the system may transmit a series of tag requests to one or more users that is dynamically adjusted based on prior tags generated by that user (or by other users). For example,
The back-end application computer server 550 may store information into and/or retrieve information from a user data store 510. The user data store 510 might, for example, store electronic records 512 representing a plurality of insurance company employees, each electronic record including a user identifier 514, a user quality 516 (e.g., described in connection with
By way of example, to support AI model development the system 500 might tag or label over 9,000 property survey images. Three different people may apply tags to each image. Note that consistency in tagging may provide confidence in labeling accuracy. Key factors that may impact low tagging consistency and strategies for mitigation may include:
-
- the number and availability of labelers (which might be mitigated by an on-demand team scale);
- domain expertise (which might be mitigated by provide precise task definitions and examples);
- variation in perception of task (which might be mitigated by encouraging accuracy and consistency across labelers using multiple votes to increase confidence);
- complexity of tagging task (which might be mitigated by split tasks into multiple iterations to first identify windows and then assess condition);
- difficulty in accurately applying a variety of tags at once (which might be mitigated by limiting the number of tags per campaign by splitting different concepts into separate tasks);
- lack of review during tagging process (which might be mitigated by implementing quality assurance early and often in the tagging process);
- user interface limitations (which might be mitigated by including integrated samples with instructions easily visible); and
- inability to track multiple taggers (which might be mitigated by tool to enable parallelism of tagging for the same image set).
One way to improve the quality of tags is to combine tags from multiple users (e.g., a final tag is only assigned if 75% of all users agree that the tag is appropriate).
The input document and tag request may then be transmitted to the set of communication addresses associated with the multiple users at 608. At 610, the system may receive document tags from the multiple users. At 612, the system may aggregate document tags. For example, the system may treat each tag as a “vote” and determine which tag received the most votes. In some embodiments, a close vote (e.g., 55% of users indicate that barbed wire is present) is automatically and dynamically supplemented by requesting additional tags from other users (e.g., other employees, experts, managers, etc.).
In some embodiments, a privilege or reward might be provided to users who supply tags. For example,
In this way, the insurer may use the tags to harness computer vision and extract custom insights from images for underwriters and claim teams. Note that AI models might be able to understand patterns related to neglect, roof condition, parking lot condition, etc. These models, however, need labeled data to work (and the more data, the better). Manually curating data is expensive, time consuming, and inefficient to scale with high quality. Some embodiments described herein use an internally hosted web application to enable crowdsourcing of labeling work within the company. If every employee spends five minutes labeling images per day, an relatively large enterprise could collect tags for millions of images in a single a year. Such an application may also enable data science teams to iterate faster on tagging and validation work during model development cycle. For example, the same tagging application can be leveraged to get feedback on models in production and help refine the models over time.
Some embodiments may associate each user with a “quality score” related to the tags that have provided. For example, a majority of an individual's tags might need align with at least one other source of “truth.” Depending on the campaign, tags may be compared to an existing ground truth (e.g., the most accurate dataset available for a given attribute) or tags sources from others who participated in the campaign.
One way to improve the quality of tags is to combine tags from multiple users (e.g., a final tag is only assigned if 75% of all users agree that the tag is appropriate).
According to some embodiments, time information may be analyzed in connection with the document tagging process. For example,
Ingestion of image information into the image repository 1050 may include key assignment and ingestion of existing tags (e.g., latitude and longitude) that are associated with the images. Information from the image repository 1050 may then be processed to determine an appropriate domain assignment 1060 (e.g., using general image tag learning and artificial intelligence) and an automated document tagging platform 1070 (e.g., to crowdsource tags from users) to create a broad set of image tags to be stored in an image tag database 1090 (which can then be used to train AI models).
In this way, the system 1000 may collect image information in an efficient and accurate manner. Note that a system might attempt to automatically “mine” or assign tags to a document (and then compare those tags to user submitted tags and/or use the automatically assigned tags to create tag requests). For example,
At 1104, the received image input data is aggregated and mapped to create composite input image data. For example, the received image input data might be rearranged, converted into a standard format, fields may be mapped in accordance with a source identifier, common items within the images may be identified and/or extracted, etc.
At 1106, an event may be automatically detected in the set of image input data triggered by a rule and an associated tag. According to some embodiments, the tag may be associated with the triggering detection of an item, such as a building, an automobile, a street sign, etc. The triggering rule might be associated with, for example, multiple sub-items being detected within a single image (e.g., both an automobile and broken glass, ice on a staircase, etc.). According to some embodiments, the triggering rule was previously defined by an administrator using a graphical user interface or an AI model trained with crowdsource supplied tags. In some cases, one or more pre-determined conditions may be applied to flag the document tag (e.g., to reduce the number of tags to be eventually reported to risk applications). For example, a pre-determined condition may require that an item must be detected a pre-determined number of times or within a pre-determined proximity of another item.
An image tagging result database may be updated at 1108 by adding an entry to the database identifying the detected event (note that, if applicable, only flagged tags may result in a new entry being created in the database). The added entry might, for example, include an image identifier, an insert date, an image source, a rule identifier, and/or a line of business.
At 1110, an indication associated with the image tagging result database may be transmitted to a plurality of risk applications. The risk applications might be associated with, for example, a workers' compensation claim, a personal risk policy, a business risk policy, an automobile risk policy, a home risk policy, a sentiment analysis, risk tag detection, a cluster analysis, a predictive model, a subrogation analysis, fraud detection, a recovery factor analysis, large loss and volatile claim detection, a premium evasion analysis, a risk policy comparison, an underwriting decision, and/or indicator incidence rate trending application. Some embodiments might represent use cases for risk applications associated with various phases of an insurance process (e.g., a prospecting flow, a quoting flow, a pricing flow, a book management flow, a policy renewal flow, etc.). Note that the transmitted indication might be used to trigger a risk application (e.g., by triggering a fraud detection analysis) and/or update a risk application (e.g., by updating a variable or weighing factor of a predictive model). According to some embodiments, the system may then receive, from at least one of the risk applications, feedback information associated with the document tag. Based on the received feedback information, the system may automatically update at least one of the rule and/or the associated tag. For example, a rule or tag might be automatically updated to improve operation of the system when it is detected that users or underwriters are constantly correcting an image data evaluation in a particular way. That is, manual adjustments to and corrections of image processing results may be automatically used by the system to learn how to improve the rules and associated tags that are generated in future evaluations.
According to some embodiments, an “automated” image mining platform 1250 may access rules in the image rules database 1210 to mine the information from the insurance policy system 1220 and/or the other image input data sources 1230. As used herein, the term “automated” may refer to, for example, actions that can be performed with little or no human intervention.
The image mining platform 1250 may store information into and/or retrieve information from the image rules database 1210 and/or an image mining result database that is output to various external risk applications 1260 (e.g., software applications or devices associated with subrogation, fraud detection, and/or recovery factor analysis). The image rules database 1210 may be a locally stored relational database or reside remote from the image mining platform 1250. The term “relational” may refer to, for example, a collection of data items organized as a set of formally described tables from which data can be accessed. Moreover, a Relational Database Management System (“RDBMS”) may be used in connection with any of the database tables described herein. According to some embodiments, a graphical administrator interface 1270 may provide an ability to access and/or modify the image rules database 1210 via the image mining platform 1250. The administrator interface 1270 might, for example, let an administrator define terms, picture dictionaries, mapping rules, etc. associated with image mining and/or crowdsource tag collection. The data sources 1230, 1232 may be thought of as “publishers” of information to be consumed by the image mining platform 1250, and the risk applications 1260 may be considered “subscribers” to information created by the image mining platform 1250. Moreover, note that the image mining platform 1250 may operate asynchronously and/or independently of any risk application 1260.
Note that a mining platform may process other types of data in addition to image information. For example,
In this embodiment, the system 1300 further includes a text mining platform 1352 that also receives information from the tag rules database 1310, the insurance policy system 1320, the image input data sources 1330 (e.g., internal to a risk enterprise), the external third-party image data 1332 (e.g., weather forecast reports), and/or the external web site 1334. The text mining platform 1352 may store information into and/or retrieve information from the tag rules database 1310 and/or a text mining result database that is output to the various external risk applications 1360.
According to some embodiments, the text mining platform 1352 may use Natural Language Processing (“NLP”) to parse data streams into phrases and Named Entity Recognition (“NER”) rules may identify important concepts that are used to augment other structured data elements as predictor variables in models. The NER rules may be stored in an NER rule library and may include individual indicators. For example, indicators associated with a subrogation analysis might include the following words or phrases: animal bite, attorney, carrier, contractor, landlord, lawsuit, low subrogation, motor vehicle accident, no subrogation, off premises, responsible party, self-inflicted, third-party, and/or zero paid. As other examples, indicators associated with a fraud detection analysis might include the following words or phrases: disputed injury, no evidence, pre-existing condition, prior history, recent hire, terminated, unhappy, un-witnessed injury, claimant lacks documentation, claimant not employee, claimant paid in cash, no Social Security number, employer paid un-reported bill, employer won't confirm information, hazardous material, and/or excluded business. As still other examples, indicators associated with a recovery factor analysis might include: alcohol, complications, diabetes, high blood pressure, narcotics, pre-existing condition, obesity, osteoarthritis, smoking, substance abuse, and/or elderly care. Note that embodiments could further include audio mining platforms as well as other types of mining platforms.
The cloud analytics 1560 may generate output for an on-premises analytic platform 1570. Note that the on-premises analytic platform 1570 might also receive other information, including third-party data (e.g., from a weather service). The on-premises analytic platform 1570 may then process the received information and transmit results to data science, machine learning, and predictive analytics 1580 and/or business intelligence reports 1590 (e.g., hosted by an SQL Server Reporting Service (“SSRS”)).
The embodiments described herein may be implemented using any number of different hardware configurations. For example,
The processor 1610 also communicates with a storage device 1630. The storage device 1630 may comprise any appropriate information storage device, including combinations of magnetic storage devices (e.g., a hard disk drive), optical storage devices, mobile telephones, and/or semiconductor memory devices. The storage device 1630 stores a program 1612 and/or document tagging engine 1614 (e.g., associated with document tagging engine plug-in) for controlling the processor 1610. The processor 1610 performs instructions of the programs 1612, 1614, and thereby operates in accordance with any of the embodiments described herein. For example, the processor 1610 may receive an input document, via the communication device 1620, and associates it with a tag request. The tagging platform automatically selects at least one electronic record associated with a first user from a user database 1700 containing electronic records associated with users (each record including at least a user identifier and a user communication address). The input document and tag request are transmitted by the processor 1610 to the communication address associated with the first user, and a document tag is received from the first user. The processor 1610 may then store the document tag in a document mining result database 1800 by adding an entry to the database identifying the received document tag and transmit an indication associated with the document mining result database 1800 to a plurality of risk applications.
The programs 1612, 1614 may be stored in a compressed, uncompiled and/or encrypted format. The programs 1612, 1614 may furthermore include other program elements, such as an operating system, a database management system, and/or device drivers used by the processor 1610 to interface with peripheral devices.
As used herein, information may be “received” by or “transmitted” to, for example: (i) the image mining apparatus 1600 from another device; or (ii) a software application or module within the image mining apparatus 1600 from another software application, module, or any other source.
In some embodiments (such as shown in
Referring to
The user identifier 1702 may be, for example, a unique alphanumeric code identifying person who may help supply document tags. The communication address might indicate how that person should be contacted and the one or more document tags 1706 may represent descriptions that have been supplied by that user. For example, the presence of “STAIRS” and “ICE” in images associated with a claim file might have been flagged by the user. According to some embodiments, the user database 1700 may store a user quality score, multiple versions of a single user (e.g., a user might be an expert in some areas but not other areas), etc.
Referring to
The document mining result identifier 1802 may be, for example, a unique alphanumeric code identifying a result of a document tagging process (and might include the tags themselves or a link/point to where those tags are located). The loss description 1804 might categorize a cause associated with a tag and the date 1806 might indicate when the loss occurred. The user identifier 1808 might indicate which person resulted in the entry being created and may be based on, or associated with, the user identifier 1702 stored in the user database 1700. The claim identifier 1810 might indicate a claim file associated with the tag and/or an associated insurance policy. Note that other identifiers may be stored in the document tagging result database in addition to, or instead of, the claim identifier 1810. Examples of such other identifiers include a party identifier, a policy identifier, an entity identifier, a tax identifier, a physician identifier, a latitude and longitude, a postal address, etc.
An administrator interface may display various graphical user interfaces to an administrator. For example,
The pulled data may then be processed in accordance with any of the embodiments described herein (e.g., in connection with crowdsource tagging). In particular, images might be automatically processed at 2030 to determine the subject or content associated with the image (e.g., a particular image might be associated with an insurance claim). Related information may be assessed at 2032, image tags may be applied at 2034, and tag rules might be matched at 2036 with user-submitted tags (e.g., to determine that an image is associated with a building, an automobile, etc.). As a result of such processing, rule matches may be outputted and routed to an email server, workflow application, calendar application, etc. at 2038. For example, entries or records may be inserted into an image tagging result database 2040 (e.g., including fields such as an image identifier, date of insertion, an image source, etc.) for later use to train an AI model. Note that a single input file or record might result in multiple results being inserted into the image tagging result database 2040.
According to some embodiments, such a data flow 2000 may allow for the use of common domain image dictionaries (e.g., including building types, weather map patterns, facial recognition, etc.). Moreover, a composite image recognition rules library may provide for daily processing of image fields and rule changes may be tracked over time for analysis in addition to the crowdsource campaigns described herein. In addition, performance monitoring might be performed in connection with indicator incidence rate trending and new rules can be introduced with minimal code changes. According to some embodiments, a batch process may create a history for new and/or changed rules associated with the data flow 2000.
According to some embodiments, the image mining associated with the data flow is a “big data” activity that may use machine learning to sift through large amounts of unstructured data to find meaningful patterns to support business decisions. As used herein, the phrase “big data” may refer to massive amounts of data that are collected over time that may be difficult to analyze and handle using common database management tools. This type of big data may include web data, business transactions, email messages, activity logs, and/or machine-generated data. In addition, data from sensors, unstructured image posted on the Internet, such as blogs and social media, may be included in embodiments described herein.
According to some embodiments, the image mining and document tagging performed herein may be associated with hypothesis testing. For example, one or more theories may be provided (e.g., “the presence of snow on an outside staircase doubles the severity of an injury”). Knowledge engineering may then translate common smart tags for industry and scenario specific business context analysis.
In some embodiments, the image mining described herein may be associated with insight discovery wherein unsupervised data mining techniques may be used to discover common patterns in data. For example, highly recurrent themes may be classified, and other concepts may then be highlighted based on a sense of adjacency to these recurrent themes. In some cases, cluster analysis and drilldown tools may be used to explore the business context of such themes. For example, sentiment analysis may be used along with crowdsource tagging campaigns to determine how an entity is currently perceived and/or to detect that a particular automobile model is frequently experiencing a specific unintended problem.
According to some embodiments, an automated image mining platform 2150 may access rules in the event rules database 2180 to mine the received images. The image mining platform 2150 may then transmit results to external systems, such as an email alert server 2162, a workflow application 2164, and/or reporting and calendar functions 2166 (e.g., executing on a server). According to some embodiments, a graphical administrator interface 2170 may provide an ability to access and/or modify the event rules database 2180 and/or a crowdsource image tagging campaign. The administrator interface 2170 might, for example, let an administrator define image dictionaries, mapping rules, etc. associated with image mining and tagging.
The image mining platform 2150 may include a number of input nodes 2152 and/or output nodes 2154, such as nodes 2152, 2154 associated with protocols and/or Application Programming Interface (“API”) connections. Information provided via the output nodes 2154 may, for example, be used to augment structured data as independent variables in predictive models (e.g., a fraud detection process might to look for a set of red flags, a large loss/volatile claims process might look for comorbidity, biological, psychological, and/or social conditions, and a premium evasion process might look for understatement of workforce given an image of a building and misclassified business flags).
The information provided via the output nodes 2154 may also, for example, act as an tag detector to drive alerts to a business, to identify risk claims that merit re-scoring, to alert a business to a change in claim state for escalation or re-assignment, and/or to transmit alarms indicating the presence of a foreign manufacturer on a general liability claim. According to some embodiments, mined and/or tagged indicators from claims may be used to sample and/or compare risk policies (e.g., to compare policies based on the incidence rate of a particular type of roof damage on buildings).
The device 2400 presents a display 2410 that may be used to graphically tie together various crowdsource tags 2420 in association with an image (e.g., of an automobile as illustrated in
The following illustrates various additional embodiments of the invention. These do not constitute a definition of all possible embodiments, and those skilled in the art will understand that the present invention is applicable to many other embodiments. Further, although the following embodiments are briefly described for clarity, those skilled in the art will understand how to make any changes, if necessary, to the above-described apparatus and methods to accommodate these and other embodiments and applications.
Although specific hardware and data configurations have been described herein, note that any number of other configurations may be provided in accordance with embodiments of the present invention (e.g., some of the information associated with the databases described herein may be combined or stored in external systems).
Applicants have discovered that embodiments described herein may be particularly useful in connection with insurance policies and associated claims. For example,
Thus, embodiments may help enable golden record set creation and custom deep learning model training using a custom web hosted application that allows crowdsourcing of labeling tasks to employees (those who choose to volunteer or opt-in). This may enable scalable visual attribute (tag) identification for images and other documents. For insurance underwriting use cases, image tags to be deployed might relate to coverage, occupancy, protection, and exposure factors identified and prioritized by underwriting experts. Overall, embodiments may help an underwriting practice make better decisions, reduce repetitive or redundant workflow components, and bring automation to new and renewal low risk business, and identify complex cases that might require manual review. Embodiments may also reduce on-site assessments, help monitor changes in condition over time, provide complementary insights to traditional data sources, automate work where lower risk is identified (i.e., straight-through or “no touch” processing), moderate risk under hazardous conditions (flood, fire, roofs), etc.
Moreover, although some embodiments have been described with respect to particular image mining approaches, note that any of the embodiments might instead be associated with other image processing techniques. For example, image processing may operate to mine certain characteristic information from various social networks to determine whether a party is engaging in certain risky behavior or providing high risk products. It is also contemplated that embodiments may process images including text in one or more languages, such English, French, Arabic, Spanish, Chinese, German, Japanese and the like. In an exemplary embodiment, a system can be employed for sophisticated image analyses, wherein image can be recognized irrespective of the image source. Any relationships between the various images can be clarified by using a rules engine that determines a distance, field-of-view, angle, etc. of an item within the images.
According to some embodiments, image data may be used in conjunction with one or more predictive models to take into account a large number of underwriting and/or other parameters. The predictive model(s), in various implementation, may include one or more of neural networks, Bayesian networks (such as Hidden Markov models), expert systems, decision trees, collections of decision trees, support vector machines, or other systems known in the art for addressing problems with large numbers of variables. Preferably, the predictive model(s) are trained on prior image data and outcomes known to the risk company. The specific image data and outcomes analyzed may vary depending on the desired functionality of the particular predictive model. The particular image data parameters selected for analysis in the training process may be determined by using regression analysis and/or other statistical techniques known in the art for identifying relevant variables and associated weighting factors in multivariable systems. The parameters can be selected from any of the structured data parameters stored in the present system (e.g., image tags and tag data), whether the parameters were input into the system originally in a structured format or whether they were extracted from previously unstructured image, such as from big data.
In the present invention, the selection of weighting factors (either on a tag level or an image source level) may improve the predictive power of the image mining and tagging. For example, more reliable image sources may be associated with a higher weighting factor, while newer or less reliable sources might be associated with a relatively lower weighting factor.
The present invention has been described in terms of several embodiments solely for the purpose of illustration. Persons skilled in the art will recognize from this description that the invention is not limited to the embodiments described, but may be practiced with modifications and alterations limited only by the spirit and scope of the appended claims.
Claims
1. A document tagging system, comprising:
- a communication device to receive an input document;
- a user data store containing electronic records associated with users, each electronic record including at least a user identifier and a user communication address; and
- a tagging platform, coupled to the communication device and the user data store, including: a computer processor; and a computer storage device in communication with said computer processor and storing instructions adapted to be executed by said computer processor to: receive the input document, associate the input document with a tag request, automatically select at least one electronic record associated with a first user, transmit the input document and tag request to the communication address associated with the first user, receive a document tag from the first user, wherein the document tag is any object describing a document, store the document tag in a document mining result database by adding an entry to the document mining result database identifying the received document tag, and transmit an indication associated with the document mining result database to a plurality of risk applications associated with phases of an insurance process, the phases comprising at least a prospecting flow, a quoting flow, a pricing flow, a book management flow, and a policy renewal flow, wherein the risk applications are associated with an enterprise, the users comprise employees of the enterprise, and access to a computer system by the first user is dependent on and provided after receipt of the document tag from the user.
2. The system of claim 1, wherein the tagging platform sends the input document to multiple users and aggregates multiple document tags from those users.
3. The system of claim 1, wherein receipt of the document tag results in an automatic generation of a subsequent tag request in accordance with a tag hierarchy.
4. The system of claim 1, wherein the electronic record includes a user quality attribute indicative of a predicted document tag accuracy.
5. The system of claim 1, wherein the document mining result database further store at least one of: (i) an amount of time associated with the document tag, (ii) a time of day associated with the document tag, (iii) a day of week associated with the document tag, and (iv) a time of year associated with the document tag.
6. The system of claim 1, wherein execution of the stored instructions further causes the computer processor to:
- receive, from at least one of the risk applications, feedback information associated with the document tag, and
- based on the received feedback information, automatically update information in the user data store.
7. The system of claim 1, wherein the input document is associated with at least one of: (i) a risk claim file, (ii) a risk claim note, (iii) a medical report, (iv) a police report, (v) social network data, (vi) web image data, (vii) Internet of Things data, (viii) Global Positioning System (“GPS”) satellite data, (ix) activity tracking data, (x) big data information, (xi) a loss, (xii) an injury, (xiii) an image associated with a final notice of loss statement, (xiv) video chat stream, (xv) optical character recognition data, (xvi) third-party data, and (xvii) a governmental agency.
8. The system of claim 1, wherein the transmitted indication is to: (i) trigger a risk application, or (ii) update a risk application.
9. The system of claim 1, wherein the transmitted indication is associated with a variable or weighing factor of a predictive model.
10. The system of claim 1, wherein the input document comprises at least one of: (i) an image, (ii) a word processing document, (iii) a video, and (iv) an audio file.
11. The system of claim 1, wherein the tag is associated with at least one of: (i) an item present within an image, and (ii) an activity associated with an image.
12. The system of claim 1, wherein the entry added to the document mining result database includes at least one of: (i) an image identifier, (ii) an insert date, (iii) an image source, (iv) a rule identifier, and (v) a line of business.
13. The system of claim 1, wherein the computer storage device further stores instructions adapted to be executed by said computer processor to:
- identify at least a portion of the input document as being connected with one of training data and non-training data, and
- associate that portion of the input document with a custom tag based on one of the training data and non-training data.
14. The system of claim 1, wherein said automatic selection is performed via cloud analytics associated with at least one of: (i) object storage, (ii) a data catalog, (iii) a data lake store, (iv) a data factory, (v) machine learning, and (vi) artificial intelligence services.
15. A computer-implemented method, comprising:
- receiving an input document at a tagging platform via a communication device;
- associating the input document with a tag request;
- automatically selecting at least one electronic record associated with a first user from a user data store containing electronic records associated with users, each electronic record including at least a user identifier and a user communication address;
- transmitting the input document and tag request to the communication address associated with the first user;
- receiving a document tag from the first user wherein the document tag is any object describing a document;
- storing the document tag in a document mining result database by adding an entry to the document mining result database identifying the received document tag; and
- transmitting an indication associated with the document mining result database to a plurality of risk applications associated with phases of an insurance process, the phases comprising at least a prospecting flow, a quoting flow, a pricing flow, a book management flow, and a policy renewal flow, wherein the risk applications are associated with an enterprise, the users comprise employees of the enterprise, and access to a computer system by the first user is dependent on and provided after receipt of the document tag from the first user.
16. The method of claim 15, wherein the tagging platform sends the input document to multiple users and aggregates multiple document tags from those users.
17. The method of claim 15, wherein receipt of the document tag results in an automatic generation of a subsequent tag request in accordance with a tag hierarchy.
18. A non-transitory computer-readable medium storing instructions adapted to be executed by a computer processor to perform a method, said method comprising:
- receiving an input document at a tagging platform via a communication device;
- associating the input document with a tag request;
- automatically selecting at least one electronic record associated with a first user from a user data store containing electronic records associated with users, each electronic record including at least a user identifier and a user communication address;
- transmitting the input document and tag request to the communication address associated with the first user;
- receiving a document tag from the first user, wherein the document tag is any object describing a document;
- storing the document tag in a document mining result database by adding an entry to the document mining result database identifying the received document tag; and
- transmitting an indication associated with the document mining result database to a plurality of risk applications associated with phases of an insurance process, the phases comprising at least a prospecting flow, a quoting flow, a pricing flow, a book management flow, and a policy renewal flow, wherein the risk applications are associated with an enterprise, the users comprise employees of the enterprise, and access to a computer system by the first user is dependent on and provided after receipt of the document tag from the user.
19. The medium of claim 18, wherein the electronic record includes a user quality attribute indicative of a predicted document tag accuracy.
| 9128959 | September 8, 2015 | Barrington et al. |
| 9560092 | January 31, 2017 | Barrington et al. |
| 10176194 | January 8, 2019 | Har-Noy et al. |
| 20080059238 | March 6, 2008 | Park |
| 20150379047 | December 31, 2015 | Har-Noy et al. |
| 20160092959 | March 31, 2016 | Gross |
| 20170221156 | August 3, 2017 | Mingarelli et al. |
| 20190171986 | June 6, 2019 | Beznos et al. |
| 20200160968 | May 21, 2020 | Lyman |
Type: Grant
Filed: Mar 4, 2020
Date of Patent: Jun 13, 2023
Patent Publication Number: 20210279212
Assignee: HARTFORD FIRE INSURANCE COMPANY (Hartford, CT)
Inventors: Saira A. Kazmi (Glastonbury, CT), Arthur Paul Drennan, III (West Granby, CT), Steven J Cultrera (Newington, CT), Vasanth Madhavan Srinivasaraghavan (Manchester, CT), Justin G. Plock (Southport, CT)
Primary Examiner: Hau H Hoang
Application Number: 16/809,121
International Classification: G06F 16/93 (20190101); G06F 16/21 (20190101); G06F 16/23 (20190101); G06Q 40/08 (20120101); G06N 20/00 (20190101); G06Q 10/105 (20230101); G06N 5/04 (20230101);
