Abstract: A computer-implemented method according to one embodiment includes receiving, at a deep neural network (DNN), a plurality of sentences each having an associated label; training the DNN, utilizing the plurality of sentences and associated labels; and producing a linguistic expression (LE) utilizing the trained DNN.

Abstract: One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from both of the in-domain data and the out-of-domain data for evaluation by each of domain and sub-domain based on building a taxonomy of both domains and sub-domains for the AI service. The held-out data is excluded from training data used for training the AI service. The processor further determines distribution underlying performance metrics for the held-out data using bootstrap validation processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics.

Abstract: One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from both of the in-domain data and the out-of-domain data for evaluation by each of domain and sub-domain based on building a taxonomy of both domains and sub-domains for the AI service. The held-out data is excluded from training data used for training the AI service. The processor further determines distribution underlying performance metrics for the held-out data using bootstrap validation processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics.

Abstract: One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from the in-domain data and the out-of-domain data for evaluation by domain and sub-domain based on building a taxonomy of domains and sub-domains for the AI service. The processor further determines distribution underlying performance metrics for the held-out data using statistical processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics. The processor further provides confidence intervals based on the performance guarantees.

Abstract: Embodiments are provided for analysis of errors of a predictive model. In some embodiments, a system can include a processor that executes computer-executable components stored in memory. The computer-executable components can include an overview component that causes a client device to present first data identifying an error corresponding to a cell of a confusion matrix for a classification model, the error representing a mismatch between a first label generated by the classification model and a second label corresponding to a ground-truth observation. The computer-executable components also can include an element view component that receives second data defining a root cause of the error. The computer-executable components can further include an error annotation component that can embed the second data into a first data structure containing the first data, resulting in a first annotated data structure.

Abstract: Methods, systems, and computer program products for table recognition in PDF documents are provided herein. A computer-implemented method includes discretizing one or more contiguous areas of a PDF document; identifying one or more white-space separator lines within the one or more discretized contiguous areas of the PDF document; detecting one or more candidate table regions within the one or more discretized contiguous areas of the PDF document by clustering the one or more white-space separator lines into one or more grids; and outputting at least one of the candidate table regions as a finalized table in accordance with scores assigned to each of the one or more candidate table regions based on (i) border information and (ii) cell structure information.

Abstract: A computer-implemented method according to one embodiment includes receiving, at a deep neural network (DNN), a plurality of sentences each having an associated label; training the DNN, utilizing the plurality of sentences and associated labels; and producing a linguistic expression (LE) utilizing the trained DNN.

Abstract: One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from the in-domain data and the out-of-domain data for evaluation by domain and sub-domain based on building a taxonomy of domains and sub-domains for the AI service. The processor further determines distribution underlying performance metrics for the held-out data using statistical processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics. The processor further provides confidence intervals based on the performance guarantees.

Abstract: Methods, systems, and computer program products for table recognition in PDF documents are provided herein. A computer-implemented method includes discretizing one or more contiguous areas of a PDF document; identifying one or more white-space separator lines within the one or more discretized contiguous areas of the PDF document; detecting one or more candidate table regions within the one or more discretized contiguous areas of the PDF document by clustering the one or more white-space separator lines into one or more grids; and outputting at least one of the candidate table regions as a finalized table in accordance with scores assigned to each of the one or more candidate table regions based on (i) border information and (ii) cell structure information.

Abstract: Methods, systems, and computer program products for user-initiated dynamic data application programming interface creation are provided herein. A computer-implemented method includes automatically transforming a user-provided query into one or more parameterized queries, generating a data application programming interface based on at least one of the parameterized queries, outputting the generated data application programming interface to the user providing the user-provided query, and storing the generated data application programming interface in a database accessible to one or more additional users.

Abstract: One embodiment provides a method for developing a text analytics program for extracting at least one target concept including: utilizing at least one processor to execute computer code that performs the steps of: initiating a development tool that accepts user input to develop rules for extraction of features of the at least one target concept within a dataset comprising textual information; developing, using the rules for feature extraction, an evaluation dataset comprising at least one document annotated with the at least one target concept to be extracted by the text analytics program; creating, using the rules for feature extraction, a rule-based annotator to extract the at least one target concept; training, using the evaluation dataset, a machine-learning annotator to extract the at least one target concept within the dataset; combining the rule-based annotator and the machine learning annotator to form a combined annotator; evaluating, using the evaluation dataset, extraction performance of the combine

Abstract: Methods, systems, and computer program products for user-initiated dynamic data application programming interface creation are provided herein. A computer-implemented method includes automatically transforming a user-provided query into one or more parameterized queries, generating a data application programming interface based on at least one of the parameterized queries, outputting the generated data application programming interface to the user providing the user-provided query, and storing the generated data application programming interface in a database accessible to one or more additional users.

Abstract: Embodiments relate to facilitating construction of concepts from a task specification. A method includes receiving, from a user via a user interface, a task specification in natural language form. The method also includes parsing the task specification into a plurality of components, and searching a database for an existing concept having a pattern that approximates at least a portion of the plurality of components. The concept includes semantic meanings that are representable by textual patterns. The method further includes identifying any components of the plurality of components that are not included in the existing concept, and building a new concept that combines the existing concept and the components of the plurality of components that are not included in the existing concept.

Abstract: One embodiment provides a method for developing a text analytics program for extracting at least one target concept including: utilizing at least one processor to execute computer code that performs the steps of: initiating a development tool that accepts user input to develop rules for extraction of features of the at least one target concept within a dataset comprising textual information; developing, using the rules for feature extraction, an evaluation dataset comprising at least one document annotated with the at least one target concept to be extracted by the text analytics program; creating, using the rules for feature extraction, a rule-based annotator to extract the at least one target concept; training, using the evaluation dataset, a machine-learning annotator to extract the at least one target concept within the dataset; combining the rule-based annotator and the machine learning annotator to form a combined annotator; evaluating, using the evaluation dataset, extraction performance of the combine

Abstract: Described herein are methods, systems and computer program products for entity resolution. Entity resolution, also known as entity matching or record linkage, seeks to identify equivalent data objects between or among datasets. An example method includes creating a deterministic model by defining an entity to be resolved, selecting two datasets for comparison, defining matching predicates for attributes of the datasets to select a set of candidate matches, and defining a precedence rule for the candidate matches to select a subset of the candidate matches. The method includes running the deterministic model on the two datasets. Running the deterministic model includes applying the matching predicates and the precedence rule to data in the datasets that correspond to the attributes. The method also includes applying a cardinality rule to results of the running, and outputting the matching candidates for which the cardinality rule is satisfied.

Abstract: Embodiments of the present invention relate to a new method of entity integration using high-level scripting languages. In one embodiment, a method of and computer product for entity integration is provided. An entity declaration is read from a machine readable medium. The entity declaration describes an entity including at least one nested entity. An index declaration is read from a machine readable medium. The index declaration describes an index of nested entities. An entity population rule is read from a machine readable medium. The entity population rule describes a mapping from an input schema to an output schema. The output schema conforms to the entity declaration. A plurality of input records is read from a first data store. The input records conform to the input schema. The entity population rule applies to the plurality of records to create a plurality of output records complying with the output schema. An index of nested entities is populated. The index complies with the index declaration.

Abstract: Embodiments of the present invention relate to a new method of entity integration using high-level scripting languages. In one embodiment, a method of and computer product for entity integration is provided. An entity declaration is read from a machine readable medium. The entity declaration describes an entity including at least one nested entity. An index declaration is read from a machine readable medium. The index declaration describes an index of nested entities. An entity population rule is read from a machine readable medium. The entity population rule describes a mapping from an input schema to an output schema. The output schema conforms to the entity declaration. A plurality of input records is read from a first data store. The input records conform to the input schema. The entity population rule applies to the plurality of records to create a plurality of output records complying with the output schema. An index of nested entities is populated. The index complies with the index declaration.

Abstract: Embodiments of the present invention relate to a new method of entity integration using high-level scripting languages. In one embodiment, a method of and computer product for entity integration is provided. An entity declaration is read from a machine readable medium. The entity declaration describes an entity including at least one nested entity. An index declaration is read from a machine readable medium. The index declaration describes an index of nested entities. An entity population rule is read from a machine readable medium. The entity population rule describes a mapping from an input schema to an output schema. The output schema conforms to the entity declaration. A plurality of input records is read from a first data store. The input records conform to the input schema. The entity population rule applies to the plurality of records to create a plurality of output records complying with the output schema. An index of nested entities is populated. The index complies with the index declaration.

Abstract: Embodiments relate to entity resolution. One aspect includes creating a deterministic model by defining an entity to be resolved, selecting two datasets for comparison, defining matching predicates for attributes of the datasets to select a set of candidate matches, and defining a precedence rule for the candidate matches to select a subset of the candidate matches. An aspect further includes running the deterministic model on the two datasets. Running the deterministic model includes applying the matching predicates and the precedence rule to data in the datasets that correspond to the attributes. An aspect also includes applying a cardinality rule to results of the running, and outputting the matching candidates for which the cardinality rule is satisfied.

Abstract: Embodiments relate to facilitating construction of concepts from a task specification. A method includes receiving, from a user via a user interface, a task specification in natural language form. The method also includes parsing the task specification into a plurality of components, and searching a database for an existing concept having a pattern that approximates at least a portion of the plurality of components. The concept includes semantic meanings that are representable by textual patterns. The method further includes identifying any components of the plurality of components that are not included in the existing concept, and building a new concept that combines the existing concept and the components of the plurality of components that are not included in the existing concept.