Abstract: A post-processing method, system, and computer program product for post-hoc improvement of instance-level and group-level prediction metrics, including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.

Abstract: One embodiment provides a method, including: receiving, from a user, a natural language query; generating an ontology subgraph by mapping the natural language query to a domain ontology and filtering the domain ontology based upon entities within the natural language query; producing a knowledge graph from (i) the generated ontology subgraph and (ii) a set of documents related to the natural language query, wherein the producing comprises extracting triples from the set of documents and aggregating the extracted triples to form the knowledge graph; and providing, to the user, a response to the received natural language query, wherein the returning the response comprises querying the produced knowledge graph using the natural language query.

Abstract: One embodiment provides a method, including: utilizing at least one processor to execute computer code that performs the steps of: receiving a partially completed query request from a user, wherein the partially completed query request comprises at least one text string provided in a query field; mapping the at least one text string to at least one ontology element contained within at least one of a plurality of partitions in an ontology graph, the graph comprising (i) ontology elements represented as nodes and (ii) connections between ontology elements represented as edges, wherein each of the plurality of partitions corresponds to a subject area of a field and comprises ontology elements corresponding to concepts related to the subject area of the field; and generating, in view of the plurality of partitions, at least one suggestion for completing the query request to the user, wherein the at least one suggestion comprises a semantically relevant suggestion corresponding to the at least one text string prov

Abstract: Methods, systems, and computer program products for carrying out interactive dialog in natural language using an ontology are provided herein. A computer-implemented method includes generating one or more domain-driven interpretations of a natural language dialogue query provided by a user via utilization of a web ontology language; determining multiple structured base queries, from among a stored collection of structured queries, that correspond to the natural language dialogue query, in view of the one or more generated domain-driven interpretations; selecting one of the multiple determined structured base queries, based on one or more items of context information pertaining to the dialogue; automatically generating a response to the selected structured base query; and outputting the generated response to the user.

Abstract: Systems and methods are provided for generating nested queries from natural language queries. In particular, system and methods are provided to implement natural language interfaces to databases (NLIDB) frameworks which are configured to apply intelligent reasoning over domain semantics to detect and generate nested queries across different domains without the need for domain specific training or utilizing domain-specific semantic templates for mapping a natural language query to a structured query.

Abstract: Methods, systems, and computer program products for automated generation of test cases for analyzing natural-language-interface-to-database systems are provided herein.

Abstract: Methods, systems and computer program products for automated test case generation are provided herein. A computer-implemented method includes selecting sample input data as a test case for a system under test, executing the test case on the system under test to obtain a result, and applying the result to a local explainer function to obtain at least a portion of a corresponding decision tree. The method further includes determining at least one path constraint from the decision tree, solving the path constraint to obtain a solution, and generating at least one other test case for the system under test based at least in part on the solution of the path constraint. The steps of the method are illustratively repeated in each of one or more additional iterations until at least one designated stopping criterion is met. The resulting test cases form a test suite for testing of a deep neural network (DNN) or other system.

Abstract: Methods, systems and computer program products for automated testing of dialog systems are provided herein. A computer-implemented method includes receiving selection of a conversation workspace of the automated dialog system and identifying test case inputs to the automated dialog system, the test case inputs comprising example user input for the given conversation workspace that has portions thereof modified and which the automated dialog system maps to a different intent and/or a different entity relative to the example user input. The method further includes generating human-interpretable explanations of mappings of portions of the test case inputs to the different intent and/or entity, generating suggestions for modifying intents, entities and dialog flows of the given conversation workspace such that the test case inputs map to the same intent and/or the same entity as their corresponding example user input, and outputting the suggestions and the human-interpretable explanations to a user.

Abstract: Methods, systems, and computer program products for user-initiated dynamic data application programming interface creation are provided herein. A computer-implemented method includes automatically transforming a user-provided query into one or more parameterized queries, generating a data application programming interface based on at least one of the parameterized queries, outputting the generated data application programming interface to the user providing the user-provided query, and storing the generated data application programming interface in a database accessible to one or more additional users.

Abstract: One embodiment provides a method, including: receiving, from a user, a natural language query; generating an ontology subgraph by mapping the natural language query to a domain ontology and filtering the domain ontology based upon entities within the natural language query; producing a knowledge graph from (i) the generated ontology subgraph and (ii) a set of documents related to the natural language query, wherein the producing comprises extracting triples from the set of documents and aggregating the extracted triples to form the knowledge graph; and providing, to the user, a response to the received natural language query, wherein the returning the response comprises querying the produced knowledge graph using the natural language query.

Abstract: One embodiment provides a method, including: receiving a natural language query from a user; identifying a plurality of interpretations for interpreting the natural language query, wherein the plurality of interpretations are based upon at least one ambiguity in the received natural language query; generating, for each of the plurality of interpretations, a plurality of example queries; generating, for each of the interpretations, both (i) an answer to the received natural language query and (ii) an answer to each of the generated plurality of example queries; and providing, to the user, (i) the generated answer for each interpretation of the natural language query and (ii) a plurality of question/answer pairs for each of the identified plurality of interpretations that assists in disambiguating the ambiguity, wherein each question/answer pair comprises at least one of the generated plurality of example queries and the corresponding generated answer to the example query.

Abstract: Methods, systems, and computer program products for bootstrapping of state-based dialog systems are provided herein. A computer-implemented method includes determining parameters for state automata by partitioning an ontology graph into sub-graphs and a knowledge graph into sub-graphs, wherein the ontology graph and the knowledge graph are based on a user question and domain knowledge pertaining to the user question; generating a structured query for each of the sub-graphs; determining intentions of a dialog pertaining to the at least one user question by translating each of the generated structured queries to a respective natural language query; creating one or more dialog states for each of the determined dialog intentions; creating one or more connecting dialog states between pairs of the created dialog states; and generating an automata dialog framework associated with the user question based on the created dialog states and the created connecting dialog states.

Abstract: Aspects of the present disclosure relate to automated ontology refinement based on query inputs and provided feedback. A query input is received for an ontology. Features of the query input are analyzed, wherein analyzation includes determining syntactical and semantic characteristics of the features of the query input. Based on the determined syntactical and semantic characteristics, ontological elements are classified for each feature of the query input. The ontological element for each feature of the query input is then compared to a set of ontological elements of the ontology. Based on the comparison, a response to the query input is received, along with a request for feedback regarding the response. Feedback is then received regarding the response. Based on the feedback, the ontology is analyzed to determine at least one deficiency of the ontology. The ontology is then refined to correct the at least one deficiency.

Abstract: Methods, systems, and computer program products for user-initiated dynamic data application programming interface creation are provided herein. A computer-implemented method includes automatically transforming a user-provided query into one or more parameterized queries, generating a data application programming interface based on at least one of the parameterized queries, outputting the generated data application programming interface to the user providing the user-provided query, and storing the generated data application programming interface in a database accessible to one or more additional users.

Abstract: The present invention may include a computing device receives a query and determines a plurality of implementations based on the query and searching the ontology driven system. The computing device may display the determined plurality of implementations and monitors the eye tracking datum for a fixation. The computing device may determine the corresponding plurality of implementations associated with a region where the fixation is directed based on determining the fixation. The computing device may determine a plurality of candidate elements of the plurality of implementations in the region based on previous determination of the corresponding plurality of implementations. The computing device may generate a plurality of preferred interpretations based on monitoring the eye tracking datum, the corresponding plurality of implementations associated with the region of the fixation, and the plurality of candidate elements.

Abstract: Methods, systems, and computer program products for translating structured languages to natural language using domain-specific ontology are provided herein. A computer-implemented method includes determining similarities among multiple natural language query interpretations derived from an input query, determining differences among the multiple natural language query interpretations, and generating natural language descriptions of each of the multiple natural language query interpretations based on analysis of the determined similarities, the determined differences, and the input query. The method also includes producing, for each of the natural language query interpretations, a natural language string that represents one or more unambiguous interpretations of the input query, wherein the producing comprises consolidating the generated natural language descriptions. Further, the method includes outputting each of the produced natural language strings to a user.

Abstract: Methods and arrangements for providing classification for a business process hierarchy. Packaged application objects are received, comprising at least one of the standard application objects and custom objects. At least one of the standard application objects is mapped to at least one node of a business process hierarchy, thereby forming at least one mapping template. At least one of the standard application objects is mapped to at least one of the custom objects to create a first mapping, and at least one of the custom objects is mapped to at least one of the standard application objects to create a second mapping. The at least one mapping template, the first mapping and the second mapping are combined, to create a mapping of at least one of the custom objects to the business process hierarchy. Other variants and embodiments are broadly contemplated herein.

Abstract: Methods, systems, and computer program products for carrying out interactive dialog in natural language using an ontology are provided herein. A computer-implemented method includes generating one or more domain-driven interpretations of a natural language dialogue query provided by a user via utilization of a web ontology language; determining multiple structured base queries, from among a stored collection of structured queries, that correspond to the natural language dialogue query, in view of the one or more generated domain-driven interpretations; selecting one of the multiple determined structured base queries, based on one or more items of context information pertaining to the dialogue; automatically generating a response to the selected structured base query; and outputting the generated response to the user.

Abstract: Methods, systems, and computer program products for translating structured languages to natural language using domain-specific ontology are provided herein. A computer-implemented method includes determining similarities among multiple natural language query interpretations derived from an input query, determining differences among the multiple natural language query interpretations, and generating natural language descriptions of each of the multiple natural language query interpretations based on analysis of the determined similarities, the determined differences, and the input query. The method also includes producing, for each of the natural language query interpretations, a natural language string that represents one or more unambiguous interpretations of the input query, wherein the producing comprises consolidating the generated natural language descriptions. Further, the method includes outputting each of the produced natural language strings to a user.

Abstract: One embodiment provides a method, including: utilizing at least one processor to execute computer code that performs the steps of: receiving a partially completed query request from a user, wherein the partially completed query request comprises at least one text string provided in a query field; mapping the at least one text string to at least one ontology element contained within at least one of a plurality of partitions in an ontology graph, the graph comprising (i) ontology elements represented as nodes and (ii) connections between ontology elements represented as edges, wherein each of the plurality of partitions corresponds to a subject area of a field and comprises ontology elements corresponding to concepts related to the subject area of the field; and generating, in view of the plurality of partitions, at least one suggestion for completing the query request to the user, wherein the at least one suggestion comprises a semantically relevant suggestion corresponding to the at least one text string prov