Abstract: Embodiments maintain a data pool that includes heterogeneous data sets, and receiving a first data batch of a data set from a data source into the data pool. Embodiments determine a current state of the data set based on a data set state diagram including a plurality of data set states, and identify a condition of the first data batch. Embodiments further set a data batch state for the first data batch, based on a data batch state diagram, and update the data batch state of a prior data batch received before the first data batch, based on the condition of the first data batch. Embodiments additionally transition the data set state diagram, based on the condition of the first data batch, to an updated data set state. Embodiments maintain a data state repository storing the data set state for each of the plurality of heterogeneous data sets.

Abstract: Embodiments of the system in include a memory that stores a metamodel including a plurality of predefined characteristics for data sets. A data repository stores a plurality of heterogeneous data sets, each of the plurality of data sets comprising a plurality of data batches received over time. An interface receives a new data set for storage into the data repository, and data health reasoner to retrieve the stored metamodel from the memory, the stored metamodel including a plurality of predefined characteristics. The data health reasoner determines measured values of a subset of the plurality of predefined characteristics identified based on the stored metamodel, and determines a set of data health metrics for the data set based on the measured values of the subset of the set of the predefined characteristics. The data health reasoner formulates a plurality of data validation assertions for the data set and apply the plurality of data validation assertions to each instance of the data set.

Abstract: Embodiments maintain a data pool that includes heterogeneous data sets, and receiving a first data batch of a data set from a data source into the data pool. Embodiments determine a current state of the data set based on a data set state diagram including a plurality of data set states, and identify a condition of the first data batch. Embodiments further set a data batch state for the first data batch, based on a data batch state diagram, and update the data batch state of a prior data batch received before the first data batch, based on the condition of the first data batch. Embodiments additionally transition the data set state diagram, based on the condition of the first data batch, to an updated data set state. Embodiments maintain a data state repository storing the data set state for each of the plurality of heterogeneous data sets.

Abstract: Embodiments of the system in include a memory that stores a metamodel including a plurality of predefined characteristics for data sets. A data repository stores a plurality of heterogeneous data sets, each of the plurality of data sets comprising a plurality of data batches received over time. An interface receives a new data set for storage into the data repository, and data health reasoner to retrieve the stored metamodel from the memory, the stored metamodel including a plurality of predefined characteristics. The data health reasoner determines measured values of a subset of the plurality of predefined characteristics identified based on the stored metamodel, and determines a set of data health metrics for the data set based on the measured values of the subset of the set of the predefined characteristics. The data health reasoner formulates a plurality of data validation assertions for the data set and apply the plurality of data validation assertions to each instance of the data set.

Abstract: Computer-implemented techniques for constructing structured database query language statements from natural language questions are disclosed. In some embodiments, a knowledge graph-based approach is employed to construct a structured database query language statement such as, for example, a Structure Query Language (SQL) statement, from a natural language question. The approach encompasses a domain knowledge graph model and a database schema wiring model. The domain knowledge graph model represents domain-specific knowledge about the underlying database data in the form of a graph having nodes and directed edges between the nodes. The database schema wiring model maps routes in the domain knowledge graph to database schema elements. The techniques allow a natural language interface system to accurately translate domain-specific natural language questions posed by users to structured database query language statements that can be executed against a structured database to answer the natural language question.

Abstract: Computer-implemented techniques for constructing structured database query language statements from natural language questions are disclosed. In some embodiments, a knowledge graph-based approach is employed to construct a structured database query language statement such as, for example, a Structure Query Language (SQL) statement, from a natural language question. The approach encompasses a domain knowledge graph model and a database schema wiring model. The domain knowledge graph model represents domain-specific knowledge about the underlying database data in the form of a graph having nodes and directed edges between the nodes. The database schema wiring model maps routes in the domain knowledge graph to database schema elements. The techniques allow a natural language interface system to accurately translate domain-specific natural language questions posed by users to structured database query language statements that can be executed against a structured database to answer the natural language question.