Abstract: A data quality assessment and monitoring tool addresses inconsistency in large data sets from differing sources, determining data quality attributes such as completeness, conformity, validity, and accuracy. Flexible taxonomies and rollup strategies accommodate diverse business unit needs across a complex enterprise, and provides insight into individual entities’ performance.

Abstract: A data quality assessment and monitoring tool addresses inconsistency in large data sets from differing sources, determining data quality attributes such as completeness, conformity, validity, and accuracy. Flexible taxonomies and rollup strategies accommodate diverse business unit needs across a complex enterprise, and provides insight into individual entities' performance. An exemplary tool comprises a data importer for importing data from a data lake; a rules manager for generating rules and rule sets; a scoring engine for generating data quality scores; a job manager; a data profiler for running data assessment tasks and collating the data quality scores for a plurality of hierarchical data entity units; a hierarchical scoring aggregator for aggregating sets of data quality scores into a plurality of first tier aggregate data quality scores and to further aggregate the first tier aggregate data quality scores into one or more second tier aggregate data quality scores; and a reporting component.

Abstract: Systems, methods, and computer-readable storage media for monitoring a pipeline of data without modifying code or otherwise impeding the pipeline. This is accomplished by identifying a class of software functions which modify other software function upon execution without modifying code for the other software functions, monitoring a pipeline of data to identify a plurality of jobs, the each job in the plurality of jobs comprising header information and body information, initiating, for each job in the plurality of jobs, a plurality of processes, modifying the plurality of processes based on the class, to yield advice modified processes, and outputting a status of the advice modified processes without modifying the pipeline.

Abstract: A data quality assessment and monitoring tool addresses inconsistency in large data sets from differing sources, determining data quality attributes such as completeness, conformity, validity, and accuracy. Flexible taxonomies and rollup strategies accommodate diverse business unit needs across a complex enterprise, and provides insight into individual entities' performance. An exemplary tool comprises a data importer for importing data from a data lake; a rules manager for generating rules and rule sets; a scoring engine for generating data quality scores; a job manager; a data profiler for running data assessment tasks and collating the data quality scores for a plurality of hierarchical data entity units; a hierarchical scoring aggregator for aggregating sets of data quality scores into a plurality of first tier aggregate data quality scores and to further aggregate the first tier aggregate data quality scores into one or more second tier aggregate data quality scores; and a reporting component.

Abstract: Systems, methods, and computer-readable storage media for monitoring a pipeline of data without modifying code or otherwise impeding the pipeline. This is accomplished by identifying a class of software functions which modify other software function upon execution without modifying code for the other software functions, monitoring a pipeline of data to identify a plurality of jobs, the each job in the plurality of jobs comprising header information and body information, initiating, for each job in the plurality of jobs, a plurality of processes, modifying the plurality of processes based on the class, to yield advice modified processes, and outputting a status of the advice modified processes without modifying the pipeline.