Abstract: In an approach to improve the extracting test datasets for testing and resource optimization, embodiments execute a test run on a full dataset of a job, and identify existing bottlenecks in the job through a run-time monitoring interface. Additionally, embodiments execute a run-time metrics analysis, a source code analysis, and a source data impact analysis of a distributed data processing engine executing a distributed data processing job, and generate, by an analysis and impact scoring engine, an impact scoring table of job transformations based on the source code analysis. Furthermore, embodiments generate data extraction rules based on the impact scoring table, and extract a test dataset based on the data extraction rules. Moreover, embodiments evaluate the data extraction rules against user defined thresholds, and prepare a representative test dataset, and output, through a user interface on a computing device, the representative test dataset to a user.

Abstract: In an approach to improve the extracting test datasets for testing and resource optimization, embodiments execute a test run on a full dataset of a job, and identify existing bottlenecks in the job through a run-time monitoring interface. Additionally, embodiments execute a run-time metrics analysis, a source code analysis, and a source data impact analysis of a distributed data processing engine executing a distributed data processing job, and generate, by an analysis and impact scoring engine, an impact scoring table of job transformations based on the source code analysis. Furthermore, embodiments generate data extraction rules based on the impact scoring table, and extract a test dataset based on the data extraction rules. Moreover, embodiments evaluate the data extraction rules against user defined thresholds, and prepare a representative test dataset, and output, through a user interface on a computing device, the representative test dataset to a user.

Abstract: Provided are techniques for automatically enhancing content items through identifying relationships. One or more content items and an associated transcript for each of the one or more content items are received. One or more terms are identified in the one or more content items. A content tree is built for each of the one or more content items using the one or more terms and corresponding contexts. A content graph is built using each content tree. One or more subgraphs of the content graph are identified. One or more paths through each of the one or more subgraphs are identified and one or more content expansion recommendations are provided. In response to selection of a content item, the one or more paths are used to provide an enhanced content item with one or more prerequisite content items and one or more subsequent content items.

Abstract: Embodiments for determining processing performed on a data element are provided. A file system call associated with a data element stored in a storage system is detected. The file system call is analyzed. Data lineage for the data element is determined based on the analyzing of the file system call.

Abstract: In an approach to improve the extracting test datasets for testing and resource optimization, embodiments execute a test run on a full dataset of a job, and identify existing bottlenecks in the job through a run-time monitoring interface. Additionally, embodiments execute a run-time metrics analysis, a source code analysis, and a source data impact analysis of a distributed data processing engine executing a distributed data processing job, and generate, by an analysis and impact scoring engine, an impact scoring table of job transformations based on the source code analysis. Furthermore, embodiments generate data extraction rules based on the impact scoring table, and extract a test dataset based on the data extraction rules. Moreover, embodiments evaluate the data extraction rules against user defined thresholds, and prepare a representative test dataset, and output, through a user interface on a computing device, the representative test dataset to a user.

Abstract: Embodiments for determining processing performed on a data element are provided. A file system call associated with a data element stored in a storage system is detected. The file system call is analyzed. Data lineage for the data element is determined based on the analyzing of the file system call.