Abstract: Systems and methods are configured to provide lifetime data valuations for a dataset that evolves across multiple machine learning training tasks by providing and updating path-dependent data valuations for data points in the dataset during each training task. A current machine learning training task may include splitting the dataset into multiple random mini-epochs and training the current machine learning model using a first random mini-epoch and an accuracy mini-epoch, which consists of high value data points from the path-dependent data valuations. The random and accuracy mini-epochs can be, during the training, iterated for a number of times during the training, while a second random mini-epoch is prefetch. During the training, the path-dependent data valuations can be updated based on data valuations during the current training and a similarity between the current machine learning model and prior trained machine learning models.

Abstract: Examples described herein relate to preparing datasets in a storage device for machine learning (ML) applications. Examples include maintaining ML facet mappings between ML facets and dataset preparation tags, deriving ML facets of a dataset stored in the storage device, and generating filtered datasets from the datasets using the ML facets and ML facet mappings. The filtered dataset is associated with improved dataset quality compared to unfiltered dataset. The storage device transmits the filtered dataset to ML applications requesting the dataset. Some examples include recommending, by the storage device, ML facets to the ML application based on performance metrics.

Abstract: Systems and methods provide a system that gathers information about data as it progresses through data processing pipelines of data analysis projects. The data analytics system derives value indicators and implicit metadata from the data processing pipelines. For example, the data analytics system may derive value indicators and implicit metadata from data-related products themselves, semantic analysis of the code/processing steps used to process the data-related products, the structure of data processing pipelines, and human behavior related to production and usage of data-related products. Once a new data analysis project is initiated, the data analytics system gathers parameters and characteristics about the new data analysis project and references the value indicators and implicit metadata to recommend useful processing steps, datasets, and/or other data-related products for the new data analysis project.

Abstract: Systems and methods are provide for automatically constructing data lineage representations for distributed data processing pipelines. These data lineage representations (which are constructed and stored in a central repository shared by the multiple data processing sites) can be used to among other things, clone the distributed data processing pipeline for quality assurance or debugging purposes. Examples of the presently disclosed technology are able to construct data lineage representations for distributed data processing pipelines by (1) generating a hash content value for universally identifying each data artifact of the distributed data processing pipeline across the multiple processing stages/processing sites of the distributed data processing pipeline; and (2) creating an data processing pipeline abstraction hierarchy for associating each data artifact to input and output events for given executions of given data processing stages (performed by the multiple data processing sites).

Abstract: Systems and methods provide a system that gathers information about data as it progresses through data processing pipelines of data analysis projects. The data analytics system derives value indicators and implicit metadata from the data processing pipelines. For example, the data analytics system may derive value indicators and implicit metadata from data-related products themselves, semantic analysis of the code/processing steps used to process the data-related products, the structure of data processing pipelines, and human behavior related to production and usage of data-related products. Once a new data analysis project is initiated, the data analytics system gathers parameters and characteristics about the new data analysis project and references the value indicators and implicit metadata to recommend useful processing steps, datasets, and/or other data-related products for the new data analysis project.

Abstract: Systems and methods provide a system that gathers information about data as it progresses through data processing pipelines of data analysis projects. The data analytics system derives value indicators and implicit metadata from the data processing pipelines. For example, the data analytics system may derive value indicators and implicit metadata from data-related products themselves, semantic analysis of the code/processing steps used to process the data-related products, the structure of data processing pipelines, and human behavior related to production and usage of data-related products. Once a new data analysis project is initiated, the data analytics system gathers parameters and characteristics about the new data analysis project and references the value indicators and implicit metadata to recommend useful processing steps, datasets, and/or other data-related products for the new data analysis project.

Abstract: Systems and methods provide a system that gathers information about data as it progresses through data processing pipelines of data analysis projects. The data analytics system derives value indicators and implicit metadata from the data processing pipelines. For example, the data analytics system may derive value indicators and implicit metadata from data-related products themselves, semantic analysis of the code/processing steps used to process the data-related products, the structure of data processing pipelines, and human behavior related to production and usage of data-related products. Once a new data analysis project is initiated, the data analytics system gathers parameters and characteristics about the new data analysis project and references the value indicators and implicit metadata to recommend useful processing steps, datasets, and/or other data-related products for the new data analysis project.

Abstract: Architectures and techniques for providing persistent volume functionality are disclosed. A storage container having a virtual storage volume to be persisted across multiple applications is created. The multiple applications hosted in one or more application containers. The storage container is placed within a virtual machine object. The virtual machine object containing the storage container is stored in a computer-readable memory as a persistent virtual storage volume.

Abstract: Systems and methods are configured to split an epoch associated with a training dataset into a plurality of mini-epochs. A machine learning model can be trained with a mini-epoch of the plurality of mini-epochs. The mini-epoch can be, during the training, iterated for a number of times during the training. One or more metrics reflective of at least one of: a training loss, training accuracy, or validation accuracy of the machine learning model associated with the mini-epoch can be received. Whether to terminate iterations of the mini-epoch early before a number of iterations of the mini-epoch reaches the number of times based on the one or more metrics can be determined. The number of iterations can be a non-zero number.

Abstract: Artificial-intelligence (AI)-based question-answer (QA) trace analysis of a text corpus that identifies answers to a natural language question and assesses the manner in which those answers evolve over time based on associated context is described herein. A set of QA trace records can be generated that includes a collection of answers derived from a text corpus in response to a posed natural language question along with contextual information relating to the answers. The set of QA trace records can be ordered based on corresponding date attributes gleaned from the contextual information to produce a time-series of QA trace records that can be processed by various types of downstream processing. Such downstream processing can include data visualization, pattern recognition, or the like for assessing how an answer to a natural language question evolves over time, identifying patterns/trends that develop over time with respect to the set of answers, and so forth.

Abstract: Systems and methods are provided for reusing machine learning models. For example, the applicability of prior models may be compared using one or more assessment values, including a similarity threshold and/or an accuracy threshold. The similarity threshold may identify a similarity of data between a first data set used to generate a first model and a new data set that is received by the system. When the similarity between these two data sets is exceeded, the system may reuse a model with the highest similarity value. When an accuracy value of the data set does not exceed an accuracy threshold, the system may initiate a retraining process to generate a second ML model associated with the second data.

Abstract: A source system generates snapshots of collected data. The snapshots have respective associated time references. Responsive to a request from a target system for data collected over a time interval, the source system generates a subset of the data collected by determining a start snapshot and an end snapshot. The start snapshot and the end snapshot are determined as a pair of snapshots that have respective associated time references that are most closely spaced and are inclusive of the time interval. The source system determines a difference in the data included in the end snapshot and the start snapshot and provides the subset of the data as the difference in the data included in the end snapshot and the start snapshot.

Abstract: Examples include bypassing a portion of an analytics workflow. In some examples, execution of an analytics workflow may be monitored upon receipt of a raw data and the execution may be interrupted at an optimal bypass stage to obtain insights data from the raw data. A similarity analysis may be performed to compare the insights data to a stored insights data in an insights data repository. Based, at least in part, on a determination of similarity, a bypass operation may be performed to bypass a remainder of the analytics workflow.

Abstract: Examples include bypassing a portion of an analytics workflow. In some examples, execution of an analytics workflow may be monitored upon receipt of a raw data and the execution may be interrupted at an optimal bypass stage to obtain insights data from the raw data. A similarity analysis may be performed to compare the insights data to a stored insights data in an insights data repository. Based, at least in part, on a determination of similarity, a bypass operation may be performed to bypass a remainder of the analytics workflow.

Abstract: Examples described herein relate to a computing system, a method and a non-transitory machine-readable medium for handling a request directed to a file in first filesystem having a filesystem instance being a content addressable storage objects. The computing system may also include a general-purpose second filesystem including its backing store within the filesystem instance of the first filesystem. Moreover, the computing system includes a first filesystem server to receive the request for an operation directed to the file in the first filesystem from an application. The first filesystem server may redirect the request to the second filesystem if the operation is a metadata operation; else redirect the request to the first filesystem.

Abstract: Architectures and techniques for providing persistent volume functionality are disclosed. A storage container having a virtual storage volume to be persisted across multiple applications is created. The multiple applications hosted in one or more application containers. The storage container is placed within a virtual machine object. The virtual machine object containing the storage container is stored in a computer-readable memory as a persistent virtual storage volume.

Abstract: An example implementation may relate to an apparatus that may identify data content of interest from data in buffers, and may store index entries representing the identified data content in a hierarchical index having different performance levels. The apparatus may include a priority manager that maintains an index scoreboard that tracks where index entries are to be stored among the different performance levels of the hierarchical index based on predetermined polices that prioritize data content of interest or functions that use data content of interest.

Abstract: Ternary content addressable memory (TCAM) structures and methods of use are disclosed. The memory architecture includes one or more ternary content addressable memory (TCAM) fields, and control logic that applies progressively discriminating data-masking and scores a closeness of a match based on matched and mismatched bits.

Abstract: Example techniques of data management in a network environment are described. In an example, a semantic pattern in a data stream transmitted from a source device to an edge device in the network environment is determined. The semantic pattern indicates relevance of data samples in the data stream for analysis of the data stream. The data stream is processed based on the semantic pattern, for storage and transmission in the network environment.

Abstract: Some examples relate to associating an insight with data. In an example, data may be received. A determination may be made that data type of the data is same as compared to an earlier data. An insight generated from the earlier data may be identified, wherein the insight may represent intermediate or resultant data generated upon processing of the earlier data by an analytics function, and wherein during generation metadata is associated with the insight. An analytics function used for generating the insight may be identified.