Abstract: A computer-implemented method according to one embodiment includes identifying a data set and meta information; and augmenting the data set with additional features in response to an automatic analysis of the data set in view of the meta information.

Abstract: A dataset including features and values associated with the features can be received. Each of the features in the dataset can be mapped to a corresponding node in a knowledge graph based on the concept represented by the corresponding node. The knowledge graph can be traversed to find a candidate node connected to at least one mapped node, the candidate node not being mapped to a feature in the dataset. A concept associated with the candidate node can be identified as a new feature. A machine learning model pipeline can use the features in the dataset and the new feature to select a subset of features for training a machine learning model.

Abstract: A computer automatically selects a machine learning model pipeline using a meta-learning machine learning model. The computer receives ground truth data and pipeline preference metadata. The computer determines a group of pipelines appropriate for the ground truth data, and each of the pipelines includes an algorithm. The pipelines may include data preprocessing routines. The computer generates hyperparameter sets for the pipelines. The computer applies preprocessing routines to ground truth data to generate a group of preprocessed sets of said ground truth data and ranks hyperparameter set performance for each pipeline to establish a preferred set of hyperparameters for each of pipeline. The computer selects favored data features and applies each of the pipelines, with associated sets of preferred hyperparameters, to score the favored data features of the preprocessed ground truth data. The computer ranks pipeline performance and selects a candidate pipeline according to the ranking.

Abstract: A black box optimization method, system, and computer program product include implementing an average gradient estimator using a forward difference of function values at multiple random directions, performing variance reduction via gradient blending with an output of the average gradient estimator using a control variate, and performing binary quantization of a result of the variance reduction.

Abstract: A method for speech recognition, the method includes: extracting time-frequency speech features from a series of reference speech elements in a first series of sampling windows; aligning reference speech elements that are not of equal time span duration; constructing a common subspace for the aligned speech features; determining a first set of coefficient vectors; extracting a time-frequency feature image from a test speech stream spanned by a second sampling window; approximating the extracted image in the common subspace for the aligned extracted time-frequency speech features with a second coefficient vector; computing a similarity measure between the first and the second coefficient vector; determining if the similarity measure is below a predefined threshold; and wherein a match between the reference speech elements and a portion of the test speech stream is made in response to a similarity measure below a predefined threshold.

Abstract: Performing adaptive cyber-security analytics including a computer implemented method that includes receiving a report on a network activity. A score responsive to the network activity and to a scoring model is computed at a computer. The score indicates a likelihood of a security violation. The score is validated and the scoring model is automatically updated responsive to results of the validating. The network activity is reported as suspicious in response to the score being within a threshold of a security violation value.

Abstract: A system and method for determining data usage based on provenance information, in a stream-processing system, includes progressively setting usage information for output stream data objects (SDOs), determining input SDOs that an output SDO depends on, based on a provenance dependency function; recursively feeding back the usage information for a subset of SDOs that can be discarded; and discarding the subset of SDOs. A system and method for data retention based on usage information, in a stream-processing system, includes managing retention of SDOs by deleting SDOs that are determined to be of null usage; and enhancing retention characteristics of SDOs that are deemed to have usage.

Abstract: A computer-implemented method for configuring a classifier structure includes receiving a classifier structure for identifying a feature of input data, adding a plurality of operating points to a parent binary classifier of the classifier structure, and setting the operating points to minimize a total cost based on resources available to the classifier structure. Each operating point controls how data is classified by the parent binary classifier for distribution to the child binary classifiers.

Abstract: A technique for identifying a defect in an object produced by a controllable process is disclosed. A first type of data generated as a result of production of the object by the controllable process is obtained. A second type of data generated as a result of production of the object by the controllable process is obtained. The first type of data and the second type of data are jointly analyzed. A defect is identified in the object based on the joint analysis of the first type of data and the second type of data.

Abstract: A keyed delay is used in the processing of streaming data to decrease the processing performed and the output provided. A first event, within a particular window, having a particular key starts a delay condition. Arriving events with the same key replace the previous arrival for that key until the delay condition is satisfied. In response thereto, the latest event with that key is output.

Abstract: Techniques for dynamically modifying inter-connections between components in an application are provided. The techniques include receiving a data producer profile for each output port within a software application to be executed on one or more processors, receiving a data subscription profile for each input port of each component of the application, establishing connections between the output ports and the input ports of the components in the application based on a comparison of each data producer profile and each data subscription profile, executing the application on one or more processors to process streams of data, receiving either or both of a new data producer profile or a new data subscription profile during the execution of the application, and establishing at least one new connection between an output port and an input port based upon a revised comparison of the received data profiles that include the new data profile.

Abstract: A technique for identifying a defect in an object produced by a controllable process. A first type of data generated as a result of production of the object by the controllable process is obtained. A second type of data generated as a result of production of the object by the controllable process is obtained. The first type of data and the second type of data are jointly analyzed. A defect is identified in the object based on the joint analysis of the first type of data and the second type of data. By way of example, the controllable process comprises a semiconductor manufacturing process such as a silicon wafer manufacturing process and the object produced by the semiconductor manufacturing process comprises a processed wafer. The first type of data comprises tool trace data and the second type of data comprises wafer image data. The tool trace data is generated by a photolithographic tool.

Abstract: A technique for analyzing two or more data streams respectively generated from two or more components of a controllable process includes the following steps. In a first step, a statistical analysis is performed on each of the two or more data streams to generate first analysis results in the form of respective statistical results for the two or more data streams. In a second step, at least a portion of the statistical results from at least one of the two or more data streams is combined with at least a portion of the statistical results from at least another one of the two or more data streams to yield second analysis results. The controllable process is adjustable based on at least one of the first analysis results and the second analysis results.

Abstract: Systems and methods for adaptive signal sampling and sample quantization for resource-constrained stream processing. Exemplary embodiments include a signal sampling and signal quantization method for a data stream from a ECG sensor into a pervasive device, the method including adaptively sampling the data stream, compressing data samples from the data stream, reducing the number of samples via quantization, defining a utility function for signal reconstruction, and formulating optimization on the reconstructed signal that is jointly sampled and quantized.

Abstract: Performing adaptive cyber-security analytics including a computer implemented method that includes receiving a report on a network activity. A score responsive to the network activity and to a scoring model is computed at a computer. The score indicates a likelihood of a security violation. The score is validated and the scoring model is automatically updated responsive to results of the validating. The network activity is reported as suspicious in response to the score being within a threshold of a security violation value.

Abstract: A system and method for dynamically building applications for stream processing includes providing processing elements with a flow specification describing each input and a stream description describing each output such that the flow specification indicates a stream or streams which are to be received based on processing information and the stream descriptions indicate the processing information. Processing elements that can be reused are identified by determining equivalence between the processing elements. Processing elements that are new and are not reusable are instantiated in a flow graph. An application is dynamically composed, using the instantiated processing elements by routing available streams to the instantiated processing elements in accordance with the flow specifications.

Abstract: A publisher-subscriber system includes a broker configured to receive and distribute at least one data stream from publishers to subscribers in accordance with subscriptions. An annotator/classifier is configured to annotate or classify the data stream by employing one or more additional stream-level attributes to create an annotated data stream wherein the annotated data stream applies the stream-level attribute to all messages therein. Subscribers can subscribe to the annotated data stream created by the annotator/classifier.

Abstract: A method is provided for generating a resource function estimate of resource usage by an instance of a processing element configured to consume zero or more input data streams in a stream processing system having a set of available resources that comprises receiving at least one specified performance metric for the zero or more input data streams and a processing power of the set of available resources, wherein one specified performance metric is stream rate; generating a multi-part signature of executable-specific information for the processing element and a multi-part signature of context-specific information for the instance; accessing a database of resource functions to identify a static resource function corresponding to the executable-specific information and a context-dependent resource function corresponding to the context-specific information; combining the static resource function and the context-dependent resource function to form a composite resource function for the instance; and applying the res

Abstract: A keyed delay is used in the processing of streaming data to decrease the processing performed and the output provided. A first event, within a particular window, having a particular key starts a delay condition. Arriving events with the same key replace the previous arrival for that key until the delay condition is satisfied. In response thereto, the latest event with that key is output.

Abstract: A system and method for determining data usage based on provenance information, in a stream-processing system, includes progressively setting usage information for output stream data objects (SDOs), determining input SDOs that an output SDO depends on, based on a provenance dependency function; recursively feeding back the usage information for a subset of SDOs that can be discarded; and discarding the subset of SDOs. A system and method for data retention based on usage information, in a stream-processing system, includes managing retention of SDOs by deleting SDOs that are determined to be of null usage; and enhancing retention characteristics of SDOs that are deemed to have usage.