Abstract: Embodiments of the present invention are directed to facilitating data preprocessing for machine learning. In accordance with aspects of the present disclosure, a training set of data is accessed. A preprocessing query specifying a set of preprocessing parameter values that indicate a manner in which to preprocess the training set of data is received. Based on the preprocessing query, a preprocessing operation is performed to preprocess the training set of data in accordance with the set of preprocessing parameter values to obtain a set of preprocessed data. The set of preprocessed data can be provided for presentation as a preview. Based on an acceptance of the set of preprocessed data, the set of preprocessed data is used to train a machine learning model that can be subsequently used to predict data.

Abstract: Implementations include receiving a user provided example value of personally identifiable information (PII). Occurrences of the received example value are automatically identified in a dataset of events, wherein each occurrence is identified in a portion of raw machine data of a respective event of the events. For each occurrence of the identified occurrences, an extraction rule is generated, which defines a pattern of the occurrence of the example value and is executable to identify PII values in portions of raw machine data of the events using the pattern. Values of the PII are identified in a set of events using a set of extraction rules comprising the extraction rule of a plurality of the occurrences.

Abstract: Operational machine components of an information technology (IT) or other microprocessor- or microcontroller-permeated environment generate disparate forms of machine data. Network connections are established between these components and processors of an automatic data intake and query system (DIQS). The DIQS conducts network transactions on a periodic and/or continuous basis with the machine components to receive the disparate data and ingest certain of the data as measurement entries of a DIQS metrics datastore that is searchable for DIQS query processing. The DIQS may receive search queries to process against the received and ingested data via an exposed network interface. In one example embodiment, a query building component conducts a user interface using a network attached client device. The query building component may elicit search criteria via the user interface using a natural language interface, construct a proper query therefrom, and present new information based on results returned from the DIQS.

Abstract: In various embodiments, a natural language (NL) application implements functionality that enables users to more effectively access various data storage systems based on NL requests. As described, the operations of the NL application are guided by, at least in part, on one or more templates and/or machine-learning models. Advantageously, the templates and/or machine-learning models provide a flexible framework that may be readily tailored to reduce the amount of time and user effort associated with processing NL requests and to increase the overall accuracy of NL application implementations.

Abstract: A non-transitory computer readable storage medium has instructions executed by a processor to receive from a network connection different sources of unstructured data, where the unstructured data has multiple modes of semantically distinct data types and the unstructured data has time-varying data instances aggregated over time. An entity combining different sources of the unstructured data is formed. A representation for the entity is created, where the representation includes embeddings that are numeric vectors computed using machine learning embedding models. These operations are repeated to form an aggregation of multimodal, time-varying entities and a corresponding index of individual entities and corresponding embeddings. Proximity searches are performed on embeddings within the index.

Abstract: As described herein, a portion of machine data of a message may be analyzed to infer, using an inference model, a sourcetype of the message. The portion of machine data may be generated by one or more components in an information technology environment. Based on the inference, a set of extraction rules associated with the sourcetype may be selected. Each extraction rule may define criteria for identifying a sub-portion of text from the portion of machine data of the message to produce a value. The set of extraction rules may be applied to the portion of machine data of the message to produce a result set that indicates a number of values identified using the set of extraction rules. Based on the result set, at least one action may be performed on one or more of inference data associated with the inference model and one or more messages.

Abstract: Embodiments of the present invention are directed to identifying related data, in particular, data associated with different source types. In embodiments, a first source type related to a second source type associated with a search query is identified. Field set pairs are identified from a first data set associated with the first source type and a second data set associated with the second source type. Each field set pair can include one field set associated with the first source type and another field set associated with the second source type. For each field set pair, an extent of similarity is determined between the corresponding field sets. Based on the extent of similarities between the corresponding field sets, at least one pair of related field sets is identified. An indication of the at least one pair of related field sets is provided, for example, for presentation to a user.

Abstract: Discovery of communication platform features or exposure of such features to the user may include generating embeddings for a variety of types of communication platform content and communications. These embeddings may be used to characterize and compare various communication platform features and ultimately expose these features to a user when the user may not have otherwise encountered them. The embeddings may additionally or alternatively be used to determine a degree of alignment.

Abstract: Described herein are technologies that facilitate effective use (e.g., indexing and searching) of non-text machine data (e.g., audio/visual data) in an event-based machine-data intake and query system.

Abstract: A non-transitory computer readable storage medium has instructions executed by a processor to receive from a network connection different sources of unstructured data. An entity is formed by combining one or more sources of the unstructured data, where the entity has relational data attributes. A representation for the entity is created, where the representation includes embeddings that are numeric vectors computed using machine learning embedding models, including trunk models, where a trunk model is a machine learning model trained on data in a self-supervised manner. An enrichment model is created to predict a property of the entity. A query is processed to produce a query result, where the query is applied to one or more of the entity, the embeddings, the machine learning embedding models, and the enrichment model.

Abstract: Described herein are technologies that facilitate effective use (e.g., indexing and searching) of non-text machine data (e.g., audio/visual data) in an event-based machine-data intake and query system.

Abstract: Methods and systems for determining event probabilities and anomalous events are provided. In one implementation, a method includes: receiving source data, where the source data is configured as a plurality of events with associated timestamps; searching the source data, where the searching provides a search result including N events from the plurality of events, where N is an integer greater than one, where each event of the N events includes a plurality of field values, where at least one event of the N events can include one or more categorical field values and one or more numerical field values; and for an event of the N events, determining a probability of occurrence for each field value of the plurality of field values; and using probabilities determined for the plurality of field values, determining a probability of occurrence for the event.

Abstract: As described herein, a portion of machine data of a message may be analyzed to infer, using an inference model, a sourcetype of the message. The portion of machine data may be generated by one or more components in an information technology environment. Based on the inference, a set of extraction rules associated with the sourcetype may be selected. Each extraction rule may define criteria for identifying a sub-portion of text from the portion of machine data of the message to produce a value. The set of extraction rules may be applied to the portion of machine data of the message to produce a result set that indicates a number of values identified using the set of extraction rules. Based on the result set, at least one action may be performed on one or more of inference data associated with the inference model and one or more messages.

Abstract: Embodiments of the present invention are directed to facilitating distributed data processing for machine learning. In accordance with aspects of the present disclosure, a set of commands in a query to process at an external computing service is identified. For each command in the set of commands, at least one compute unit including at least one operation to perform at the external computing service is identified. Each of the at least one compute unit associated with each command is analyzed to identify an optimized manner in which to execute the set of commands at the external computing service. An indication of the optimized manner in which to execute the set of commands and a corresponding set of data is provided to the external computing service to utilize for executing the set of commands at the external computing service.

Abstract: A communication platform may comprise different systems for helping a user discover features of the platform. However, the systems may generate different results. An application programming interface (API) may receive such outputs and may be configured to select between the outputs based on detecting a state at a user's computing device and/or using a machine-learned model to weight the outputs and/or probabilities associated therewith using a target metric. The API may then rank the outputs and select from among them based at least in part on the target metric.

Abstract: Described herein is a technology that facilitates the production of and the use of automated datagens for event-based systems. A datagen (i.e., data-generator or data generation system) is a component, module, or subsystem of computer systems that searches, monitors, and analyzes machine data. Existing datagens are not capable of detecting an anomaly in machine data. An anomaly is a variance in the input data stream that exceeds some acceptable amount of deviation from the norm (i.e., standard, expectation, etc.). An embodiment of datagen, in accordance with the technology described herein, detects anomalies in the input machine data.

Abstract: Discovery of communication platform features or exposure of such features to the user may include generating embeddings for a variety of types of communication platform content and communications. These embeddings may be used to characterize and compare various communication platform features and ultimately expose these features to a user when the user may not have otherwise encountered them. The embeddings may additionally or alternatively be used to determine a degree of alignment.

Abstract: A non-transitory computer readable storage medium has instructions executed by a processor to receive from a network connection different sources of unstructured data. An entity is formed by combining one or more sources of the unstructured data, where the entity has relational data attributes. A representation for the entity is created, where the representation includes embeddings that are numeric vectors computed using machine learning embedding models, including trunk models, where a trunk model is a machine learning model trained on data in a self-supervised manner. An enrichment model is created to predict a property of the entity. A query is processed to produce a query result, where the query is applied to one or more of the entity, the embeddings, the machine learning embedding models, and the enrichment model.

Abstract: A non-transitory computer readable storage medium has instructions executed by a processor to receive from a network connection different sources of unstructured data, where the unstructured data has multiple modes of semantically distinct data types and the unstructured data has time-varying data instances aggregated over time. An entity combining different sources of the unstructured data is formed. A representation for the entity is created, where the representation includes embeddings that are numeric vectors computed using machine learning embedding models. These operations are repeated to form an aggregation of multimodal, time-varying entities and a corresponding index of individual entities and corresponding embeddings. Proximity searches are performed on embeddings within the index.

Abstract: Operational machine components of an information technology (IT) or other microprocessor- or microcontroller-permeated environment generate disparate forms of machine data. Network connections are established between these components and processors of an automatic data intake and query system (DIQS). The DIQS conducts network transactions on a periodic and/or continuous basis with the machine components to receive the disparate data and ingest certain of the data as measurement entries of a DIQS metrics datastore that is searchable for DIQS query processing. The DIQS may receive search queries to process against the received and ingested data via an exposed network interface. In one example embodiment, a query building component conducts a user interface using a network attached client device. The query building component may elicit search criteria via the user interface using a natural language interface, construct a proper query therefrom, and present new information based on results returned from the DIQS.