Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.

Abstract: Classification for data intake operations in an enterprise ensures that sensitive data is not disseminated inappropriately, but incurs substantial time, effort and expense. A method of classifying data in a large set of data repositories captures a set of raw rules resulting from inputs indicative of evaluations and conclusions of data classification operations, typically by logging data classification operations, and identifies patterns in the set of raw rules by consolidating duplicative conditions and eliminating inconsequential conditions. External conditions and observations may be referenced for applying a context to the rules based on a usage or domain of the data, and data sets of disparate entities may be examined for anonymizing the data and combining with other sets of anonymized data.

Abstract: Classification for data intake operations in an enterprise ensures that sensitive data is not disseminated inappropriately, but incurs substantial time, effort and expense. A method of classifying data in a large set of data repositories captures a set of raw rules resulting from inputs indicative of evaluations and conclusions of data classification operations, typically by logging data classification operations, and identifies patterns in the set of raw rules by consolidating duplicative conditions and eliminating inconsequential conditions. External conditions and observations may be referenced for applying a context to the rules based on a usage or domain of the data, and data sets of disparate entities may be examined for anonymizing the data and combining with other sets of anonymized data.

Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.

Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.