Abstract: Embodiments of the present disclosure relate to generating a high level security policy for a data repository without knowledge of the access control, entitlement, and other models of the data repository. A set of abstractions that define a security policy language may be generated based on data in a data repository collection. The set of abstractions may define a security policy language, which may be provided to a security administrator who can define a security policy with the security policy language. The security policy may be translated into a common physical language to generate a common physical policy. The processing device may then translate the common physical policy into a set of commands for each of one or more data repositories that the data repository collection is comprised of.

Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.

Abstract: Embodiments of the present disclosure relate to utilizing an existing login process of a data repository to enable the data repository to delegate MFA functionality to an external MFA system. When a purported user attempts to log in to the data repository, a delegation module within the login process may insert a record into a table associated with the login process. A program executing on a security device external to the data repository may periodically poll the table for new records and upon detecting the new record, may call the external MFA system to verify the login attempt. The external MFA system may indicate to the program whether the login attempt was verified and the program may update the table with the indication. Upon detecting the indication, the delegation module may complete or terminate the login attempt based on the indication.

Abstract: Embodiments of the present disclosure relate to generating a high level security policy for a data repository without knowledge of the access control, entitlement, and other models of the data repository. A set of abstractions that define a security policy language may be generated based on data in a data repository collection. The set of abstractions may define a security policy language, which may be provided to a security administrator who can define a security policy with the security policy language. The security policy may be translated into a common physical language to generate a common physical policy. The processing device may then translate the common physical policy into a set of commands for each of one or more data repositories that the data repository collection is comprised of.

Abstract: Classification for data intake operations in an enterprise ensures that sensitive data is not disseminated inappropriately, but incurs substantial time, effort and expense. A method of classifying data in a large set of data repositories captures a set of raw rules resulting from inputs indicative of evaluations and conclusions of data classification operations, typically by logging data classification operations, and identifies patterns in the set of raw rules by consolidating duplicative conditions and eliminating inconsequential conditions. External conditions and observations may be referenced for applying a context to the rules based on a usage or domain of the data, and data sets of disparate entities may be examined for anonymizing the data and combining with other sets of anonymized data.

Abstract: Classification for data intake operations in an enterprise ensures that sensitive data is not disseminated inappropriately, but incurs substantial time, effort and expense. A method of classifying data in a large set of data repositories captures a set of raw rules resulting from inputs indicative of evaluations and conclusions of data classification operations, typically by logging data classification operations, and identifies patterns in the set of raw rules by consolidating duplicative conditions and eliminating inconsequential conditions. External conditions and observations may be referenced for applying a context to the rules based on a usage or domain of the data, and data sets of disparate entities may be examined for anonymizing the data and combining with other sets of anonymized data.

Abstract: Data storage for unstructured data such as JSON data stored as collections of documents transforms the JSON data into a columnar form of storing unstructured data by grouping similar fields together for facilitating retrieval of the individual fields from a range of documents. Groups of fields are stored in individual files for each field. Compound data such as arrays and subdocuments are also broken down into files for each atomic field. In other words, a compound document structure that defines a hierarchy or “tree” of fields is flattened such that each “leaf” of the tree is stored in a separate file.

Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.

Abstract: A log message classifier employs machine learning for identifying a corresponding parser for interpreting the incoming log message and for retraining a classification logic model processing the incoming log messages. Voluminous log messages generate a large amount of data, typically in a text form. Data fields are parseable from the message by a parser that knows a format of the message. The classification logic is trained by a set of messages having a known format for defining groups of messages recognizable by a corresponding parser. The classification logic is defined by a random forest that outputs a corresponding group and confidence value for each incoming message. Groups may be split to define new groups based on a recurring matching tail (latter portion) of the incoming messages. A trend of decreased confidence scores triggers a periodic retraining of the random forest, and may also generate an alert to operators.

Abstract: A query server performs method of generating a query result using an aggregation pipeline by identifying, based on a query, a sequence of operations to be applied to documents from an unstructured database, in which a portion of the operations are dependent on other operations in the sequence of operations. The pipeline determines, from the operations, lightweight and heavyweight operations, in which the heavyweight operations generate a materialized result have a substantial impact on processing resources. The pipeline defers the lightweight operations until a materialized result is needed, for performing with a corresponding heavyweight operation, in which the materialized result includes either creation of a new document or movement of substantial data from a document. Lightweight operations are grouped with heavyweight operations such that multiple operations can be collapsed into a single operation that act upon the data together thus avoiding the number of materializations.

Abstract: Data traffic is monitored on a network with data access elements thereof collected and compared to security rules. An audit data collection is sent to a repository responsive to data access elements matching a condition of the security rules, where security rules having the condition designate the audit data collection and repository. A tag to data traffic is applied responsive to the matching condition. Comparing of collected data access elements to the corresponding security rules having the matching condition is discontinued responsive to applying the tag. The tag indicates a repository and the data traffic includes a connection and session. An audit data collection is sent to the repository indicated by the tag for a data access responsive to the tag in the tagged data traffic. The method continues sending audit data for future data accesses in the tagged data traffic without comparing to the corresponding security rules again.

Abstract: A query engine for an unstructured database satisfies window based queries and analytics by defining a window of documents, and performing analytics on the window using a default value for omitted field. A tabular index containing only values needed for analytics and document ordering defines each window. The tabular index includes all fields from each document that are required to satisfy the query, retrieved on a single pass by the query engine so that multiple fetches to the same document are avoided. Since each document in the window need not contain all the same fields as the other documents, an adapter includes logic for defining a default or placeholder value for a field called for in an analytic computation but nonexistent in a particular document. By retrieving only the computationally relevant fields, and by performing the retrieval only once on each document, the I/O overhead is greatly reduced.

Abstract: An analytics processing system generates analytics from a collection of unstructured data by identifying trends in the data and deriving associations or correlations between series of values. Each series is generated from a set of field labeled values in the set, and compared to other series in the collection. Identified relationships in the series are scored based on depiction of an illustrative, predictive, or non-random association, and ranked by a scoring metric for analytical value. A visualization of the relationships are ranked and rendered such that the visualization highlights the association in a manner not achievable by simple inspection of the field values. Relationships are graphed by lines, circles, bars (histogram) on labeled axes based on the series. In this manner, a user may generate analytic results from a large data set, and pinpoint significant associations by paging through renderings scored as the most illustrative of notable trends.

Abstract: Data traffic is monitored on a network with data access elements thereof collected and compared to security rules. An audit data collection is sent to a repository responsive to data access elements matching a condition of the security rules, where security rules having the condition designate the audit data collection and repository. A tag to data traffic is applied responsive to the matching condition. Comparing of collected data access elements to the corresponding security rules having the matching condition is discontinued responsive to applying the tag. The tag indicates a repository and the data traffic includes a connection and session. An audit data collection is sent to the repository indicated by the tag for a data access responsive to the tag in the tagged data traffic. The method continues sending audit data for future data accesses in the tagged data traffic without comparing to the corresponding security rules again.

Abstract: Data traffic is monitored on a network with data access elements thereof collected and compared to security rules. An audit data collection is sent to a repository responsive to data access elements matching a condition of the security rules, where security rules having the condition designate the audit data collection and repository. A tag to data traffic is applied responsive to the matching condition. Comparing of collected data access elements to the corresponding security rules having the matching condition is discontinued responsive to applying the tag. The tag indicates a repository and the data traffic includes a connection and session. An audit data collection is sent to the repository indicated by the tag for a data access responsive to the tag in the tagged data traffic. The method continues sending audit data for future data accesses in the tagged data traffic without comparing to the corresponding security rules again.

Abstract: Data traffic is monitored on a network and data access elements thereof are collected. The collected data access elements are compared to security rules. A first audit data collection is sent to a first repository in response to one or more data access elements of a first data access matching a first condition of one of the security rules. The one of the security rules having the first condition designates the first audit data collection and the first repository. A second audit data collection is sent to a second repository in response to one or more data access elements of a second data access matching a second condition of one of the security rules. The one of the security rules having the second condition designates the second audit data collection and the second repository.

Abstract: Data traffic is monitored on a network with data access elements thereof collected and compared to security rules. An audit data collection is sent to a repository responsive to data access elements matching a condition of the security rules, where security rules having the condition designate the audit data collection and repository. A tag to data traffic is applied responsive to the matching condition. Comparing of collected data access elements to the corresponding security rules having the matching condition is discontinued responsive to applying the tag. The tag indicates a repository and the data traffic includes a connection and session. An audit data collection is sent to the repository indicated by the tag for a data access responsive to the tag in the tagged data traffic. The method continues sending audit data for future data accesses in the tagged data traffic without comparing to the corresponding security rules again.

Abstract: An analytics processing system generates analytics from a collection of unstructured data by. transforming a received source of input data from an unstructured database into a delimiterless form, and iteratively moving portions of the delimiterlesss input data from a solid-state memory to a shared memory adapted for parallel operations with a plurality of GPU cores. The method stores computational data, such as values for matching, in a high speed memory responsive to operations with the shared memory, in which the high-speed memory remains static for the duration of the iterations. A host CPU invokes the plurality of cores for performing the parallel operations on the computational data and the portions of the delimiterless input data, and stores a result in a general memory accessible from a graphical user interface (GUI). The GPU cores parallelize the matching task of the input data from the unstructured database against the match data.

Abstract: A tabular (relational) DB interface is responsive to SQL commands for accessing unstructured data bases. An application receives a relational data command or query and maps fields from the relational query to fields in an unstructured data store including documents arranged in a hierarchy and unbounded by fixed types or field lengths. The application generates field names by concatenating nested hierarchy field names to define unique “flat file” field names in a tabular form. The application generates a catalog defining the mapping which is used as metadata for accessing the unstructured data to satisfy the relational query. Use of the metadata avoids copying or translating the unstructured data store to a tabular form because the unstructured data collection remains unmodified, and is accessed via the catalog.

Abstract: Data traffic is monitored on a network and data access elements thereof are collected. The collected data access elements are compared to security rules. A first audit data collection is sent to a first repository in response to one or more data access elements of a first data access matching a first condition of one of the security rules. The one of the security rules having the first condition designates the first audit data collection and the first repository. A second audit data collection is sent to a second repository in response to one or more data access elements of a second data access matching a second condition of one of the security rules. The one of the security rules having the second condition designates the second audit data collection and the second repository.