Abstract: Methods and system described herein are directed to identifying anomalous behaving components of a distributed computing system. Methods and system collect log messages generated by a set of event log source running in the distributed computing system within an observation time window. Frequencies of various types of event messages generated within the observation time window are determined for each of the log sources. A similarity value is calculated for each pair of event sources. The similarity values are used to identify similar clusters of event sources of the distributed computing system for various management purposes. Components of the distributed computing system that are used to host the event source outliers may be identified as potentially having problems or may be an indication of future problems.

Abstract: The current document is directed to methods and systems that process, classify, efficiently store, and display large volumes of event messages generated in modern computing systems. In a disclosed implementation, event messages are assigned types and transformed into event records with well-defined fields that contain field values. Recurring patterns of event messages, referred to as “transactions,” are identified within streams or sequences of time-associated event messages and streams or sequences of time-associated event records.

Abstract: Methods and systems to narrow a search for potential sources of problems in a distributed computing system are described. A volatile event type of event messages recorded in an event-log file is identified. The volatile event type is an event type that may have unexpectedly increased in frequency over an observation time window. An historical period of time may be selected to search for potential sources of the volatile event type. Frequencies of event messages in the event-log file with the same event type as the volatile event type are determined for time intervals of the historical period of time. A time interval of the historical period of time with a largest increase in frequency of event messages is identified. A list of event messages of the event-log file in a selected sub-time interval of the sub-time intervals of the time interval are displayed in a graphical user interface.

Abstract: Methods and systems to identify log write instructions of a source code as potential sources of an event message of interest are described. Methods identify non-parametric tokens, such as text strings and natural language words and phrases, of an event message of interest. Candidate log write instructions and associated line numbers in a source code are identified. Non-parametric tokens of each event message of the one or more candidate log write instructions are determined. A confidence score is calculated for each candidate log write instruction based the number of non-parametric tokens the event message of interest and event message of the candidate log write instruction have in common. The candidate log write instructions are rank ordered based on the corresponding one or more confidence scores and the rank ordered candidate log write instructions and associated line numbers of the source code may be displayed in a graphical user interface.

Abstract: Methods and systems to sample event messages are described. As event messages are generated by one or more sources, the event messages are stored in a storage queue. An event message policy that represents conditions for storing event messages in a sample log file are input. For each event message output from the storage queue, the event message may be stored in a sample log file when one or more of the conditions of the event message policy are satisfied. The event messages of the sample log file may be displayed in a graphical user interface that enables a user to change the event message policy.

Abstract: Methods and systems that detect computer system anomalies based on log file sampling are described. Computers systems generate log files that record various types of operating system and software run events in event messages. For each computer system, a sample of event messages are collected in a first time interval and a sample of event messages are collected in a recent second time interval. Methods calculate a difference between the event messages collected in the first and second time intervals. When the difference is greater than a threshold, an alert is generated. The process of repeatedly collecting a sample of event messages in a recent time interval, calculating a difference between the event messages collected in the recent and previous time intervals, comparing the difference to the threshold, and generating an alert when the threshold is violated may be executed for each computer system of a cluster of computer systems.

Abstract: Selecting a storage device to be mapped to a requested virtual disk includes maintaining attributes for a set of storage devices including a cost attribute, a capacity attribute and quality-of-service attributes. A request for a virtual disk includes request parameters including a price parameter, a capacity parameter and quality-of-service parameters. A device selection function generates a score for each storage device based on the request parameters and the storage device attributes, and identifies a best suited storage device by comparing the respective scores of the storage devices. The virtual disk is then created in satisfaction of the request with a mapping to the identified storage device to provide underlying physical data storage for the virtual disk.

Abstract: The current document is directed to methods and systems that process, classify, efficiently store, and display large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event messages to clusters. Event types associated with the clusters are used for gathering information from various information sources with which to automatically annotate event messages displayed to system administrators, maintenance personnel, and other users of event messages.

Abstract: The current document is directed to systems, and methods incorporated within the systems, that carry out probability-distribution-based analysis of log-file entries. A monitoring subsystem within a distributed computer system uses probability-distribution-based analysis of log-file entries to detect changes in the state of the distributed computer system. A log-file-analysis subsystem within a distributed computer system uses probability-distribution-based analysis of log-file entries to identify subsets of log-file entries that predict anomalies and impending problems in the distributed computer system. In many implementations, a numerical comparison of probability distributions of log-file-entry types is used to detect state changes in the distributed computer system.

Abstract: The current document is directed to methods and systems for processing, classifying, and efficiently storing large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to clusters based on metrics computed for the event messages. In addition, a significance value is determined for each received event message. When the significance value exceeds a threshold value, one or more actions are taken, including marking an event record corresponding to the event message, storing an event record corresponding to the event message in a significant-event log, and generating a notice or alarm.

Abstract: The current document is directed to methods and systems for processing, classifying, and efficiently storing large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are normalized to identify non-parameter tokens within the event messages. The non-parameter event tokens are used to compute a metric for each event message. The metrics are used, in turn, to identify a type-associated cluster to which to assign each received event message. The type-associated clusters are created dynamically as streams of event messages are processed. The type-associated clusters may be dynamically split and merged to refine event-message typing.

Abstract: The current document is directed to methods and systems for processing, classifying, and efficiently storing large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event massages to clusters.

Abstract: A computer of a data storage system carries out user input-output requests for data of a set of virtual disks by performing back-end input-output requests to a set of overprovisioned storage devices providing underlying physical storage. A set of time series of utilization measures are continually generated and stored, with each reflecting an amount of a given virtual disk consumed by user data at a given time. The time series are used to generate predicted utilization measures, each being an estimate of an amount of a given virtual disk expected to be consumed by user data at a future time. A fullness probability measure is generated by aggregating the predicted utilization measures, and it reflects a probability that the set of overprovisioned storage devices will be fully consumed by user data at the future time.

Abstract: A system for organizing email includes a modeler operable to infer a temporal sequential process from a corpus of unlabeled email messages, and a categorizer operable to accept an incoming message and map the aforesaid incoming message to a transition in the aforesaid temporal sequential process.

Abstract: Synchronizing resources shared among a plurality of devices includes determining access information about each of the resources, determining activity for each of the resources on at least a subset of the devices based on the access information, estimating future use of the devices, providing an estimate of a desire to have particular resources on particular devices according to the activity and future use, and synchronizing resources according to the estimate of a desire to have particular resources on particular devices. The resources may be data files. Access information may include information about when the resources were read and/or may include information about when the resources were renamed, created, and modified. Estimating a desire to have particular resources on particular devices may take into account a user's desire to maintain backup files and/or maintain certain types of files on certain devices.

Abstract: A device, system, and method are directed towards enabling a user to employ a set of database-like structured query expressions to manage data retrieval over a network, and the transformation and/or normalization of the data. In one embodiment, the retrieval expressions are configured as database-like structured query commands that may be performed upon at least a non-database arrangement of content over the network. In one embodiment, retrieved data is converted to at least one format intermediate to a first and second format in a sequence of transformations.

Abstract: A system for organizing email comprising: (a) a modeler operable to infer a temporal sequential process from a corpus of unlabeled email messages; and (b) a categorizer operable to accept an incoming message and map the aforesaid incoming message to a transition in the aforesaid temporal sequential process.

Abstract: A system for organizing email comprising: (a) a modeler operable to infer a temporal sequential process from a corpus of unlabeled email messages; and (b) a categorizer operable to accept an incoming message and map the aforesaid incoming message to a transition in the aforesaid temporal sequential process.

Abstract: A procedure is disclosed for automatically constructing wrappers for performing information-extraction from sites such as Internet resources that display relevant information, interspersed with extraneous text fragments, such as HTML formatting commands or advertisements. The procedure has three basic steps. First, a set of example pages are collected with a subroutine named GatherExamples. Gather Examples is provided with information describing how to pose example queries to the site whose wrapper is to be learned. Second, these example pages are labeled by a subroutine named LabelExamples—i.e., the information to be extracted from each example is identified for use in the third step. The LabelExamples subroutine uses a general framework for labeling pages using site-specific heuristics called recognizers, as well as allowing users to correct and modify the recognized instances. Finally, the labeled example pages are passed to a BuildWrapper subroutine, which constructs a wrapper.