Abstract: A method of dynamic cluster manager failover includes routing data traffic associated with managing a plurality of indexers in a cluster to a first cluster manager, wherein the first cluster manager is associated with an active role and is operable to manage the plurality of indexers in the cluster. The method also includes transmitting periodic heartbeat request messages from a second cluster manager of the cluster to the first cluster manager, wherein the second cluster manager is associated with a standby role. Further, the method includes detecting, at the second cluster manager, a loss of heartbeat response messages from the first cluster manager. Also, the method includes receiving information from a set of indexers regarding a status of the first cluster manager and in response to a determination that the status of the first cluster manager is offline, promoting the second cluster manager to switch over to the active role.

Abstract: A computing device receives an ingest preview request to preview events to be stored by at least one indexer. Responsive to the ingest preview request, the computing device sends a subscription request to the forwarders. The forwarders receive the subscription request and intercept the events that are being sent to at least one of the indexers. The forwarders then clone matching events to the subscription request and responds to the computing device with the matching events. When the computing device receives the matching events, the computing device adds the matching events to a dispatch directory. The user interface is then populated with events in the dispatch directory.

Abstract: A method for file system destinations includes obtaining events for storage on one or more of the storage systems. For each event, the method includes extracting at least one field value from the event, comparing the at least one field value to configurations of the storage systems to identify at least one storage system of the plurality of storage systems having a matching configuration, transmitting the event to an ingest module queue for the at least one storage system, selecting a partition for the event based on the at least one field value to obtain a selected partition, mapping the selected partition to a file using a partition mapping, and appending the event to the file on the at least one storage system.

Abstract: A computing device receives an ingest preview request to preview events to be stored by at least one indexer. Responsive to the ingest preview request, the computing device sends a subscription request to the forwarders. The forwarders receive the subscription request and intercept the events that are being sent to at least one of the indexers. The forwarders then clone matching events to the subscription request and responds to the computing device with the matching events. When the computing device receives the matching events, the computing device adds the matching events to a dispatch directory. The user interface is then populated with events in the dispatch directory.

Abstract: A method for file system destinations includes obtaining events for storage on one or more of the storage systems. For each event, the method includes extracting at least one field value from the event, comparing the at least one field value to configurations of the storage systems to identify at least one storage system of the plurality of storage systems having a matching configuration, transmitting the event to an ingest module queue for the at least one storage system, selecting a partition for the event based on the at least one field value to obtain a selected partition, mapping the selected partition to a file using a partition mapping, and appending the event to the file on the at least one storage system.

Abstract: A computer-implemented method of providing for dynamic cluster manager failover includes routing data traffic associated with managing network components in a cluster to an active cluster manager, where the active cluster manager manages a plurality of network components in the cluster. The method also includes transmitting periodic heartbeat request messages from a standby cluster manager to the active cluster manager. Further, the method includes detecting a loss of heartbeat response messages from the active cluster manager, where the heartbeat response messages are transmitted from the standby cluster manager to the active cluster manager in response to the periodic heartbeat request messages. The method also includes promoting the standby cluster manager to an active role and re-routing the data traffic associated with managing the network components to the currently active cluster manager.

Abstract: Multiple storage system event handling includes obtaining multiple events for storage on multiple storage systems. For each of the multiple events, field values from each event are extracted. The field values are matched to configurations of the storage systems to identify a subset of the storage system having a matching configuration. The event is transmitted to the subset. The multiple events are transmitted to heterogeneous subsets.

Abstract: Ingest health monitoring includes receiving an event stream of events in a data intake and query system to store on at least one storage system and obtaining an event from the event stream. Ingest health monitoring further includes transmitting the event to a selected ingest module queue for the event, updating an output rate indicator counter for the selected ingest module queue when failure to store the event in the ingest module queue occurs, obtaining the event from the selected ingest module queue, processing the event to generate a file for the event, and transmitting the file to the at least one storage system. Ingest health monitoring further includes updating the write failure indicator counter for a storage system of the at least one storage system when failure to transmit to the storage system occurs and updating the user interface based on the output rate indicator counter and the write failure indicator counter.

Abstract: In accordance with various embodiments of the present disclosure, a first instance of a data intake and query system (DIQS) may receive latency data that indicates latency states of second instances of the DIQS, the latency states indicative of latencies associated with processing of event data by the plurality of second instances. The first instance may then determine overall latency state of the first instance based, at least in part, on determining number or percentage of the first instance and the second instances of the DIQS having one or more particular latency states, and determining whether the number or percentage of the first instance and the f second instances of the DIQS having the one or more particular latency states is equal to or exceeds a threshold. The first instance may then present the overall latency state of the first instance.

Abstract: Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

Abstract: Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.

Abstract: Embodiments of the present disclosure provide solutions for determining an elected search head captain is unqualified for the position, identifying a more qualified search head, and transferring the captain position to the more qualified search head. A method is provided that includes referencing qualification parameters in an elected search head captain, determining whether the newly elected search head captain is qualified for the position based on the parameters, identifying a more qualified search head to be the search head captain if the newly elected search head captain is determined to be unqualified for the position, and transferring the position of captain to the more qualified search head. The qualification parameters may include, for example, a pre-determined static flag set by an administrator of the search environment, and configuration replication status that corresponds to the most recent configuration state of the search head as recorded by the previous search head captain.

Abstract: The present disclosure provides solutions for determining the divergence (delta) between the current and previous reference data structures for mutable data in a search head. A method is provided that includes updating a pre-existing lookup table in a search head, generating a delta file that identifies the divergence between the updated and previous lookup table, and distributing the delta file to other components in the search environment. The compatibility of the delta file is checked with the local instance of the lookup table in each search component, and the lookup table is applied if compatibility is determined. However, if the delta file is determined to not be compatible with the current version of a local lookup table in an indexer, the entire lookup table sent to the requesting indexer instead.

Abstract: Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

Abstract: Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.

Abstract: Systems and methods are disclosed for monitoring features of a computing device of a distributed computing system using a self-monitoring module. The self-monitoring module can include multiple feature-specific monitoring modules and one or more parent nodes for the feature-specific monitoring modules. A feature-specific monitoring module can identify or detect a fault status change, such as a fault condition or fault resolution, for one or more features. Based on the identified fault conditions or fault resolutions, the feature-specific monitoring module can determine an internal status and communicate an updated status to a parent node.

Abstract: Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

Abstract: Embodiments of the present disclosure provide solutions for determining an elected search head captain is unqualified for the position, identifying a more qualified search head, and transferring the captain position to the more qualified search head. A method is provided that includes referencing qualification parameters in an elected search head captain, determining whether the newly elected search head captain is qualified for the position based on the parameters, identifying a more qualified search head to be the search head captain if the newly elected search head captain is determined to be unqualified for the position, and transferring the position of captain to the more qualified search head. The qualification parameters may include, for example, a pre-determined static flag set by an administrator of the search environment, and configuration replication status that corresponds to the most recent configuration state of the search head as recorded by the previous search head captain.

Abstract: Embodiments of the present disclosure provide solutions for determining an elected search head captain is unqualified for the position, identifying a more qualified search head, and transferring the captain position to the more qualified search head. A method is provided that includes referencing qualification parameters in an elected search head captain, determining whether the newly elected search head captain is qualified for the position based on the parameters, identifying a more qualified search head to be the search head captain if the newly elected search head captain is determined to be unqualified for the position, and transferring the position of captain to the more qualified search head. The qualification parameters may include, for example, a pre-determined static flag set by an administrator of the search environment, and configuration replication status that corresponds to the most recent configuration state of the search head as recorded by the previous search head captain.

Abstract: The present disclosure provides solutions for determining the divergence (delta) between the current and previous reference data structures for mutable data in a search head. A method is provided that includes updating a pre-existing lookup table in a search head, generating a delta file that identifies the divergence between the updated and previous lookup table, and distributing the delta file to other components in the search environment. The compatibility of the delta file is checked with the local instance of the lookup table in each search component, and the lookup table is applied if compatibility is determined. However, if the delta file is determined to not be compatible with the current version of a local lookup table in an indexer, the entire lookup table sent to the requesting indexer instead.