Abstract: Techniques for implementing dynamic timeout-based fault detection in a distributed system are provided. In one set of embodiments, a node of the distributed system can set a timeout interval to a minimum value and transmit poll messages to other nodes in the distributed system. The node can further wait for acknowledgement messages from all of the other nodes, where the acknowledgement messages are responsive to the poll messages, and can check whether it has received the acknowledgement messages from all of the other nodes within the timeout interval. If the node has failed to receive an acknowledgement message from at least one of the other nodes within the timeout interval and if the timeout interval is less than a maximum value, the node can increment the timeout interval by a delta value and can repeat the setting, the transmitting, the waiting, and the checking steps.

Abstract: Some embodiments provide a method for assigning different service path identifiers to each of a set of different service paths along each of which a same set of service operations are performed on a set of packets. The method retrieves an available service path identifier from an identifier storage that stores service path identifiers for multiple service paths. The method generates a set of proposed service path identifiers based on the retrieved available service path identifier and the number of service paths in the set of service paths. The method performs a write operation on the identifier storage based on the set of proposed service path identifiers, and based on a determination that the write operation was successful, assigns the set of proposed service path identifiers to the set of service paths. Based on the set of service path identifiers, the method forwards the set of packets along the set of service paths.

Abstract: Certain aspects described herein are directed to methods and systems for updating a status of a first process. In certain aspects, a method includes periodically examining a first table stored in a consistent and distributed database, wherein the table includes a first indication of a status of the first process and a second indication of a status of a second process and the first process and the second process are clustered for implementing functions of a distributed application in a distributed manner. The method further includes determining that the first indication has not changed during a first time period and changing the status of the first process in a second table to reflect that the first process is non-operational, wherein a different share of workload associated with the distributed application is performed by the second process as compared to when the first process was operational.

Abstract: Certain embodiments described herein are directed to methods and systems for adding one or more nodes to a first cluster including a first node in a computer system. A method performed by the first node comprises receiving a first request from a second node to join the first cluster. The method also comprises retrieving a first cluster configuration associated with the first cluster from a distributed database through a first database server (DBS) and creating a second cluster configuration using the first cluster configuration and information received from the second node as part of the request. The method further comprises populating a first one or more local trust stores of a first one or more processes executing on the first node with a second one or more security certificates of a second one or more processes executing on the second node. The method further comprises writing the second cluster configuration to the distributed database and returning the second cluster configuration to the second node.

Abstract: A log unit provides a shared log for recording updates on data objects. Garbage collection is performed locally and in-place by the log unit. In a marking portion of the garbage collection process, the log unit identifies and marks log entries that record mergeable updates. In a deallocation portion of the process, the log unit merges one or more mergeable log entries and deallocates at least portions of the merged log entries.

Abstract: A log unit provides a shared log for recording updates on data objects. Garbage collection is performed locally and in-place by the log unit. In a marking portion of the garbage collection process, the log unit identifies and marks log entries that record supersedable updates. In a deallocation portion of the process, the log unit deallocates at least portions of the marked log entries that contain supersedable updates.

Abstract: Techniques for implementing dynamic timeout-based fault detection in a distributed system are provided. In one set of embodiments, a node of the distributed system can set a timeout interval to a minimum value and transmit poll messages to other nodes in the distributed system. The node can further wait for acknowledgement messages from all of the other nodes, where the acknowledgement messages are responsive to the poll messages, and can check whether it has received the acknowledgement messages from all of the other nodes within the timeout interval. If the node has failed to receive an acknowledgement message from at least one of the other nodes within the timeout interval and if the timeout interval is less than a maximum value, the node can increment the timeout interval by a delta value and can repeat the setting, the transmitting, the waiting, and the checking steps.

Abstract: A shared log in a distributed system provides for direct access to the most current data state of an object stored in the shared log. Directly accessing the data state of an object obviates the need for a client to replay all the transactions stored in the shared log made on the object.

Abstract: In accordance with disclosed embodiments, a shared log system includes a sequencer to verify transactions that comprise a source object and one or more members of the source object (source data objects), a target object and one or more members of the target object (target data objects), and a snapshot time reference. The sequencer verifies transaction using the snapshot time to determine whether the data states of the source data objects at the time of the snapshot time represent their most current data state in a shared log.

Abstract: Embodiments of the present disclosure relate to techniques for maintaining a state of a distributed system. In particular, certain embodiments relate to identifying a function. Some embodiments relate to, upon determining that the function comprises an annotation indicating that the function is capable of modifying the state of the distributed system, transforming the function to allow the function to generate updates to a state machine.

Abstract: In a computer-implemented method for managing deletion of logical objects, a request to delete a logical object of a managed system is received, wherein the logical object is referenced by a second logical object, and wherein deletion of the logical object does not result in deletion of the second logical object. The logical object is marked as deleted such that the logical object is unavailable for reference by the second logical object. The logical object and data associated with the logical object are maintained in a memory space of the managed system for access by a module of the managed system, such that the memory space is unavailable for any other use by the managed system.

Abstract: In a computer-implemented method for collecting firewall flow records, firewall flow records are received from a plurality of data end nodes of a virtualized infrastructure comprising a distributed firewall according to a collection schedule, wherein the collection schedule defines which data end nodes of the plurality of data end nodes from which firewall flow records are collected, a frequency of collection of firewall flow records from the data end nodes, and an amount of firewall flow records collected from the data end nodes. Firewall flow records received at a firewall flow record collection queue are processed, such that the received firewall flow records are prepared for storage at a flow record data store.

Abstract: A shared log system can provide a key-index structure (main map) that allows clients (users) to access the state of objects stored in the shared log system. The client can define secondary indices on the main map to provide access to the data that are tailored to the client's data needs. Each client can generate their own customized set of secondary indices different from other clients.

Abstract: In accordance with disclosed embodiments, a shared log system includes a sequencer to verify transactions that comprise a source object and one or more members of the source object (source data objects), a target object and one or more members of the target object (target data objects), and a snapshot time reference. The sequencer verifies transaction using the snapshot time to determine whether the data states of the source data objects at the time of the snapshot time represent their most current data state in a shared log.

Abstract: A shared log in a distributed system provides for direct access to the most current data state of an object stored in the shared log. Directly accessing the data state of an object obviates the need for a client to replay all the transactions stored in the shared log made on the object.

Abstract: Embodiments of the present disclosure relate to techniques for maintaining a state of a distributed system. In particular, certain embodiments relate to identifying a function. Some embodiments relate to, upon determining that the function comprises an annotation indicating that the function is capable of modifying the state of the distributed system, transforming the function to allow the function to generate updates to a state machine.

Abstract: Described herein are systems, methods, and software to enhance the management of data objects in a data storage system. In one implementation, a client in a data object environment is configured to identify a request for a data object in a first version from process on the client. Once the request is identified and the data object is provided or made available to the requesting process, the object storage system identifies a modification request for the data object to modify the data object from the first version to a second version. In response to the modification request, the object storage system generates an undo log entry to reflect the changes from the first version to the second version and updates the data object to the second version.

Abstract: In a computer-implemented method for managing deletion of logical objects, a request to delete a logical object of a managed system is received, wherein the logical object is referenced by a second logical object, and wherein deletion of the logical object does not result in deletion of the second logical object. The logical object is marked as deleted such that the logical object is unavailable for reference by the second logical object. The logical object and data associated with the logical object are maintained in a memory space of the managed system for access by a module of the managed system, such that the memory space is unavailable for any other use by the managed system.

Abstract: In a computer-implemented method for collecting firewall flow records, firewall flow records are received from a plurality of data end nodes of a virtualized infrastructure comprising a distributed firewall according to a collection schedule, wherein the collection schedule defines which data end nodes of the plurality of data end nodes from which firewall flow records are collected, a frequency of collection of firewall flow records from the data end nodes, and an amount of firewall flow records collected from the data end nodes. Firewall flow records received at a firewall flow record collection queue are processed, such that the received firewall flow records are prepared for storage at a flow record data store.

Abstract: A system that implements detection and reconciliation of system resource metadata for a distributed storage system is described. A node may obtain resource metadata specific to the node from another node that maintains system resource metadata for a distributed storage system. Based on the resource metadata specific to the node, a determination may be made that the node is not reconciled with the system resource metadata. A corrective operation may be performed to reconcile the node with the system resource metadata. A corrective operation may include terminating a resource, making unavailable a resource, modifying resource attributes, or sending a resource metadata update to system resource metadata for correction.