Abstract: Techniques are disclosed relating to managing distributed storage of data for various entities according to classifications for these entities. A database node of a distributed storage system may receive, from a first entity of a plurality of entities, a request to store a set of data. The database node may further obtain metadata associated with the first entity, wherein the metadata specifies one of a plurality of classifications for the entities. The database node may provide the set of data to one or more of a plurality of caches for storage. The caches may be located in two or more availability zones and are configured to store the set of data based on the classification for the first entity identified in the metadata associated with the first entity. The database node may also store the set of data in a shared object storage coupled to the database node.

Abstract: Techniques are disclosed relating to upgrade groups. A node of a computer system may access metadata assigned to the node during deployment of the node. The node may be one of a plurality of nodes associated with a service that is implemented by the computer system. The node may perform an operation on the metadata to derive a group identifier for the node and the group identifier may indicate the node's membership in one of a set of groups of nodes managed by the service. The node may then store the group identifier in a location accessible to the service.

Abstract: Techniques are disclosed relating to managing distributed storage of data across availability zones and the replication of data in case of storage server failures. A distributed storage system may include storage servers distributed across availability zones with an auditor instantiated in at least one storage server. The auditor manages the replication of data in the event of one or more storage servers failure in an availability zone. In the event of the failure, the auditor may determine the extent of the failure and whether the failure involves a small number or a large number of storage servers. In the event a large number of storage servers being affected, the auditor may delay replication of data to see if the failure is temporary and avoid unwanted data transfer of large amounts of data across availability zones.

Abstract: Techniques are disclosed relating to storing location information about storage nodes in cookies. A cloud-based service may send location requests to an orchestration service that instantiated storage nodes included in a storage cluster of the cloud-based service. The cloud-based service may receive location information that identifies in which computer zone that a given storage node is located. The cloud-based service may store the location information in cookies at a metadata store that is shared among the storage nodes. The cloud-based service may receive, from a client node, a search request to identify ones of the storage nodes that store particular data. The cloud-based service may return a set of cookies corresponding to identified storage nodes. The set of cookies may enable the client node to determine whether there is a storage node that stores the particular data and is within the same computer zone as the client node.

Abstract: Techniques are disclosed relating to storing location information about storage nodes in cookies. A cloud-based service may send location requests to an orchestration service that instantiated storage nodes included in a storage cluster of the cloud-based service. The cloud-based service may receive location information that identifies in which computer zone that a given storage node is located. The cloud-based service may store the location information in cookies at a metadata store that is shared among the storage nodes. The cloud-based service may receive, from a client node, a search request to identify ones of the storage nodes that store particular data. The cloud-based service may return a set of cookies corresponding to identified storage nodes. The set of cookies may enable the client node to determine whether there is a storage node that stores the particular data and is within the same computer zone as the client node.

Abstract: Techniques are disclosed relating to storing location information about storage nodes in cookies. A cloud-based service may send location requests to an orchestration service that instantiated storage nodes included in a storage cluster of the cloud-based service. The cloud-based service may receive location information that identifies in which computer zone that a given storage node is located. The cloud-based service may store the location information in cookies at a metadata store that is shared among the storage nodes. The cloud-based service may receive, from a client node, a search request to identify ones of the storage nodes that store particular data. The cloud-based service may return a set of cookies corresponding to identified storage nodes. The set of cookies may enable the client node to determine whether there is a storage node that stores the particular data and is within the same computer zone as the client node.

Abstract: Techniques are disclosed relating to managing distributed storage of data for various entities according to classifications for these entities. A database node of a distributed storage system may receive, from a first entity of a plurality of entities, a request to store a set of data. The database node may further obtain metadata associated with the first entity, wherein the metadata specifies one of a plurality of classifications for the entities. The database node may provide the set of data to one or more of a plurality of caches for storage. The caches may be located in two or more availability zones and are configured to store the set of data based on the classification for the first entity identified in the metadata associated with the first entity. The database node may also store the set of data in a shared object storage coupled to the database node.

Abstract: Techniques are disclosed relating to managing distributed storage of data across availability zones and the replication of data in case of storage server failures. A distributed storage system may include storage servers distributed across availability zones with an auditor instantiated in at least one storage server. The auditor manages the replication of data in the event of one or more storage servers failure in an availability zone. In the event of the failure, the auditor may determine the extent of the failure and whether the failure involves a small number or a large number of storage servers. In the event a large number of storage servers being affected, the auditor may delay replication of data to see if the failure is temporary and avoid unwanted data transfer of large amounts of data across availability zones.

Abstract: Techniques are disclosed relating to managing distributed storage of data with low latency access and durable storage. A distributed storage system may include storage caches distributed across availability zones in front of a shared object storage. Database nodes collocated with the storage caches may handle the storage, retrieval, and manipulation of data in the storage caches and the shared object storage. The shared object storage cache provides persistent storage of data for the distributed storage system while storage of the data in the storage caches is implemented to provide low latency retrieval of the storage. The storage caches across the availability zones may be populated through the shared object storage rather than moving data across availability zones.

Abstract: Techniques are disclosed relating to a database system synchronizing a current state of the database system among a plurality of nodes configured to handle requests for data of the database system stored in a distributed storage with one of the plurality of nodes being currently active and the other nodes of the plurality of nodes being currently standby nodes. In various embodiments, the active node receives a request to perform a transaction that includes committing data to the distributed storage. In response to receiving the request, in some embodiments, the active node commits the data to the distributed storage to update the current state of the database system and causes storing of metadata that provides an indication of the commitment in a transaction log stored in the distributed storage.

Abstract: Techniques are disclosed relating to fencing out a first one of a plurality of nodes configured to handle requests for data stored in a distributed storage. A database system, in various embodiments, stores a value indicating that the first node is permitted to update a catalog stored at a metadata server. In response to a determination to prevent the first node from updating the catalog and writing to the distributed storage, in various embodiments, the database system updates the value to indicate that the first node is not permitted to update the catalog and instructs the distributed storage to prevent write operations to a particular portion allocated to the first node for writing data.

Abstract: In some embodiments, a server node may transmit metrics to a client piggy-backed on (appended to) responses to regular read/write requests. Other embodiments may be disclosed and/or claimed.

Abstract: In some embodiments, a server node may transmit metrics to a client piggy-backed on (appended to) responses to regular read/write requests. Other embodiments may be disclosed and/or claimed.

Abstract: Among heterogeneous nodes in a cluster, one node's capabilities can vary widely from that of others in the same cluster, for example, in a cluster of storage servers. As each of the clients/writers to the cluster may contact different server nodes in the cluster over time, with each contact the client/writer may receive server metric information, including both the contacted node's current metrics as well as metrics for any other nodes of which the contacted node is aware. The client/writer may provide updated information to the contacted node regarding the metrics for other nodes of which the client/writer is aware or has contacted. The exchange of metric information may be piggybacked upon read/write requests to each node to conserve bandwidth.

Abstract: Distributed Systems are becoming very large and are being served by 10s of 1000s of individual machines. Among heterogeneous nodes, one node's capabilities can vary widely from that of others in the same cluster, for example, in a cluster of storage servers. Each of the clients/writers may retrieve, periodically, from all the servers, important metrics such as free disk space capacity, average disk IO utilization over the last N minutes, etc. We call these client to nodes (servers) messages as “periodic metrics query.” Using received metrics, clients dynamically assign weights to each of the servers as an indication of their current capabilities. Server metrics and or weights are distributed across the clients, and each client may make independent placement decisions based on their view of the cluster. Because clients are using substantially the same information for making decisions on placement, the result is improved balance across the servers.

Abstract: Techniques are disclosed relating to fencing out a first one of a plurality of nodes configured to handle requests for data stored in a distributed storage. A database system, in various embodiments, stores a value indicating that the first node is permitted to update a catalog stored at a metadata server. In response to a determination to prevent the first node from updating the catalog and writing to the distributed storage, in various embodiments, the database system updates the value to indicate that the first node is not permitted to update the catalog and instructs the distributed storage to prevent write operations to a particular portion allocated to the first node for writing data.

Abstract: Techniques are disclosed relating to a database system synchronizing a current state of the database system among a plurality of nodes configured to handle requests for data of the database system stored in a distributed storage with one of the plurality of nodes being currently active and the other nodes of the plurality of nodes being currently standby nodes. In various embodiments, the active node receives a request to perform a transaction that includes committing data to the distributed storage. In response to receiving the request, in some embodiments, the active node commits the data to the distributed storage to update the current state of the database system and causes storing of metadata that provides an indication of the commitment in a transaction log stored in the distributed storage.

Abstract: Log entries are provided with unique entry identifiers, which may be sequenced in an incremental or decremental order, to create or to update a distributed log which may be replicated and distributed to multiple servers in a network. The entry identifiers may be appended to their respective log entries. Files, records or data which are identified by their respective log entries may be transmitted regardless of the sequence of the entry identifiers.

Abstract: Distributed Systems are becoming very large and are being served by 10s of 1000s of individual machines. Among heterogeneous nodes, one node's capabilities can vary widely from that of others in the same cluster, for example, in a cluster of storage servers. Each of the clients/writers may retrieve, periodically, from all the servers, important metrics such as free disk space capacity, average disk IO utilization over the last N minutes, etc. We call these client to nodes (servers) messages as “periodic metrics query.” Using received metrics, clients dynamically assign weights to each of the servers as an indication of their current capabilities. Server metrics and or weights are distributed across the clients, and each client may make independent placement decisions based on their view of the cluster. Because clients are using substantially the same information for making decisions on placement, the result is improved balance across the servers.

Abstract: Among heterogeneous nodes in a cluster, one node's capabilities can vary widely from that of others in the same cluster, for example, in a cluster of storage servers. As each of the clients/writers to the cluster may contact different server nodes in the cluster over time, with each contact the client/writer may receive server metric information, including both the contacted node's current metrics as well as metrics for any other nodes of which the contacted node is aware. The client/writer may provide updated information to the contacted node regarding the metrics for other nodes of which the client/writer is aware or has contacted. The exchange of metric information may be piggybacked upon read/write requests to each node to conserve bandwidth.