Abstract: Techniques are described for managing access to data storage in a plurality of bitstore nodes. In some situations, a data storage service uses multiple bitstore nodes to store data accessible via a network, such as the Internet. In some situations, multi-level cache systems are employed by each bitstore node and managed, for example to increase throughput of the data storage service.

Abstract: Techniques are described for automatically managing the use of a buffer cache in volatile memory, including in at least some situations to distribute the buffer cache across the volatile memory of a pool of multiple computing devices. Furthermore, the data that is stored in such a distributed buffer cache may in some situations be stored in a redundant manner, such that the stored data may be recovered from a subset of the computing devices in the pool, such as by using erasure coding techniques. In some situations, the described techniques are used to provide a distributed database buffer cache to support operations of a database server program for a database—thus, as database data is retrieved from a non-volatile storage device, that database data may also be redundantly stored across the distributed database buffer cache for later retrieval and use.

Abstract: Techniques are described for managing data storage and retrieval, such as by generating a hash value for a beginning and/or ending portion of a new data group to be stored, attempting to match each generated hash value to a plurality of hash values associated with other existing data groups that are already stored, and comparing the new data group and an identified existing data group to determine any identical extended initial or final portion of the two data groups. The new data group may then be stored as the portion unique to the new data group, along with a reference to the existing identical extended portion of the existing data group. The new stored data group may be later retrieved using the unique portion of the data group, along with the existing extended portions corresponding to other stored data groups for any stored references.

Abstract: Techniques are described for managing data storage and retrieval, such as by generating a hash value for a beginning and/or ending portion of a new data group to be stored, attempting to match each generated hash value to a plurality of hash values associated with other existing data groups that are already stored, and comparing the new data group and an identified existing data group to determine any identical extended initial or final portion of the two data groups. The new data group may then be stored as the portion unique to the new data group, along with a reference to the existing identical extended portion of the existing data group. The new stored data group may be later retrieved using the unique portion of the data group, along with the existing extended portions corresponding to other stored data groups for any stored references.