Abstract: Methods, systems, and computer-readable media for a client-side filesystem for a remote repository are disclosed. One or more files of a repository are sent from a storage service to a client device. The file(s) are obtained by the client using a credential sent by a repository manager. Local copies of the file(s) are accessible via a local filesystem mounted at the client device. One or more new files associated with the repository are generated at the client device. Using the credential, the one or more new files are obtained at the storage service from the client device. The one or more new files are added to the repository.

Abstract: Resource data objects describing resources in a system may be maintained in multiple different hierarchies for applying policies to manage the resources. Lookup requests may access the different hierarchies to determine which policies are applicable to a given resource based on the policies identified in each of the hierarchies. Modifications to hierarchies may be performed in isolation so that the application of policies in other hierarchies is unchanged by modifications to a different hierarchy. Access restrictions may be enforced with respect to hierarchies so that different users may be permitted access to different hierarchies for system resource management.

Abstract: Multiple edits to a hierarchical data structure may be atomically applied. A request to perform modifications with respect to a portion or the entire hierarchical data structure may be received. A copy of the requested portion of the hierarchical data structure may be created separate from the hierarchical data structure. The portion of the hierarchical data structure may remain available for read access. Modifications may be applied to the copy of the portion of the hierarchical data structure. In response to a request to commit the modifications to the portion of the hierarchical data structure, the copy of the portion of the hierarchical data structure may atomically replace the portion of the hierarchical data structure.

Abstract: Methods, systems, and computer-readable media for generating message envelopes for heterogeneous events are disclosed. A plurality of events are received. The events are associated with computing resources in a multi-tenant environment. The events comprise an event formatted using a content schema and a different event formatted using a different content schema. A message is generated comprising a message envelope and the event. The message envelope comprises a plurality of field names and one or more field values formatted using an envelope schema. A different message is generated comprising a different message envelope and the different event. The different message envelope comprises the field names and one or more different field values formatted using the envelope schema. The message is filtered based at least in part on the message envelope, and the different message is filtered based at least in part on the different message envelope.

Abstract: Resource data objects describing resources in a system may be maintained in multiple different hierarchies for applying policies to manage the resources. Lookup requests may access the different hierarchies to determine which policies are applicable to a given resource based on the policies identified in each of the hierarchies. Modifications to hierarchies may be performed in isolation so that the application of policies in other hierarchies is unchanged by modifications to a different hierarchy. Access restrictions may be enforced with respect to hierarchies so that different users may be permitted access to different hierarchies for system resource management.

Abstract: A replication chain comprising one or more replication nodes of a multi-tenant stream management system is assigned to store data records of a partition of a particular data stream. A data record of the partition is received at a selected replication node of the replication chain. In a sequential order, a respective replica of the data record is stored at each replication node of the chain. An acknowledgement of a successful storage of the data record is provided after the replications are completed.

Abstract: Multi-party updates may be performed for distributed systems. An agreement request may be received that proposes updates to a distributed system. An authorization scheme for the agreement request may be determined and approvers for the proposed updates identified according to the authorization scheme. Notifications may be provided to the approvers indicating the proposed updates to the distributed system. Responses from the approvers may be evaluated to determine whether the authorization scheme is satisfied for the proposed updates. If the authorizations scheme is satisfied, then the proposed updates may be performed to the distributed system.

Abstract: Virtual resource provisioning may be enhanced by coloring virtual resource instances and/or underlying implementation resources. Particular resource colors may be associated with particular treatments during allocation of implementation resources to virtual resources. There may be different types of colors corresponding to different types of allocation treatment. Exclusory colors may be utilized to reduce clustering of virtual resources with respect to implementation resources. Assignment of exclusory colors to virtual resources can help strike a balance between lower costs through efficient implementation resource utilization and higher fault tolerance through spreading across an available implementation resource pool. Inclusive colors may be utilized to require and/or prefer allocation of virtual resources to implementation resources painted with the inclusive color. Proximity colors may be utilized to enhance a computational performance of a set of virtual resources.

Abstract: A computer system, such as a data storage system, implements techniques for segregating and controlling access to data stored in multiple regions. In some embodiments, redundancy coded shards generated from the data and stored in durable storage of a data storage system is allocated across multiple regions, but in a fashion that prevents actors with access to regions outside that of a “home” region from recovering a sufficient number of unique shards to regenerate the data represented thereby. In some embodiments, encryption is used to segregate the data by encrypting the generated shards, then storing the cryptographic information on or otherwise controlling access on hosts or other devices of only the home region.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: A computer system, such as a data storage system, implements techniques for segregating and controlling access to data stored in multiple regions. In some embodiments, redundancy coded shards generated from the data and stored in durable storage of a data storage system is allocated across multiple regions, but in a fashion that prevents actors with access to regions outside that of a “home” region from recovering a sufficient number of unique shards to regenerate the data represented thereby. In some embodiments, encryption is used to segregate the data by encrypting the generated shards, then storing the cryptographic information on or otherwise controlling access on hosts or other devices of only the home region.

Abstract: Multi-party updates may be performed for distributed systems. An agreement request may be received that proposes updates to a distributed system. An authorization scheme for the agreement request may be determined and approvers for the proposed updates identified according to the authorization scheme. Notifications may be provided to the approvers indicating the proposed updates to the distributed system. Responses from the approvers may be evaluated to determine whether the authorization scheme is satisfied for the proposed updates. If the authorizations scheme is satisfied, then the proposed updates may be performed to the distributed system.

Abstract: Multiple edits to a hierarchical data structure may be atomically applied. A request to perform modifications with respect to a portion or the entire hierarchical data structure may be received. A copy of the requested portion of the hierarchical data structure may be created separate from the hierarchical data structure. The portion of the hierarchical data structure may remain available for read access. Modifications may be applied to the copy of the portion of the hierarchical data structure.

Abstract: Distributed system resources may be managed by applying user created policies to the resources. To ensure that valid policies are applied, remote validation for the policies may be implemented. A validation event for a policy may be detected. A remote validation agent may be identified for the policy and a validation request sent to the remote validation agent that includes information for validating the policy. The remote validation agent may return a validation result for the policy. If valid, a policy action that triggered the remote validation event for the policy may be allowed. If invalid, the policy action that triggered the remote validation event for the policy may be denied.

Abstract: Resource data objects describing resources in a system may be maintained in multiple different hierarchies for applying policies to manage the resources. Lookup requests may access the different hierarchies to determine which policies are applicable to a given resource based on the policies identified in each of the hierarchies. Modifications to hierarchies may be performed in isolation so that the application of policies in other hierarchies is unchanged by modifications to a different hierarchy. Access restrictions may be enforced with respect to hierarchies so that different users may be permitted access to different hierarchies for system resource management.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: A distributed data store may maintain versioned hierarchical data structures. Different versions of a hierarchical data structure may be maintained consistent with a transaction log for the hierarchical data structure. When access requests directed to the hierarchical data structure are received, a version of the hierarchical data structure may be identified for processing an access request. For access requests with snapshot isolation, the identified version alone may be sufficient to consistently process the access request. For access requests with higher isolation requirements, such as serializable isolation, transactions based on the access request may be submitted to the transaction log so that access requests resulting in committed transactions may be allowed, whereas access requests resulting in conflicting transactions may be denied.

Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.

Abstract: A replication chain comprising one or more replication nodes of a multi-tenant stream management system is assigned to store data records of a partition of a particular data stream. A data record of the partition is received at a selected replication node of the replication chain. In a sequential order, a respective replica of the data record is stored at each replication node of the chain. An acknowledgement of a successful storage of the data record is provided after the replications are completed.