Abstract: In various embodiments, methods and systems for providing access to file systems are provided. An SFA-based operation having an operation modifier for a file in a file system is received. The SFA-based operation is defined based on a file-representational state transfer (REST) interface. The SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol. The SFA-based operation is executed using the operation modifier. A REST-based operation for the file in the distributed file storage is received. The REST-based operation is defined based on the SFA-REST interface. The operation modifier is referenced in order to execute the REST-based operation. The REST-based operation based on the SFA-REST interface. A schema can be implemented for accessing the file system. The schema includes tables to store files, the table comprises fields corresponding to elements of a SFA-REST interface. A snapshot component can be implemented to backup file shares in the file system.

Abstract: Systems and methods are provided for assigning and associating resources in a cloud computing environment. Virtual machines in the cloud computing environment can be assigned or associated with pools corresponding to users as dedicated, standby, or preemptible machines. The various states provide users with the ability to reserve a desired level of resources while also allowing the operator of the cloud computing environment to increase resource utilization.

Abstract: Presented herein are methods of replicating versioned and hierarchical data structures, as well as data structures representing complex transactions. Due to interdependencies between data entities and a lack of guaranteed message ordering, simple replication methods employed for simple data types cannot be used. Operations on data structures exhibit dependencies between the messages making up the operations. This strategy can be extended to various types of complex transactions by considering certain messages to depend on other messages or on the existence of other entries at the data store. Regardless of origin, these dependencies can be enforced by suspending the processing of messages with unsatisfied dependencies until all of its dependencies have been met. Alternately, transactions can be committed immediately, creating entities that include versioned identifiers for each of their dependencies. These entities can then be garbage collected of the parent objects are not subsequently created.

Abstract: Systems and methods are provided for assigning and associating resources in a cloud computing environment. Virtual machines in the cloud computing environment can be assigned or associated with pools corresponding to users as dedicated, standby, or preemptible machines. The various states provide users with the ability to reserve a desired level of resources while also allowing the operator of the cloud computing environment to increase resource utilization.

Abstract: Embodiments of the present invention relate to systems, methods, and computer storage media for geographically organizing the storage of data and hosted services in a distributed computing environment. An allocation constraint is utilized to identify and select a geo-region in which an affinity group that is comprised of a hosted service and/or data is to be physically stored. Additionally, a geo-domain within the geo-region is identified for hosting the hosted service and/or data of the affinity group. A stamp, such as a storage stamp or a computation stamp, is identified on the geo-domain for storing the hosted service and/or the data. The hosted service is stored in an identified computation stamp and the data is stored in an identified storage stamp. In an additional exemplary embodiment, the identification of a geo-domain involves analyzing resource information related to potential geo-domains within a geo-region.

Abstract: Systems and methods are provided that enable a general framework for partitioning application-defined computations (e.g., jobs) in a scalable environment. The general framework decouples partitioning of a computation from the other aspects of the computation. As a result, the effort required to define an application-defined job is reduced or minimized, as the user is not required to provide a partitioning algorithm. A user can optionally take advantage of a partitioning framework by providing application-defined interfaces to perform the desired job. Optionally, a user can provide additional information to allow for modification of how partitions are assigned.

Abstract: Partition management for a scalable, structured storage system is provided. The storage system provides storage represented by one or more tables, each of which includes rows that represent data entities. A table is partitioned into a number of partitions, each partition including a contiguous range of rows. The partitions are served by table servers and managed by a table master. Load distribution information for the table servers and partitions is tracked, and the table master determines to split and/or merge partitions based on the load distribution information.

Abstract: Systems and methods are provided that enable a general framework for partitioning application-defined computations (e.g., jobs) in a scalable environment. The general framework decouples partitioning of a computation from the other aspects of the computation. As a result, the effort required to define an application-defined job is reduced or minimized, as the user is not required to provide a partitioning algorithm. A user can optionally take advantage of a partitioning framework by providing application-defined interfaces to perform the desired job. Optionally, a user can provide additional information to allow for modification of how partitions are assigned.

Abstract: In various embodiments, methods and systems for implementing a distributed metadata management system in distributed storage systems are provided. A distributed storage system operates based on data storage resources (e.g., extents and streams). The distributed metadata management system is implemented for extent and stream metadata to facilitate the scalability of metadata processing. The distributed storage system implements extent managers and stream managers that independently manage extent and stream metadata, respectively. The extent managers are associated with an extent table that stores extent metadata. The stream managers are associated with streams that store associations with extents. The distributed metadata management system can also utilize a bootstrap layer that leverages components of a legacy distributed storage system to facilitate distributed management of extent and stream metadata.

Abstract: In various embodiments, methods and systems for implementing garbage collection in distributed storage systems are provided. The distributed storage system operates based on independent management of metadata of extent and stream data storage resources. A hybrid garbage collection system based on reference counting garbage collection operations and mark-and-sweep garbage collection operations is implemented. An extent lifetime table that tracks reference weights and mark sequences for extents is initialized and updated based on indications from extent managers and stream managers, respectively. Upon determining that an extent is to be handed-off from weighted reference counting garbage collection operations to mark-and-sweep garbage collection operations, a reference weight field for the extent is voided and a mark sequence field of the extent is updated. The mark sequence field is updated with a latest global sequence number.

Abstract: In various embodiments, methods and systems for integrated resource allocation and loading balancing are provided. A global resource allocator receives usage information of resources in a cloud computing system. The usage information is associated with a plurality of accounts and consumer operations pairs on servers of the cloud computing system. For selected account and consumer operation pairs associated with a particular resource, allocation targets are determined and communicated to the corresponding server of the selected account and consumer operation pairs. The servers use the resource based on the allocation targets. A load balancer receives the usage information the resource and the allocation targets. The allocation targets indicate a load by the selected account and consumer operation pairs on their corresponding servers. The load balancer performs a load balancing operation to locate a server with a capacity to process the allocated target of the selected account and consumer operation pairs.

Abstract: Cloud computing platforms having computer-readable media that perform methods for facilitating communications with storage. A request having a first-interface format to access storage is intercepted. The first interface format of the request supports access to a virtual hard drive (VHD). The request is translated to a blob request having a blob interface format. The blob interface format of the blob request supports access to a plurality of blobs of data in a blob store. The blob request is communicated to a blob interface such that the blob request is executed in managing the plurality of blobs.

Abstract: Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.

Abstract: Cloud computing platforms having computer-readable media that perform methods for facilitating communications with storage. A request having a first-interface format to access storage is intercepted. The first interface format of the request supports access to a virtual hard drive (VHD). The request is translated to a blob request having a blob interface format. The blob interface format of the blob request supports access to a plurality of blobs of data in a blob store. The blob request is communicated to a blob interface such that the blob request is executed in managing the plurality of blobs.

Abstract: A method includes determining that a trigger condition of a triggered optimization module of a plurality of optimization modules is met and optimizing scalable storage based on an optimization routine. The optimization routine includes providing a plurality of candidate operations and for a selected optimization module of the plurality of optimization modules that has a higher priority than the triggered optimization module, removing a candidate operation from the plurality of candidate operations that would diminish a modeled state of the scalable storage for the selected optimization module. The optimization routine also includes determining at least one operation of the plurality of candidate operations that would improve the modeled state of the scalable storage for the triggered optimization module and updating the modeled state of the scalable storage to model executing the at least one operation. The method further includes executing the at least one operation.

Abstract: Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.

Abstract: One method includes receiving partitions, generating assignment plans for assignment of the partitions to servers based on dimensional values of dimensions as determined by different assignment heuristics, selecting one of the assignment plans for execution based on analyzing the dimensional values in accordance with the assignment plans, and executing the selected assignment plan on scalable storage. Another method includes determining, for a triggered optimization module that a server is over utilized on a dimension, selecting candidate operations for partitions assigned to the server, for a higher priority optimization module than the triggered optimization module, removing a candidate operation from the candidate operations that would diminish a modeled state of scalable storage, determining an operation of the candidate operations that would improve the modeled state of the scalable storage with respect to a metric of the dimension on the server, and executing the operation on the scalable storage.

Abstract: Systems, methods, and computer storage media for upgrading a domain in a distributed computing environment are provided. Upgrading of the domain includes preparing for the upgrade, upgrading, and finalizing the upgrade. The preparation of the domain includes ensuring predefined quantities of role instances are available in domains other than the upgrade domain. The preparation also includes ensuring that a predefined number of extent replicas are available in domains other than the upgrade domain. The preparation may also include checkpointing partitions within the upgrade domain to facilitate faster loading once transferred to a domain other than the upgrade domain. The finalization may include allowing nodes within the upgrade domain to resume functionality that was suspended during the upgrade.

Abstract: In various embodiments, systems and methods are presented for providing resources by way of a platform as a service in a distributed computing environment to perform a job. The system may be comprised of a number of components, such as a task machine, a task location service machine, and a high-level location service machines that in combination are useable to accomplish functions provided herein. It is contemplated that the system performs methods for providing resources by determining resources of the system, such as virtual machines, and applying auto-scaling rules to the system to scale those resources. Based on the determination of the auto-scaling rules, the resources may be allocated to achieve a desired result.

Abstract: Embodiments of the present invention relate to asynchronously replicating data in a distributed computing environment. To achieve asynchronous replication, data received at a primary data store may be annotated with information, such as an identifier of the data. The annotated data may then be communicated to a secondary data store, which may then write the data and annotated information to one or more logs for eventual replay and committal at the secondary data store. The primary data store may communicate an acknowledgment of success in committing the data at the primary data store as well as of success in writing the data to the secondary data store. Additional embodiments may include committing the data at the secondary data store in response to receiving an instruction that authorizes committal of data through an identifier.