Abstract: Embodiments are directed to a file system engine that provides a file system that includes a plurality of blocks on a file storage tier with portion of the blocks associated with a cache storage tier. A cache engine provides heat extents that include a heat score and a run length value such that the heat extents are represent activities associated with the blocks. Heat extents associated with file system activities are determined based on a position of the blocks that corresponds with the run length values of the heat extents and sectors on the file storage tier. Heat scores associated with the heat extents may be modified based on the activities and distribution models to conform the heat extents with the distribution models. Blocks associated with heat scores that are less than a threshold value provided by the distribution models may be disassociated from the cache storage tier.

Abstract: Embodiments are directed to a file system engine that provides a file system that includes a plurality of blocks on a file storage tier with portion of the blocks associated with a cache storage tier. A cache engine provides heat extents that include a heat score and a run length value such that the heat extents are represent activities associated with the blocks. Heat extents associated with file system activities are determined based on a position of the blocks that corresponds with the run length values of the heat extents and sectors on the file storage tier. Heat scores associated with the heat extents may be modified based on the activities and distribution models to conform the heat extents with the distribution models. Blocks associated with heat scores that are less than a threshold value provided by the distribution models may be disassociated from the cache storage tier.

Abstract: Embodiments are directed to managing data in a file system. A pre-fetch engine may receive requests from a client the file system that includes a pre-fetch storage tier and a file storage tier of storage devices. The pre-fetch engine determines a pre-fetch policy based on the requests such that the pre-fetch policy determines which blocks to copy to the pre-fetch storage tier. And, the pre-fetch policy may be associated with a score model that includes score rules where one of the rules may be associated with a client score. The pre-fetch engine may obtain scores associated with the score rules such that the scores are based on previous requests made by the client. In response to scores exceeding a threshold value, the pre-fetch engine may copy the blocks to the pre-fetch storage tier. The pre-fetch engine may update the scores based on the performance of the pre-fetch policy.

Abstract: Embodiments are directed to a file system engine that provides a file system that includes a plurality of blocks on a file storage tier with portion of the blocks associated with a cache storage tier. A cache engine provides heat extents that include a heat score and a run length value such that the heat extents are represent activities associated with the blocks. Heat extents associated with file system activities are determined based on a position of the blocks that corresponds with the run length values of the heat extents and sectors on the file storage tier. Heat scores associated with the heat extents may be modified based on the activities and distribution models to conform the heat extents with the distribution models. Blocks associated with heat scores that are less than a threshold value provided by the distribution models may be disassociated from the cache storage tier.

Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.

Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.

Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.

Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.

Abstract: A facility that for a multithreaded program executing on a root machine causes the threads of the program to be executed in a relative scheduling that produces an interesting result. The facility suspends execution of the program. The facility then tests a plurality of relative thread schedulings on one or more virtual machines and observes the result. Based upon the observed result the facility selects one of the tested relative thread schedulings. The facility then resumes execution of the program using the selected relative thread scheduling.

Abstract: A facility for supporting the analysis of a multithreaded program is described. For each of a number of threads of the multithreaded program, the facility identifies a semantically meaningful point in the execution of the thread. The facility interrupts the execution of each thread at the point identified for the thread.

Abstract: A facility that for a multithreaded program executing on a root machine causes the threads of the program to be executed in a relative scheduling that produces an interesting result. The facility suspends execution of the program. The facility then tests a plurality of relative thread schedulings on one or more virtual machines and observes the result. Based upon the observed result the facility selects one of the tested relative thread schedulings. The facility then resumes execution of the program using the selected relative thread scheduling.

Abstract: Embodiments of the invention relate generally to incremental computing. Specifically, embodiments of the invention include systems and methods for the concurrent processing of multiple, incremental changes to a data value while at the same time monitoring and/or enforcing threshold values for that data value. Embodiments of the invention also include systems and methods of managing utilization of a resource of a computer system having a number of threads.

Abstract: A program code data structure is described. The data structure contains substantive code of a multithreaded computer program that includes a plurality of communicating instructions that effect communication between threads. The data structure further contains, at each of one or more points in the substantive code, an indication that, when the computer program is being executed, one or more communicating instructions that are in a selected relationship with the indication should be executed by any thread that reaches the point in a particular order relative to execution of one or more other communicating instructions by other threads. The contents of the data structure may be used to determine an order in which to execute communicating instructions.

Abstract: A facility for identifying functionally distinct memory access reorderings for a multithreaded program is described. The facility monitors execution of the program to detect, for each of one or more memory locations, an order in which the memory location was accessed by the threads of the program, each access being at least one of a read access and a write access. Among a number of possible memory access reorderings of a read access by a reading thread to a location and a write access by a writing thread to the same location where the write access preceded the read access, the facility identifies as functionally distinct memory access reorderings those possible memory access reorderings where the reading thread could have become newly aware of changed state of the writing thread as a result of the indicated read access.

Abstract: A hardware and/or software facility for executing a multithreaded program is described. The facility causes each of a plurality of machines to execute the multithreaded program deterministically, such that the deterministic execution of the multithreaded program is replaced across the plurality of machines. The facility detects a problem in the execution of the multithreaded Program by one of the plurality of machines. In response, the facility adjusts the execution of the multithreaded program by at least one of the machines of the plurality.

Abstract: In one embodiment, systems and methods are provided for nonlinear journaling. In one embodiment, groups of data designated for storage in a data storage unit are journaled into persistent storage. In one embodiment, the journal data is recorded nonlinearly. In one embodiment, a linked data structure records data and data descriptors in persistent storage.

Abstract: In one embodiment, systems and methods are provided for nonlinear journaling. In one embodiment, groups of data designated for storage in a data storage unit are journaled into persistent storage. In one embodiment, the journal data is recorded nonlinearly. In one embodiment, a linked data structure records data and data descriptors in persistent storage.

Abstract: In some embodiments, storage devices, such as a storage drive or a storage node, in an array of storage devices may be reintroduced into the array of storage devices after a period of temporary unavailability without fully rebuilding the entire previously unavailable storage device.

Abstract: Systems and methods are provided for passing control of message dependencies and ordering to one or more network nodes participating in a transaction. Thus, messages can be sent to the participating nodes and the participating nodes can handle the messages according to their respective dependencies without first sending a response or waiting for the next message to arrive. In one embodiment, at least one message is flagged with a checkpoint to indicate that messages sent before the checkpoint are to be handled by the participating nodes before messages after the checkpoint. Messages may also be flagged with error-handling information. For example, a message may be marked to indicate that if the message fails, no further messages are to be sent to the node had the failure. Responses to a group of messages can be collected in a single operation.

Abstract: Systems and methods are provided for scanning files and directories in a distributed file system on a network of nodes. The nodes include metadata with attribute information corresponding to files and directories distributed on the nodes. In one embodiment, the files and directories are scanned by commanding the nodes to search their respective metadata for a selected attribute. At least two of the nodes are capable of searching their respective metadata in parallel. In one embodiment, the distributed file system commands the nodes to search for metadata data structures having location information corresponding to a failed device on the network. The metadata data structures identified in the search may then be used to reconstruct lost data that was stored on the failed device.