Abstract: Processing a file system operation is disclosed. An indication of a desired operation of a distributed file system is received. A metadata node for the desired operation is identified. It is indicated to the identified metadata node to process the desired operation. In the event the identified metadata node becomes not fully functional before the processing by the identified metadata node is confirmed, the distributed file system is analyzed to determine whether to indicate again to process the desired operation.

Abstract: Switching an active metadata node is disclosed. An indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system is received. The standby metadata node is included in a server. A request that indicates that the standby metadata node would like to become an exclusive metadata node writer of a transaction log is sent. A confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log is received. Based at least in part on the confirmation, an update that the standby metadata node has become the new active metadata node of the distributed file system is provided.

Abstract: A system and method of managing a cluster of distributed machines is described. A cluster manager receives status updates regarding tasks running on each machine in the cluster from a task tracker running on the machine. The cluster manager receives resource requests from a job tracker created by a client wishing to run a job in the cluster. The cluster manager is responsible for implementing push-based fair scheduling of resources to the job trackers. The job tracker is responsible for running tasks for one job in the resource identified by the cluster manager. In one embodiment, the job tracker can run in the client for small jobs and in the cluster for larger jobs. The cluster manager can also be restarted, for example, for software updates without restraining the cluster.

Abstract: A system and method of managing a cluster of distributed machines is described. A cluster manager receives status updates regarding tasks running on each machine in the cluster from a task tracker running on the machine. The cluster manager receives resource requests from a job tracker created by a client wishing to run a job in the cluster. The cluster manager is responsible for implementing push-based fair scheduling of resources to the job trackers. The job tracker is responsible for running tasks for one job in the resource identified by the cluster manager. In one embodiment, the job tracker can run in the client for small jobs and in the cluster for larger jobs. The cluster manager can also be restarted, for example, for software updates without restraining the cluster.

Abstract: Switching an active metadata node is disclosed. An indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system is received. The standby metadata node is included in a server. A request that indicates that the standby metadata node would like to become an exclusive metadata node writer of a transaction log is sent. A confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log is received. Based at least in part on the confirmation, an update that the standby metadata node has become the new active metadata node of the distributed file system is provided.

Abstract: Processing a file system operation is disclosed. An indication of a desired operation of a distributed file system is received. A metadata node for the desired operation is identified. It is indicated to the identified metadata node to process the desired operation. In the event the identified metadata node becomes not fully functional before the processing by the identified metadata node is confirmed, the distributed file system is analyzed to determine whether to indicate again to process the desired operation.