Abstract: The present invention relates to methods for processing jobs within a cluster architecture. One method comprises the pausing of a job when waiting upon external dependencies. Another method comprises the transmission of messages relating to the ongoing processing of jobs back to a client via a persistent messaging channel. Yet another method comprises determining capacity at a node before allocating a job for processing by the node or adding the job to a cluster queue. A system for processing jobs within a cluster architecture is also disclosed.