Abstract: Apparatus and method for performing real-time monitoring of server-side resources required to satisfy a client-side request in a distributed data storage environment, such as in a cloud computing or HPC (high performance computing) network. A client device is configured to issue a service request to carry out a service application associated with one or more server nodes. A request scheduler forwards the service request from the client device to a selected server node associated with the service request. A service log accumulates entries associated with data transfer operations carried out by the server node responsive to the service request over each of a succession of time periods. A service monitor accumulates, for each of the succession of time periods, information associated with the data transfer operations. A monitor tool aggregates the cumulative information to provide an indication of server-side resources utilized to satisfy the service request.

Abstract: Apparatus and method for performing real-time monitoring of server-side resources required to satisfy a client-side request in a distributed data storage environment, such as in a cloud computing or HPC (high performance computing) network. A client device is configured to issue a service request to carry out a service application associated with one or more server nodes. A request scheduler forwards the service request from the client device to a selected server node associated with the service request. A service log accumulates entries associated with data transfer operations carried out by the server node responsive to the service request over each of a succession of time periods. A service monitor accumulates, for each of the succession of time periods, information associated with the data transfer operations. A monitor tool aggregates the cumulative information to provide an indication of server-side resources utilized to satisfy the service request.

Abstract: Methods are presented suitable for clustering computational event logs (2) including a method for calculating a metric distance between characters of different event messages (4) by comparing both characters to a comparative set of characters. Methods are presented for calculating a metric distance between two event messages (4) comprising determining character metric distances between characters in the compared words and word metric distances between the words in the compared events (4), Methods are presented for creating an area (8) in metric space corresponding to a new cluster (6) when a further event message (26) is found in an overlap region (24) of existing clusters (6, 8). Methods are presented in populating and constructing an event table.

Abstract: Apparatus and method for scheduling requests for data transfers in a multi-device storage system. In some embodiments, a system includes at least one server coupled to a pool of storage devices to transfer data from the storage devices to client devices responsive to requests. A request scheduler is adapted to receive into a memory a plurality of requests each having a service identifier (ID) and a payload size, to set a deadline for each request responsive to the service ID and the payload size, to forward the requests to the server for processing in an order based on service ID and, responsive to the deadline being reached for a selected request, to advance the selected request for immediate processing by the server.

Abstract: A method of scheduling requests from various services to a data storage resource, includes receiving service requests, the service requests including metadata specifying a service ID and a data size of payload data associated with the request, at least some of the service IDs having service throughput metadata specifying a required service throughput associated therewith; arranging the requests into FIFO throttled queues based on the service ID; setting a deadline for processing of a request in a throttled queue, the deadline selected in dependence upon the size of the request and the required service throughput associated therewith; providing a time credit value for each throttled queue, the time credit value including an accumulated value of the time by which a deadline for that queue has been missed; comparing the time credit value of a throttled queue to the time required to service the next request in that throttled queue.

Abstract: Apparatus and method for scheduling requests for data transfers in a multi-device storage system. In some embodiments, a system includes at least one server coupled to a pool of storage devices to transfer data from the storage devices to client devices responsive to requests. A request scheduler is adapted to receive into a memory a plurality of requests each having a service identifier (ID) and a payload size, to set a deadline for each request responsive to the service ID and the payload size, to forward the requests to the server for processing in an order based on service ID and, responsive to the deadline being reached for a selected request, to advance the selected request for immediate processing by the server.

Abstract: There is provided a method of scheduling requests from a plurality of services to at least one data storage resource. The method comprises receiving, on a computer system, service requests from said plurality of services. The service requests comprise metadata specifying a service ID and a data size of payload data associated with said service request, and at least some of said service IDs have service throughput metadata specifying a required service throughput associated therewith. The method further includes arranging, in a computer system, said requests into FIFO throttled queues based on said service ID and then setting a deadline for processing of a request in a throttled queue. The deadline is selected in dependence upon the size of the request and the required service throughput associated therewith. Then, the deadline of each throttled queue is monitored and, if a request in a throttled queue has reached or exceeded the deadline the request is processed in a data storage resource.

Abstract: Methods are presented suitable for clustering computational event logs (2) including a method for calculating a metric distance between characters of different event messages (4) by comparing both characters to a comparative set of characters. Methods are presented for calculating a metric distance between two event messages (4) comprising determining character metric distances between characters in the compared words and word metric distances between the words in the compared events (4), Methods are presented for creating an area (8) in metric space corresponding to a new cluster (6) when a further event message (26) is found in an overlap region (24) of existing clusters (6, 8). Methods are presented in populating and constructing an event table.

Abstract: A method of scheduling requests from various services to a data storage resource, includes receiving service requests, the service requests including metadata specifying a service ID and a data size of payload data associated with the request, at least some of the service IDs having service throughput metadata specifying a required service throughput associated therewith; arranging the requests into FIFO throttled queues based on the service ID; setting a deadline for processing of a request in a throttled queue, the deadline selected in dependence upon the size of the request and the required service throughput associated therewith; providing a time credit value for each throttled queue, the time credit value including an accumulated value of the time by which a deadline for that queue has been missed; comparing the time credit value of a throttled queue to the time required to service the next request in that throttled queue.

Abstract: There is provided a method of scheduling requests from a plurality of services to at least one data storage resource. The method comprises receiving, on a computer system, service requests from said plurality of services. The service requests comprise metadata specifying a service ID and a data size of payload data associated with said service request, and at least some of said service IDs have service throughput metadata specifying a required service throughput associated therewith. The method further includes arranging, in a computer system, said requests into FIFO throttled queues based on said service ID and then setting a deadline for processing of a request in a throttled queue. The deadline is selected in dependence upon the size of the request and the required service throughput associated therewith. Then, the deadline of each throttled queue is monitored and, if a request in a throttled queue has reached or exceeded the deadline the request is processed in a data storage resource.