Abstract: A method for storing time-based data streams in a high-capacity network is provided. A time-based data storage request is received from an application. The data storage request is associated with one or more data streams. Each of the data streams includes a plurality of time-ordered items having a header. The header includes two or more timestamps representing a time interval associated with each of the plurality of time-ordered items. The received data storage request is processed to accumulate time-ordered data items in a plurality of data files and to identify time-based information and corresponding virtual offset information related to the accumulated data items. The identified time-based information and the virtual offset information related to the accumulated data items are stored in a data repository. A determination is made whether adjustment is necessary for sorting the stored data in the repository. The sorting adjustment is selectively performed responsive to the determination.

Abstract: A data storage system includes storage nodes adapted to provide retrieval of time-based data in response to a receipt of a time-based data retrieval request associated with data streams. Each of the data streams includes time-ordered data items having a header. The header includes timestamps representing a time interval associated with the data items. The data storage system further includes applications issuing the time-based data retrieval requests associated with the data streams and session managers coupled to the applications and storage nodes. Shared resources are dynamically allocated between the applications submitting the requests. Data requested by the requests is retrieved from the storage nodes using timestamp based indexes associated with the data streams in a substantially parallel manner. The retrieved data is merge-sorted by time. The sorted data is transmitted to the applications based on a delivery rate controlled by each application.

Abstract: A data storage system includes storage nodes adapted to provide retrieval of time-based data in response to a receipt of a time-based data retrieval request associated with data streams. Each of the data streams includes time-ordered data items having a header. The header includes timestamps representing a time interval associated with the data items. The data storage system further includes applications issuing the time-based data retrieval requests associated with the data streams and session managers coupled to the applications and storage nodes. Shared resources are dynamically allocated between the applications submitting the requests. Data requested by the requests is retrieved from the storage nodes using timestamp based indexes associated with the data streams in a substantially parallel manner. The retrieved data is merge-sorted by time. The sorted data is transmitted to the applications based on a delivery rate controlled by each application.

Abstract: A method for storing time-based data streams in a high-capacity network is provided. A time-based data storage request is received from an application. The data storage request is associated with one or more data streams. Each of the data streams includes a plurality of time-ordered items having a header. The header includes two or more timestamps representing a time interval associated with each of the plurality of time-ordered items. The received data storage request is processed to accumulate time-ordered data items in a plurality of data files and to identify time-based information and corresponding virtual offset information related to the accumulated data items. The identified time-based information and the virtual offset information related to the accumulated data items are stored in a data repository. A determination is made whether adjustment is necessary for sorting the stored data in the repository. The sorting adjustment is selectively performed responsive to the determination.