Abstract: A system, a method and a computer readable medium for storing data elements and related data provenance information. The data elements may be represented in a hyper-table having rows and columns which may be indexed. The data-values of the corresponding data-elements in the hyper-cells may be retrieved based on the indices. Snapshots of the indices may be generated at pre-determined time periods. Checkpoints of the hyper-table may be generated at time periods that are based on transactions on the hyper-table. The hyper-table is capable of being queried as the hyper-table existed at certain time-periods, and data-values of the data-elements may be retrieved as the data-elements existed at such time-periods.

Abstract: A system, method and computer readable medium for indexing streaming data. Data may be received from distributed devices connected via a network. Data elements may be stored and allocated to data blocks and events of the block stores. Non-text data may be converted into a text representation. The data may be split into terms, and term frequencies of each term within each of the event may be calculated. Block-level term frequency statics may be calculated based on the term frequencies. Tree index structures, such as the Y-tree index, may be generated based on the block-level term frequency data. The Y-tree index structures may use the terms as keys and pointers to the corresponding data blocks and block-level term frequency data. A search query may be performed over the tree index structures.

Abstract: A data classification system comprising a server adapted to receive data elements from a computer network, compute a distance between the elements and a plurality of representative elements such that where the distance is less than a threshold, the input elements are associated with the representative element and where the distance is not less than the threshold, the input element is stored as a new representative element. A method of classifying network traffic is further disclosed that can include providing a server adapted to receive data from a network, compute a distance between the data elements and representative data elements such that where the distance is less than a threshold, the input elements are associated with the representative element and where the distance is not less than the threshold, the input element is stored as a new representative element.

Abstract: Systems and methods for storing and retrieving data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data-elements may be allocated to data-blocks of a block-store that have assigned block-identifiers and further allocated to events of the data-blocks that have assigned token-names. Stream-schema of the received data-streams may comprise a list of token-names and an index-definition for each corresponding data-stream. Indices may be generated for the event-allocated data-elements. A query may be executed in order to retrieve data-elements of the received data-streams based on the indices.

Abstract: A system and method for storing data-files stored on distributed devices connected to a network. Data-elements of the data-files are allocated to data-blocks stored on the distributed devices. Key-identifiers are calculated for each of the data-blocks based on the allocated data-elements. The key-identifiers are stored in distributed tables stored on the distributed devices. Index-nodes are generated for the data-files based on the data-blocks. A Paxos algorithm is executed for the index-nodes based on the key-identifiers to provide a consensus of the data-files that are stored on the plurality of distributed devices.

Abstract: A system and method for storing data-files stored on distributed devices connected to a network. Data-elements of the data-files are allocated to data-blocks stored on the distributed devices. Key-identifiers are calculated for each of the data-blocks based on the allocated data-elements. The key-identifiers are stored in distributed tables stored on the distributed devices. Index-nodes are generated for the data-files based on the data-blocks. A Paxos algorithm is executed for the index-nodes based on the key-identifiers to provide a consensus of the data-files that are stored on the plurality of distributed devices.

Abstract: Systems and methods for storing and retrieving data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data-elements may be allocated to data-blocks of a block-store that have assigned block-identifiers and further allocated to events of the data-blocks that have assigned token-names. Stream-schema of the received data-streams may comprise a list of token-names and an index-definition for each corresponding data-stream. Indices may be generated for the event-allocated data-elements. A query may be executed in order to retrieve data-elements of the received data-streams based on the indices.

Abstract: Systems, methods and computer readable medium for storing data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data elements may be allocated to data blocks of a block store that have assigned block identifiers and further allocated to events of the data blocks. Stream schema of the received plurality of data streams may have the same stream schema, and indices may be generated indices based on the order of the event allocated data elements. Stream schema of the received data streams may comprise a list of token names. Token names may be assigned to the event allocated data elements. Indices may be generated for the event allocated data elements based on the stream schema.

Abstract: A system and method for storing data-files stored on distributed devices connected to a network. Data-elements of the data-files are allocated to data-blocks stored on the distributed devices. Key-identifiers are calculated for each of the data-blocks based on the allocated data-elements. The key-identifiers are stored in distributed tables stored on the distributed devices. Index-nodes are generated for the data-files based on the data-blocks. A Paxos algorithm is executed for the index-nodes based on the key-identifiers to provide a consensus of the data-files that are stored on the plurality of distributed devices.

Abstract: Systems and methods for storing and retrieving data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data-elements may be allocated to data-blocks of a block-store that have assigned block-identifiers and further allocated to events of the data-blocks that have assigned token-names. Stream-schema of the received data-streams may comprise a list of token-names and an index-definition for each corresponding data-stream. Indices may be generated for the event-allocated data-elements. A query may be executed in order to retrieve data-elements of the received data-streams based on the indices.

Abstract: A system, method and computer readable medium for indexing streaming data. Data may be received from distributed devices connected via a network. Data elements may be stored and allocated to data blocks and events of the block stores. Non-text data may be converted into a text representation. The data may be split into terms, and term frequencies of each term within each of the event may be calculated. Block-level term frequency statics may be calculated based on the term frequencies. Tree index structures, such as the Y-tree index, may be generated based on the block-level term frequency data. The Y-tree index structures may use the terms as keys and pointers to the corresponding data blocks and block-level term frequency data. A search query may be performed over the tree index structures.

Abstract: A system, a method and a computer readable medium for storing data elements and related data provenance information. The data elements may be represented in a hyper-table having rows and columns which may be indexed. The data-values of the corresponding data-elements in the hyper-cells may be retrieved based on the indices. Snapshots of the indices may be generated at pre-determined time periods. Checkpoints of the hyper-table may be generated at time periods that are based on transactions on the hyper-table. The hyper-table is capable of being queried as the hyper-table existed at certain time-periods, and data-values of the data-elements may be retrieved as the data-elements existed at such time-periods.

Abstract: Systems, methods and computer readable medium for storing data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data elements may be allocated to data blocks of a block store that have assigned block identifiers and further allocated to events of the data blocks. Stream schema of the received plurality of data streams may have the same stream schema, and indices may be generated indices based on the order of the event allocated data elements. Stream schema of the received data streams may comprise a list of token names. Token names may be assigned to the event allocated data elements. Indices may be generated for the event allocated data elements based on the stream schema.

Abstract: Systems and methods for storing and retrieving data elements transmitted via data streams received from distributed devices connected via a network. The received data elements may be stored in block stores on the distributed devices. The stored data-elements may be allocated to data-blocks of a block-store that have assigned block-identifiers and further allocated to events of the data-blocks that have assigned token-names. Stream-schema of the received data-streams may comprise a list of token-names and an index-definition for each corresponding data-stream. Indices may be generated for the event-allocated data-elements. A query may be executed in order to retrieve data-elements of the received data-streams based on the indices.

Abstract: A system and method for storing data-files stored on distributed devices connected to a network. Data-elements of the data-files are allocated to data-blocks stored on the distributed devices. Key-identifiers are calculated for each of the data-blocks based on the allocated data-elements. The key-identifiers are stored in distributed tables stored on the distributed devices. Index-nodes are generated for the data-files based on the data-blocks. A Paxos algorithm is executed for the index-nodes based on the key-identifiers to provide a consensus of the data-files that are stored on the plurality of distributed devices.