Abstract: Systems and methods for data ingest in interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated. The interest-driven business intelligence system may maintain a set of registered data ingest instruction data that includes at least one registered data ingest instruction data. Each of the at least one registered data ingest instruction data includes an identifier and data ingest instruction data associated with the identifier. The system may receive a request to generate data using registered data instruction data. The request may include the identifier of the registered data instruction data. Data is generated using the data ingest instruction data associated with the requested identifier and at least one of raw data, source data, and aggregate data, and provided for use.

Abstract: Systems and methods for interest-driven data visualization systems in accordance with embodiments of the invention are illustrated.

Abstract: Systems and methods for interest-driven business intelligence systems including event-oriented data in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven business intelligence system includes raw data storage configured to contain raw data and perform ETL processes, a data mart configured to contain metadata that describes the raw data, and an intermediate processing layer, wherein the intermediate processing layer is configured to compile an interest-driven data pipeline configured to generate ETL processing jobs to generate event-oriented data from the raw data by filtering the raw data based on the metadata describing the raw data, determining ordering data based on the metadata describing the raw data, aligning the filtered raw data based on the ordering data, and generating event-oriented data based on the aligned filtered raw data, and storing the event-oriented data in the data mart.

Abstract: Systems and methods for interest-driven business intelligence systems including geo-spatial data in accordance with embodiments of the invention are illustrated.

Abstract: Systems and methods for interest-driven business intelligence systems including segment data in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven data visualization system includes a processor and a memory wherein an interest-driven data visualization application configures the processor to define reporting data requirements, including at least one piece of reporting data metadata, generate data retrieval job data, transmit the data retrieval job data to an interest-driven business intelligence server system, receive source data from the interest-driven business intelligence server system, create at least one piece of reporting data using the received source data, the data description metadata, and the reporting data metadata, and generate a set of segment data based on the reporting data, where a piece of segment data in the set of segment data includes a subset of the reporting data.

Abstract: Described herein are methods, systems, apparatuses and products for adaptive parallel data processing. An aspect provides providing a map phase in which at least one map function is applied in parallel on different partitions of input data at different mappers in a parallel data processing system; providing a communication channel between mappers using a distributed meta-data store, wherein said map phase comprises mapper data processing adapted responsive to communication with said distributed meta-data store; and providing data accessible by at least one reduce phase node in which at least one reduce function is applied. Other embodiments are disclosed.

Abstract: Systems and methods for interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven distributed data server system includes a processor, an interest-driven distributed data server application, aggregate data storage, and a data source directory, wherein the interest-driven distributed data server application configures the processor to obtain reporting data requirements, determine at least one data dimension using the reporting data requirements, identify at least one data source identified that can provide data associated with the data dimension, generate data source job data, transmit the data source job data to at least one data source, and receive job response data, store the received job response data in the aggregate data storage, build reporting data based on the reporting data requirements, and transmit the reporting data.

Abstract: Systems and methods for interest-driven data sharing in interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated.

Abstract: Systems and methods for interest-driven data visualization systems in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven data visualization system includes a processor, a memory connected configured to store an interest-driven data visualization application, and metadata storage, wherein the interest-driven data visualization application configures the processor to define reporting data requirements, generate data retrieval job data based on the reporting data requirements, transmit the data retrieval job data, receive aggregate data, create at least one piece of reporting data using the received aggregate data and the reporting data requirements, associate visualization metadata with the reporting data describing the visual appearance of the at least one piece of reporting data, and generate a report using the reporting data requirements and the visualization metadata.

Abstract: Interest-driven Business Intelligence (BI) systems in accordance with embodiments of the invention are illustrated. In one embodiment of the invention, a data processing system includes raw data storage containing raw data, metadata storage containing metadata that describes the raw data, and an interest-driven data pipeline that is automatically compiled to generate reporting data using the raw data, wherein the interest-driven data pipeline is compiled based upon reporting data requirements automatically derived from at least one report specification defined using the metadata.

Abstract: Interest-driven Business Intelligence (BI) systems in accordance with embodiments of the invention are illustrated. In one embodiment of the invention, a data processing system includes raw data storage containing raw data, metadata storage containing metadata that describes the raw data, and an interest-driven data pipeline that is automatically compiled to generate reporting data using the raw data, wherein the interest-driven data pipeline is compiled based upon reporting data requirements automatically derived from at least one report specification defined using the metadata.

Abstract: Interest-driven Business Intelligence (BI) systems in accordance with embodiments of the invention are illustrated. In one embodiment of the invention, a data processing system includes raw data storage containing raw data, metadata storage containing metadata that describes the raw data, and an interest-driven data pipeline that is automatically compiled to generate reporting data using the raw data, wherein the interest-driven data pipeline is compiled based upon reporting data requirements automatically derived from at least one report specification defined using the metadata.

Abstract: Described herein are methods, systems, apparatuses and products for adaptive parallel data processing. An aspect provides providing a map phase in which at least one map function is applied in parallel on different partitions of input data at different mappers in a parallel data processing system; providing a communication channel between mappers using a distributed meta-data store, wherein said map phase comprises mapper data processing adapted responsive to communication with said distributed meta-data store; and providing data accessible by at least one reduce phase node in which at least one reduce function is applied. Other embodiments are disclosed.

Abstract: Disclosed is a method of encoding JavaScript Object Notation (JSON) documents in an inverted index, wherein a tree representation of a JSON document is first generated, and, next, the JSON document is shredded into a list of <value, path, type, jdewey> tuples for each atom node, n, in the tree, where value is a label associated with n, path is a concatenation of node labels associated with ancestors of n, type is a description of a type of value, and jdewey of n is a partial Dewey code of its closest ancestor array node, if one exists, or empty, otherwise. Lastly, an inverted index is built using <path, type, value> as index term, and jdewey as payload. A method is also described to search the inverted index.

Abstract: A dynamic Bloom filter comprises a cascaded set of Bloom filters. The system estimates or guesses a cardinality of input items, selects a number of hash functions based on the desired false positive rate, and allocates memory for an initial Bloom filter based on the estimated cardinality and desired false positive rate. The system inserts items into the initial Bloom filter and counts the bits set as they are inserted. If the number of bits set in the current Bloom filter reaches a predetermined target, the system declares the current Bloom filter full. The system recursively generates additional Bloom filters as needed for items remaining after the initial Bloom filter is filled; items are checked to eliminate duplicates. Each of the set of Bloom filters is individually queried to identify a positive or negative in response to a query.

Abstract: An extensible identification system for nodes in a hierarchy is described wherein each node is assigned a concatenation of decimal based values. The values assigned uniquely identify the node, provides an order for the node, and identifies its parent, child, and sibling relationships with other nodes Furthermore, the IDs assigned can be encoded to be byte comparable. Furthermore, the ID's assigned to nodes need not be modified when changes (adding/deleting a child node or a subtree of nodes) are made in the hierarchy. Additionally, in the event of such a change, the order and relationships between the parent, child, and sibling nodes are retained.

Abstract: A dynamic foldering system automatically manages the creation and deletion of run-time dynamic folders and selection of documents, items, or object graphs found in the run-time dynamic folders. The system comprises a query/predicate for a design-time folder group based on which dynamic folders are automatically managed, a variable binding mechanism, automatic identifications of dynamic folders containing a particular object, parallel navigation, and customized combination of foldering results. The dynamic folder hierarchy is defined by a hierarchy of design-time folder groups defined by a query/predicate on a document comprising structured or semi-structured data. Modifiable criteria for creating the dynamic folder hierarchy are provided by the user. Variable binding provides powerful query/predicate definitions on hierarchical data or graph-structured data. Given a document, the system automatically identifies which dynamic folders contain the document.

Abstract: A cache-aware Bloom filter system segments a bit vector of a cache-aware Bloom filter into fixed-size blocks. The system hashes an item to be inserted into the cache-aware Bloom filter to identify one of the fixed-size blocks as a selected block for receiving the item and hashes the item k times to generate k hashed values for encoding the item for insertion in the in the selected block. The system sets bits within the selected block with addresses corresponding to the k hashed values such that accessing the item in the cache-aware Bloom filter requires accessing only the selected block to check the k hashed values. The size of the fixed-size block corresponds to a cache-line size of an associated computer architecture on which the cache-aware Bloom filter is installed.

Abstract: Disclosed is an evaluation technique for text search with black-box scoring functions, where it is unnecessary for the evaluation engine to maintain details of the scoring function. Included is a description of a system for dealing with blackbox searching, proofs of correctness, as well experimental evidence showing that the performance of the technique is comparable in efficiency to those techniques used in custom-built engines.

Abstract: The task of estimating the number of distinct values (DVs) in a large dataset arises in a wide variety of settings in computer science and elsewhere. The present invention provides synopses for DV estimation in the setting of a partitioned dataset, as well as corresponding DV estimators that exploit these synopses. Whenever an output compound data partition is created via a multiset operation on a pair of (possibly compound) input partitions, the synopsis for the output partition can be obtained by combining the synopses of the input partitions. If the input partitions are compound partitions, it is not necessary to access the synopses for all the base partitions that were used to construct the input partitions. Superior (in certain cases near-optimal) accuracy in DV estimates is maintained, especially when the synopsis size is small. The synopses can be created in parallel, and can also handle deletions of individual partition elements.