Abstract: A method for managing big metadata using columnar techniques includes receiving a query request requesting data blocks from a data table that match query parameters. The data table is associated with system tables that each includes metadata for a corresponding data block of the data table. The method includes generating, based on the query request, a system query to return a subset of rows that correspond to the data blocks that match the query parameters. The method further includes generating, based on the query request and the system query, a final query to return a subset of data blocks from the data table corresponding to the subset of rows. The method also includes determining whether any of the data blocks in the subset of data blocks match the query parameters, and returning the matching data blocks when one or more data blocks match the query parameters.

Abstract: A method for optimizing a database query includes receiving a database query from a user requesting a database to conditionally return one or more rows stored at the database and characterizing one or more user filters. Each user filter filters rows from the database. The method includes determining that a security filter restricts the user from accessing one or more rows of the database. For each respective user filter, the method includes classifying the respective user filter as a safe user filter incapable of reporting an error or an unsafe user filter capable of reporting an error. The method also includes determining a filter execution order of the one or more user filters and the security filter based on the classified one or more user filters and executing, using the determined filter execution order, the one or more user filters and the security filter.

Abstract: A method for managing big metadata using columnar techniques includes receiving a query request requesting data blocks from a data table that match query parameters. The data table is associated with system tables that each includes metadata for a corresponding data block of the data table. The method includes generating, based on the query request, a system query to return a subset of rows that correspond to the data blocks that match the query parameters. The method further includes generating, based on the query request and the system query, a final query to return a subset of data blocks from the data table corresponding to the subset of rows. The method also includes determining whether any of the data blocks in the subset of data blocks match the query parameters, and returning the matching data blocks when one or more data blocks match the query parameters.

Abstract: A method for managing big metadata using columnar techniques includes receiving a query request requesting data blocks from a data table that match query parameters. The data table is associated with system tables that each includes metadata for a corresponding data block of the data table. The method includes generating, based on the query request, a system query to return a subset of rows that correspond to the data blocks that match the query parameters. The method further includes generating, based on the query request and the system query, a final query to return a subset of data blocks from the data table corresponding to the subset of rows. The method also includes determining whether any of the data blocks in the subset of data blocks match the query parameters, and returning the matching data blocks when one or more data blocks match the query parameters.

Abstract: Systems and methods for generating distributed dataflow graphs and processing data elements in parallel utilizing the distributed dataflow graphs are provided. A sequential dataflow graph is formed from incoming data elements, and a variety of heuristics is applied to the sequential dataflow graph to determine which of the data transformation steps within the graph are capable of being processed multiple times in parallel. Once determined, the sequential dataflow graph is divided into subgraphs, which are then replicated, e.g., based on available resources and.or external constraints. The resulting subgraphs are connected, based on the semantics of each vertex, and a distributed dataflow graph is generated, which can efficiently process data elements, for instance, for data warehousing and the like.

Abstract: Computer-readable media, systems, and methods for building a multidimensional data cube having one or more high-cardinality attributes are described. In embodiments, data is extracted from one or more databases. It is determined that one or more instances of the data are fact data and one or more instances of the data are dimension data. Each member of the fact data is one instance of a dimension and each instance of the dimension data includes an attribute for grouping the fact data. Moreover, in embodiments it is determined that one or more instances of the dimension data are high-cardinality attributes. The one or more high-cardinality attributes are processed with fact data and stored in fact tables on a computer storage medium.

Abstract: A system and techniques for executing queries of a sparse multidimensional data space in the presence of expressions. A dimension is associated with a set of data references, which reference stored data items. A query expression has dimension or a data reference as an operand. A result space with one or more result dimensions is defined based on the query. A plan is built and executed for retrieving the stored data items associated with the result dimensions, and forming “calculated data items” by evaluating the expression. Executing the plan produces a stream of data tuples, each having a set of data components storing the retrieved data items and/or the calculated data items. Data components are either empty or have data values. An iterative algorithm is employed on the stream to locate the data tuples that are empty at the intersection of each result dimension.

Abstract: Systems and method for creating multidimensional data cubes containing data domains for analyzing large amounts of data are provided. Data domains may be included in the major object of a multidimensional data cube. Further embodiments of the present invention provide methods for querying multidimensional data cubes having data domains. Embodiments of the present invention provide for defining data domains by any object in the major object model and for defining parent and child data domains.

Abstract: The subject invention pertains to the integration of an object model and a multidimensional database query language such as MDX. In particular, multidimensional declarative query language entities can be exposed through objects in an object model. The object model enables generation and employment of procedural language functions, routines, or procedures that interact with multidimensional database data. This effectively provides an extension for multidimensional query languages. For example, procedures can be stored and invoked from a declarative language query. Furthermore, the object model disclosed herein can expose a context object to enable conditional procedures based on the context of execution.

Abstract: Provided are systems and methods that facilitate direct write back in a multi-dimensional database. The system includes a delta cache component that receives a user request to change an original cell value and determines a delta value based at least in part upon the changed cell value. Also included is a write back partition component that selectively updates a data cell based at least in part upon the delta value without updating corresponding data cell values. The system and methods allow attributes to be added to any dimension of a cube without affecting the write back data. Adding, modifying or removing a hierarchy has no affect on write back data nor does deleting a dimension that is not referenced by a write back.

Abstract: A system, method, and related techniques are disclosed for processing data across a distributed network to a plurality of machines. The method may include receiving a first user-supplied transform and generating a first package based on the first user-supplied transform. The method may further include receiving a designated key and generating a second package based on the key. Furthermore, the method may include receiving a second user-supplied transform and generating a third package based on the second user-supplied transform. Moreover, the method may include distributing the first, second, and third packages to a plurality of machines using a cluster API.

Abstract: The subject invention pertains to systems and methods for interacting with fact dimensions. In particular, systems and methods are disclosed that optimize performance and scalability with respect to processing queries that involve fact dimensions. Furthermore, queries involving fact dimensions can be evaluated in distinct manners. For instance, queries can be processed such that regular dimensions restrict the scope of the data and only fact dimension members that are relevant to that scope are exposed.

Abstract: A simple interface may be provided that enables the user to define parameters for aggregation of a semi-additive measure. The interface may enable the user to designate a measure as a semi-additive measure and to pair the measure with an additive aggregation function. The interface may also enable the user to select non-additive dimensions and to pair each non-additive dimension with a corresponding aggregation function. One such aggregation function is a by account aggregation function, which enables each account in an account dimension to be aggregated across a corresponding non-additive dimension according to an associated account type.

Abstract: The subject invention pertains to a centralized key performance indicator (KPI) framework and systems and methods of utilization. In particular, KPIs can be defined and centrally stored as data or metadata in a data store. Systems and methods are provided to enable, among other things, KPI definition, storage, identification of defined or available KPIs, and interaction with the KPIs. For instance, an interface is provided to allow KPI data to be requested and retrieved from a source such as a database. A generic application program or users thereof can then interact with and display KPI data without knowing anything a priori about the stored data or the structure thereof.

Abstract: The subject invention pertains to systems and methods that facilitate development and debugging of calculations in a multidimensional data environment or model. Data cube calculation scripts, including those provided in MDX (Multidimensional Expressions), can be executed or applied incrementally. Breakpoints can be set with respect to a calculation script and the cube state can be viewed and queried via a user interface. Users are enabled by the subject systems and methods to step through calculation scripts, identify the effects on a cube, and alter the script if the desired results failed to occur. In addition to debugging, the systems and methods can be employed for initial development of scripts.

Abstract: A system and techniques for executing queries of a sparse multidimensional data space in the presence of expressions. A dimension is associated with a set of data references, which reference stored data items. A query expression has dimension or a data reference as an operand. A result space with one or more result dimensions is defined based on the query. A plan is built and executed for retrieving the stored data items associated with the result dimensions, and forming “calculated data items” by evaluating the expression. Executing the plan produces a stream of data tuples, each having a set of data components storing the retrieved data items and/or the calculated data items. Data components are either empty or have data values. An iterative algorithm is employed on the stream to locate the data tuples that are empty at the intersection of each result dimension.

Abstract: The subject invention relates to systems and methods that extend the network data access capabilities of mark-up language protocols. In one aspect, a network data transfer system is provided. The system includes a protocol component that employs a computerized mark-up language to facilitate data interactions between network components, whereby the data interactions were previously limited or based on a statement command associated with the markup language. An extension component operates with the protocol component to support the data transactions, where the extension component supplies at least one other command from the statement command to facilitate the data interactions.

Abstract: A system that facilitates analyzing content of a multi-dimensional structure comprises a calculation component that receives statements in a declarative language relating to one or more of an assignment and calculation and executes such statements against a multi-dimensional structure. A pass generation component creates a pass in order to maintain content of the multi-dimensional structure as it existed prior to execution of the statement, the pass is accessible upon reference to such pass.

Abstract: A system that facilitates one or more of querying and updating a multi-dimensional structure comprises a component that receives a statement in a declarative language relating to a typed object associated with a multi-dimensional structure. A conversion component analyzes context associated with the statement and automatically converts the object to a disparate type as a function of the analysis. For example, an execution engine can comprise the conversion component, and the execution engine can be an Online Analytical Processing (OLAP) engine.

Abstract: Systems and methods for storing and retrieving data items in multidimensional databases are provided. Data partition templates are generated for grouping data partitions that are similar to one another, that is, that contain information specified according to particular common characteristics (for instance, time, product, geography, etc.). The data partition template includes one or more rules concerning how the data stored in partitions associated with the data partition template should be processed. A template object having the rule(s) associated therewith is generated for the data partition template. Once a data partition template is generated, a plurality of partitions are generated in accordance therewith. Each of the plurality of partitions utilizes the template object associated with the data partition template. In this way, the overhead associated with managing the partitions is significantly decreased.