Abstract: Methods, systems, and computer-readable media of columnar storage of a database index are disclosed. A particular columnar index includes a column store that stores rows of the columnar index in a column-wise fashion and a delta store that stores rows of the columnar index in a row-wise fashion. The column store also includes an absence flag array. The absence flag array includes entries that indicate whether certain rows have been logically deleted from the column store.

Abstract: A data normalization system is described herein that represents multiple data types that are common within database systems in a normalized form that can be processed uniformly to achieve faster processing of data on superscalar CPU architectures. The data normalization system includes changes to internal data representations of a database system as well as functional processing changes that leverage normalized internal data representations for a high density of independently executable CPU instructions. Because most data in a database is small, a majority of data can be represented by the normalized format. Thus, the data normalization system allows for fast superscalar processing in a database system in a variety of common cases, while maintaining compatibility with existing data sets.

Abstract: Architecture that includes an index creation algorithm that utilizes available resources and dynamically adjusts to successfully scale with increased resources and be able to do so for any data distribution. The resources can be processing resources, memory, and/or input/output, for example. A finer level of granularity, called a segment, is utilized to process tuples in a partition while creating an index. The segment also aligns with compression techniques for the index. By choosing an appropriate size for a segment and using load balancing the overall time for index creation can be reduced. Each segment can then be processed by a single thread thereby limiting segment skew. Skew is further limited by breaking down the work done by a thread into parallelizable stages.

Abstract: Architecture that includes an index creation algorithm that utilizes available resources and dynamically adjusts to successfully scale with increased resources and be able to do so for any data distribution. The resources can be processing resources, memory, and/or input/output, for example. A finer level of granularity, called a segment, is utilized to process tuples in a partition while creating an index. The segment also aligns with compression techniques for the index. By choosing an appropriate size for a segment and using load balancing the overall time for index creation can be reduced. Each segment can then be processed by a single thread thereby limiting segment skew. Skew is further limited by breaking down the work done by a thread into parallelizable stages.

Abstract: The degree of parallel processing used to build a database index can be dynamically adjusted based on actual memory usage of individual parallel processing units. Memory can be reserved to prevent an out-of-memory condition. A predetermined number of initial parallel processing units can be activated. The actual usage of resources by the initial activated parallel processing unit(s) can be measured to establish an initial baseline for resource consumption per parallel processing unit. The baseline for resource consumption per parallel processing unit can be used to determine how many additional parallel processing units are activated. The actual resource usage of each parallel processing unit can be measured and used to refine the baseline memory usage. The refined average memory usage can be used to determine how many additional parallel processing units are activated.

Abstract: A data normalization system is described herein that represents multiple data types that are common within database systems in a normalized form that can be processed uniformly to achieve faster processing of data on superscalar CPU architectures. The data normalization system includes changes to internal data representations of a database system as well as functional processing changes that leverage normalized internal data representations for a high density of independently executable CPU instructions. Because most data in a database is small, a majority of data can be represented by the normalized format. Thus, the data normalization system allows for fast superscalar processing in a database system in a variety of common cases, while maintaining compatibility with existing data sets.

Abstract: A data normalization system is described herein that represents multiple data types that are common within database systems in a normalized form that can be processed uniformly to achieve faster processing of data on superscalar CPU architectures. The data normalization system includes changes to internal data representations of a database system as well as functional processing changes that leverage normalized internal data representations for a high density of independently executable CPU instructions. Because most data in a database is small, a majority of data can be represented by the normalized format. Thus, the data normalization system allows for fast superscalar processing in a database system in a variety of common cases, while maintaining compatibility with existing data sets.

Abstract: Architecture that provides for greater interoperability between column stores and row stores by leveraging the advantages both have to offer. The architecture operates automatically (e.g., dynamically) to move between row oriented processing mode and batch processing mode, and the combination thereof, when it is more beneficial to run in one mode relative to the other mode, or both modes. The auto-switching of data processing between batch and row oriented mode occurs during the execution of a single query. The architecture can automatically modify an operator in the query tree and/or remove an operator if desired at runtime for more efficient processing. This approach also accounts for memory constraints for either of row or column processing.

Abstract: Aspects of the subject matter described herein relate to optimized reverse key indexes. In aspects, a dispersion function disperses index values such that they are distributed across multiple pages of an index. The dispersion function utilizes a dispersion factor that indicates to what extent the index values are dispersed. Because the index values are dispersed, contention regarding inserts may be reduced or eliminated and other advantages realized.

Abstract: A data normalization system is described herein that represents multiple data types that are common within database systems in a normalized form that can be processed uniformly to achieve faster processing of data on superscalar CPU architectures. The data normalization system includes changes to internal data representations of a database system as well as functional processing changes that leverage normalized internal data representations for a high density of independently executable CPU instructions. Because most data in a database is small, a majority of data can be represented by the normalized format. Thus, the data normalization system allows for fast superscalar processing in a database system in a variety of common cases, while maintaining compatibility with existing data sets.

Abstract: Database management is described. A source data structure is copied to create a new data structure. Changes to the source data structure that occur during and after creation of the new data structure are accumulated in a file before they are added to the new data structure. Changes included in the file are subsequently applied to the second data structure.

Abstract: Architecture that includes an index creation algorithm that utilizes available resources and dynamically adjusts to successfully scale with increased resources and be able to do so for any data distribution. The resources can be processing resources, memory, and/or input/output, for example. A finer level of granularity, called a segment, is utilized to process tuples in a partition while creating an index. The segment also aligns with compression techniques for the index. By choosing an appropriate size for a segment and using load balancing the overall time for index creation can be reduced. Each segment can then be processed by a single thread thereby limiting segment skew. Skew is further limited by breaking down the work done by a thread into parallelizable stages.

Abstract: Architecture that provides for greater interoperability between column stores and row stores by leveraging the advantages both have to offer. The architecture operates automatically (e.g., dynamically) to move between row oriented processing mode and batch processing mode, and the combination thereof, when it is more beneficial to run in one mode relative to the other mode, or both modes. The auto-switching of data processing between batch and row oriented mode occurs during the execution of a single query. The architecture can automatically modify an operator in the query tree and/or remove an operator if desired at runtime for more efficient processing. This approach also accounts for memory constraints for either of row or column processing.

Abstract: Methods, systems, and computer-readable media of columnar storage of a database index are disclosed. A particular columnar index includes a column store that stores rows of the columnar index in a column-wise fashion and a delta store that stores rows of the columnar index in a row-wise fashion. The column store also includes an absence flag array. The absence flag array includes entries that indicate whether certain rows have been logically deleted from the column store.

Abstract: Methods, systems, and computer-readable media of database mirroring are disclosed. A particular method includes initiating a transaction that modifies one or more pages of a first database. Each page includes a structure modification operation (SMO) bit and initiating the transaction includes setting the SMO bit of each of the one or more pages to a first value. One or more first records are created at a transaction log of the first database. The transaction log is useable at a second database to mirror the transaction. Each first record indicates the setting of a SMO bit of a particular page to the first value. The database transaction is performed, and the SMO bit of each of the one or more pages is set to a second value. One or more second records are created at the transaction log, each second record indicating the setting of a SMO bit of a particular page to the second value. The method includes committing the transaction.

Abstract: Various technologies and techniques are disclosed for efficiently processing DML SQL statements through minimal logging and optimized insertions. Rows are inserted into a table in a database in a sorted order. When an insertion of a particular row into the table causes a page split and a new page created during the page split is empty, the new page is locked until an end of a transaction associated with the insertion is completed. When the page split is caused by appending monotonically increasing values at an end of the table, the sorted order will guarantee that the new page is empty. Minimal logging is performed. When the transaction associated with the insertion is completed, a forced checkpoint is performed.

Abstract: A method and system for fragment-based serialization places one or more object members in fragments. Fragments may comprise a header and a payload. A header can provide useful information about the fragment, such as an indication of fragment type and an indication of fragment length. A payload may comprise one or more members of an object. Primitive members may be stored in a Binary Fragment with a record format payload. LOB and FS members may be stored in fragments that have a Value Type field for setting forth additional properties of the fragment. Collections may be stored in a series of fragments, a first fragment to indicate a start of a collection, one or more second fragments to serialize collection elements, and a Terminator Fragment to indicate the end of a collection. Fragment-serialized objects minimize storage overhead while providing fast instantiation and low-cost location and updating.

Abstract: Aspects of the subject matter described herein relate to optimized reverse key indexes. In aspects, a dispersion function disperses index values such that they are distributed across multiple pages of an index. The dispersion function utilizes a dispersion factor that indicates to what extent the index values are dispersed. Because the index values are dispersed, contention regarding inserts may be reduced or eliminated and other advantages realized.

Abstract: Various technologies and techniques are disclosed for efficiently processing DML SQL statements through minimal logging and optimized insertions. Rows are inserted into a table in a database in a sorted order. When an insertion of a particular row into the table causes a page split and a new page created during the page split is empty, the new page is locked until an end of a transaction associated with the insertion is completed. When the page split is caused by appending monotonically increasing values at an end of the table, the sorted order will guarantee that the new page is empty. Minimal logging is performed. When the transaction associated with the insertion is completed, a forced checkpoint is performed.

Abstract: Database management is described. A source data structure is copied to create a new data structure. Changes to the source data structure that occur during and after creation of the new data structure are accumulated in a file before they are added to the new data structure. Changes included in the file are subsequently applied to the second data structure.