Abstract: Embodiments of the invention are described being used to distribute a database across a plurality of compute nodes of a parallel computing system. That is, a method is described for creating a fully in-memory database on the parallel computing system. Further, data compression techniques are described being used to increase the performance of the in-memory database by compressing some portions of the database to fit within a single node or a logically or physically related group of nodes.

Abstract: Techniques disclosed herein describe joining processing elements (e.g., of a streams processing environment) using optimization techniques, such as by whole program optimization. An optimization tool evaluates multiple processing elements based on one or more optimization rules. The optimization tool identifies, based on the evaluation, processing elements that can be merged. The optimization tool joins the identified processing elements together.

Abstract: A method and apparatus performs peer-to-peer file transfers on a High Performance Computing (HPC) cluster such as a Beowulf cluster. A peer-to-peer file tracker (PPFT) allows operating system, application and data files to be moved from a pre-loaded node to another node of the HPC cluster. A peer-to-peer (PTP) client is loaded into the nodes to facilitate PTP file transfers to reduce loading on networks, network switches and file servers to reduce the time needed to load the nodes with these files to increase overall efficiency of the multi-node computing system. The selection of the nodes participating in file transfers can be based on network topology, network utilization, job status and predicted network/computer utilization. This selection can be dynamic, changing during the file transfers as resource conditions change. The policies used to choose resources can be configured by an administrator.

Abstract: An apparatus, program product and method dynamically optimize prepared statements resident in a statement pool to permit subsequent reuses of such statements to utilize the optimized representations of such statements. The optimization of pooled statements may be performed responsive to the tracked usage of the statements in a statement pool, to direct the overhead associated with such optimization to those statements that are more likely to be utilized in the future. Among a multitude of possible optimizations, a pooled statement that requests data in a format other than that utilized by the database being accessed may be modified to request the data in the format used by the database, thereby decreasing the overhead in the database associated with format conversions.

Abstract: A method and apparatus loads data to an in-memory database across multiple nodes in a parallel computing system. A database loader uses SQL flags, historical information gained from monitoring prior query execution times and patterns, and node and network configuration to determine how to effectively cluster data attributes across multiple nodes. The database loader may also allow a system administrator to force placement of database structures in particular nodes.

Abstract: Embodiments of the invention enhance a garbage collection process running on a parallel system or distributed computing environment. Using a garbage collector in such an environment allows a more in-depth analysis to be performed than is possible on other systems. This is because the number of compute nodes present in many parallel systems, and the connections between them, allows the overhead of doing advanced analysis to be spread across the nodes and the results of that analysis to be shared among the nodes.

Abstract: A method and apparatus creates and manages persistent memory (PM) in a multi-node computing system. A PM Manager in the service node creates and manages pools of nodes with various sizes of PM. A node manager uses the pools of nodes to load applications to the nodes according to the size of the available PM. The PM Manager can dynamically adjust the size of the PM according to the needs of the applications based on historical use or as determined by a system administrator. The PM Manager works with an operating system kernel on the nodes to provide persistent memory for application data and system metadata. The PM Manager uses the persistent memory to load applications to preserve data from one application to the next. Also, the data preserved in persistent memory may be system metadata such as file system data that will be available to subsequent applications.

Abstract: A database dynamic partial uncompression mechanism determines when to dynamically uncompress one or more compressed portions of a database table that also includes uncompressed portions. A query may include an express term that specifies whether or not to skip compressed portions. In addition, a query may include associated information that specifies whether or not to skip compressed portions, and one or more thresholds that may be used to determine if the system is too busy to perform uncompression. A display mechanism may also determine whether or not to display compressed portions. The uncompression may occur at the database server or at a client. The database dynamic partial uncompression mechanism thus performs dynamic uncompression in a way that preferably uncompresses one or more compressed portions of a partially compressed database table only when the compressed portions satisfy a query and/or need to be displayed.

Abstract: An apparatus and method for a query optimizer improves string searches in a computer database that sequentially search for a string in a database record. The query optimizer optimizes the query to search records of a database from a specified start position other than the beginning of the record. The specified start position of the search may be determined by from historical information stored from previous searches. Alternatively, the query optimizer determines the specified start position of the search based on an overriding starting position provided by a system administrator. The query optimizer may also direct that the database record be reorganized to more efficiently search for strings in the record.

Abstract: Embodiments of the invention provide techniques for generating statistics for optimizing database queries containing user-defined functions (UDFs). In general, the statistics may be generated based on output values produced during past executions of a UDF. The statistics may also be generated based on input values received during past executions of the UDF. Additionally, the statistics may include input and output value pairs, such that a UDF output may be determined based on a UDF input. The generated statistics may be used by a query optimizer to determine an efficient query plan for executing the database query.

Abstract: An apparatus, program product and method utilize one or more usage statistics, which are reflective of what data is actually and/or likely to be used by a client when processing a database query, to reduce or eliminate the transfer of unnecessary data between a client and server in a client-server database management system. Information such as the columns that are or have been used and/or the number of rows that have been accessed, may be utilized to identify certain data that is likely to not be used during processing of the database query. Based upon the data that has been identified, a subset of the data that would otherwise be returned as a result of the database query may be identified and returned to a client, often substantially reducing the amount of data required to be retrieved by a client in connection with a database query.

Abstract: A method, apparatus, and program product manage scheduling of a plurality of jobs in a parallel computing system of the type that includes a plurality of computing nodes and is disposed in a data center. The plurality of jobs are scheduled for execution on a group of computing nodes from the plurality of computing nodes based on the physical locations of the plurality of computing nodes in the data center. The group of computing nodes is further selected so as to distribute at least one of a heat load and an energy load within the data center. The plurality of jobs may be additionally scheduled based upon an estimated processing requirement for each job of the plurality of jobs.

Abstract: The disclosure herein provides data security on a parallel computer system using virtual private networks connecting the nodes of the system. A mechanism sets up access control data in the nodes that describes a number of security classes. Each security class is associated with a virtual network. Each user on the system is associated with one of the security classes. Each database object to be protected is given an attribute of a security class. Database objects are loaded into the system nodes that match the security class of the database object. When a query executes on the system, the query is sent to a particular class or set of classes such that the query is only seen by those nodes that are authorized by the equivalent security class. In this way, the network is used to isolate data from users that do not have proper authorization to access the data.

Abstract: A code optimizer is used to optimize a computer program that references a database by reading database metadata and making suitable optimizations based on the metadata. By taking into account the metadata of a database referenced in the computer program, the optimizer may make suitable optimizations to the computer program. Such optimizations include, without limitation, removing unnecessary calls to the database, removing unnecessary loops, removing unnecessary database operations, providing compile-time errors, and replacing dynamic calls with static data.

Abstract: A method for optimizing efficiency and power consumption in a hybrid computer system is disclosed. The hybrid computer system may comprise one or more front-end nodes connected to a multi-node computer system. Portions of an application may be offloaded from the front-end nodes to the multi-node computer system. By building historical profiles of the applications running on the multi-node computer system, the system can analyze the trade offs between power consumption and performance. For example, if running the application on the multi-node computer system cuts the run time by 5% but increases power consumption by 20% it may be more advantageous to simply run the entire application on the front-end.

Abstract: In a first aspect, a method is provided that includes the steps of (1) pre-computing a query result for each of a plurality of whole segments of data included in a database; (2) receiving a query specifying a defined range of data in the database; (3) determining if any of the whole segments are within the defined range; (4) performing the query on any partial segments of data within the defined range; and (5) determining the result of the query based on the pre-computed query results for any whole segments determined to be within the defined range and the result of the query on any partial segments within the defined range. Numerous other aspects are provided.

Abstract: A method, apparatus and program product for allocating resources in a logically partitioned multiprocessor environment. Resource usage is monitored in a first logical partition in the logically partitioned multiprocessor environment to predict a future underutilization of a resource in the first logical partition. An application executing in a second logical partition in the logically partitioned multiprocessor environment is configured for execution in the second logical partition with an assumption made that at least a portion of the underutilized resource is allocated to the second logical partition during at least a portion of the predicted future underutilization of the resource.

Abstract: A method and apparatus creates and manages persistent memory (PM) in a multi-node computing system. A PM Manager in the service node creates and manages pools of nodes with various sizes of PM. A node manager uses the pools of nodes to load applications to the nodes according to the size of the available PM. The PM Manager can dynamically adjust the size of the PM according to the needs of the applications based on historical use or as determined by a system administrator. The PM Manager works with an operating system kernel on the nodes to provide persistent memory for application data and system metadata. The PM Manager uses the persistent memory to load applications to preserve data from one application to the next. Also, the data preserved in persistent memory may be system metadata such as file system data that will be available to subsequent applications.

Abstract: The disclosure herein provides data security on a parallel computer system using virtual private networks connecting the nodes of the system. A mechanism sets up access control data in the nodes that describes a number of security classes. Each security class is associated with a virtual network. Each user on the system is associated with one of the security classes. Each database object to be protected is given an attribute of a security class. Database objects are loaded into the system nodes that match the security class of the database object. When a query executes on the system, the query is sent to a particular class or set of classes such that the query is only seen by those nodes that are authorized by the equivalent security class. In this way, the network is used to isolate data from users that do not have proper authorization to access the data.

Abstract: A method and apparatus optimizes job and data distribution on a multi-node computing system. A job scheduler distributes jobs and data to compute nodes according to priority and other resource attributes to ensure the most critical work is done on the nodes that are quickest to access and with less possibility of node communication failure. In a tree network configuration, the job scheduler distributes critical jobs and data to compute nodes that are located closest to the I/O nodes. Other resource attributes include network utilization, constant data state, and class routing.