Abstract: A system includes a production cluster with a first plurality of nodes. The production cluster executes a workload. Jobs associated with the workload are allocated, according to a first configuration, across the first plurality of nodes. A workload simulator is coupled to the production cluster and a test cluster. The workload simulator extracts production cluster data, which includes production capability information, workload data, and production cluster usage information, as well as test capability information. The workload simulator determines a first job type to include in a simulated workload to be executed on the test cluster and a number of jobs of the first job type to include in the simulated workload. The system also includes a test cluster which includes a second plurality of nodes. The second plurality of nodes includes fewer nodes than does the first plurality of nodes. The test cluster executes the simulated workload.

Abstract: Preprocessing of data destined for storage in a data lake is accomplished upstream, such as at edge nodes. The preprocessing includes filtering data that is deemed to be unnecessary for subsequent analytical use purposes. An initial intelligent determination is performed on whether a data feed is to be preprocessed at (i) the data lake, or (ii) upstream of the data lake, such as at an edge node. Once upstream preprocessing has been determined, an intelligent determination of which edge node is to be chosen for preprocessing is performed. The determination on which edge node is to be chosen for preprocessing is based on response times between the application server and the edge nodes and network bandwidth usage encountered by the network transmitting the data feed.

Abstract: A memory stores a first cache and a second cache. A processor copies a first portion of data from a first table stored in a database into a second table. The processor further determines that a second portion of data from the first table will be overwritten and copies the second portion into a third table. The processor further determines that a probability that a user will access a third portion of the first table is greater than a threshold and copies the third portion into the first cache. The processor further determines a fourth portion of the first table that the user accesses at a frequency greater than a set frequency and copies the fourth portion into the second cache.

Abstract: A request for one or more files is received. It is determined that the requested one or more files have been archived at an archival storage tier. Access to a first portion of the one or more requested files stored at the archival storage tier is requested. Access is provided to the first portion in response to the request. Based at least in part on a metric associated with a utilization of the first portion, it is determined when to request access to a second portion of the one or more requested files stored at the archival storage tier.

Abstract: A memory stores a first and second cache. A processor stores a first query in a transaction log. The processor selects information from the transaction log, uses it to determine that a user will likely access a first portion of the database at a future date, and copies that portion into the first cache prior to the date. The processor further copies a second, frequently accessed portion of the database into the second cache. The processor additionally determines that a portion of data is in both caches and deletes it from the second cache. The processor receives a second query from the user, determines that it is directed at data in the first cache and accesses that cache. The processor also determines that the second query is not directed at data in the first cache but is directed at data in the second cache, and accesses that cache.

Abstract: A production cluster executes a workload, such that jobs associated with the executed workload are allocated, according to a first configuration. A cluster monitor extracts production cluster information from the production cluster, monitors configuration information during execution of the workload, and transmits each to a cluster tuner. The cluster tuner receives the information and determines a first recommended configuration for the production cluster. The cluster tuner causes the test cluster to execute a simulated workload according to the first recommended configuration. In response to determining that the first recommended configuration results in a decrease in resource consumption, the cluster tuner causes the production cluster to operate according to the first recommended configuration.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions that receive a structured query language (SQL) query, identify a bucket, identify metadata relationships specified in the SQL query, and execute the SQL query to generate a list of objects included in the bucket and having metadata satisfying the metadata relationships.

Abstract: A production cluster includes a first plurality of nodes and executes a workload, such that jobs associated with the executed workload are allocated, according to a first configuration, across the first plurality of nodes. A cluster monitor extracts production cluster information from the production cluster, monitors configuration information during execution of the workload, and transmits each to a cluster tuner. The cluster tuner includes a second plurality of nodes that is less than the first plurality of nodes of the production cluster. The cluster tuner receives the information and determines a first recommended configuration for the production cluster. The cluster tuner causes the test cluster to execute a simulated workload according to the first recommended configuration. In response to determining that the first recommended configuration results in a decrease in resource consumption, the cluster tuner causes the production cluster to operate according to the first recommended configuration.

Abstract: A system includes a production cluster with a first plurality of nodes. The production cluster executes a workload. Jobs associated with the workload are allocated, according to a first configuration, across the first plurality of nodes. A workload simulator is coupled to the production cluster and a test cluster. The workload simulator extracts production cluster data, which includes production capability information, workload data, and production cluster usage information, as well as test capability information. The workload simulator determines a first job type to include in a simulated workload to be executed on the test cluster and a number of jobs of the first job type to include in the simulated workload. The system also includes a test cluster which includes a second plurality of nodes. The second plurality of nodes includes fewer nodes than does the first plurality of nodes. The test cluster executes the simulated workload.

Abstract: A production cluster includes a first plurality of nodes and executes a workload, such that jobs associated with the executed workload are allocated, according to a first configuration, across the first plurality of nodes. A cluster monitor extracts production cluster information from the production cluster, monitors configuration information during execution of the workload, and transmits each to a cluster tuner. The cluster tuner includes a second plurality of nodes that is less than the first plurality of nodes of the production cluster. The cluster tuner receives the information and determines a first recommended configuration for the production cluster. The cluster tuner causes the test cluster to execute a simulated workload according to the first recommended configuration. In response to determining that the first recommended configuration results in a decrease in resource consumption, the cluster tuner causes the production cluster to operate according to the first recommended configuration.

Abstract: A memory stores a first cache and a second cache. A processor copies a first portion of data from a first table stored in a database into a second table. The processor further determines that a second portion of data from the first table will be overwritten and copies the second portion into a third table. The processor further determines that a probability that a user will access a third portion of the first table is greater than a threshold and copies the third portion into the first cache. The processor further determines a fourth portion of the first table that the user accesses at a frequency greater than a set frequency and copies the fourth portion into the second cache.

Abstract: A memory stores a first cache and a second cache. A processor copies a first portion of data from a first table stored in a database into a second table, performs statistical analysis on the first portion, and stores the results into the second table. The processor further determines that a second portion of data from the first table will be overwritten, copies the second portion into a third table, performs statistical analysis on the second portion and stores the results into the third table. The processor further determines that a probability that a user will access a third portion of the first table is greater than a threshold and copies the third portion into the first cache. The processor further determines a fourth portion of the first table that the user accesses at a frequency greater than a set frequency and copies the fourth portion into the second cache.

Abstract: A resource offloading system comprises a relational database configured to store relational queries including searching features, a server configured to offload the relational queries to non-relational queries, and a non-relational database configured to store the non-relational queries. The server is configured to generate access patterns for each relational query based on searching features included in the relational query, classify the access pattern based on a terminology defined by a searching feature into a classification, rank the classified access patterns, generate candidate access patterns from the ranked access patterns, create non-relational queries based on the candidate access patterns which are ranked beyond a threshold, and store the non-relational queries in the non-relational database.

Abstract: A memory stores a first and second cache. A processor stores a first query in a transaction log. The processor selects information from the transaction log, uses it to determine that a user will likely access a first portion of the database at a future date, and copies that portion into the first cache prior to the date. The processor further copies a second, frequently accessed portion of the database into the second cache. The processor additionally determines that a portion of data is in both caches and deletes it from the second cache. The processor receives a second query from the user, determines that it is directed at data in the first cache and accesses that cache. The processor also determines that the second query is not directed at data in the first cache but is directed at data in the second cache, and accesses that cache.

Abstract: A memory stores a first and second cache. A processor stores a first query in a transaction log. The processor selects information from the transaction log, uses it to determine that a user will likely access a first portion of the database at a future date, and copies that portion into the first cache prior to the date. The processor further copies a second, frequently accessed portion of the database into the second cache. The processor additionally determines that a portion of data is in both caches and deletes it from the second cache. The processor receives a second query from the user, determines that it is directed at data in the first cache and accesses that cache. The processor also determines that the second query is not directed at data in the first cache but is directed at data in the second cache, and accesses that cache.

Abstract: A resource offloading system comprises a relational database configured to store relational queries including searching features, a server configured to offload the relational queries to non-relational queries, and a non-relational database configured to store the non-relational queries. The server is configured to generate access patterns for each relational query based on searching features included in the relational query, classify the access pattern based on a terminology defined by a searching feature into a classification, rank the classified access patterns, generate candidate access patterns from the ranked access patterns, create non-relational queries based on the candidate access patterns which are ranked beyond a threshold, and store the non-relational queries in the non-relational database.

Abstract: A memory stores a first and second cache. A processor stores a first query in a transaction log. The processor selects information from the transaction log, uses it to determine that a user will likely access a first portion of the database at a future date, and copies that portion into the first cache prior to the date. The processor further copies a second, frequently accessed portion of the database into the second cache. The processor additionally determines that a portion of data is in both caches and deletes it from the second cache. The processor receives a second query from the user, determines that it is directed at data in the first cache and accesses that cache. The processor also determines that the second query is not directed at data in the first cache but is directed at data in the second cache, and accesses that cache.

Abstract: A memory stores a first cache and a second cache. A processor copies a first portion of data from a first table stored in a database into a second table, performs statistical analysis on the first portion, and stores the results into the second table. The processor further determines that a second portion of data from the first table will be overwritten, copies the second portion into a third table, performs statistical analysis on the second portion and stores the results into the third table. The processor further determines that a probability that a user will access a third portion of the first table is greater than a threshold and copies the third portion into the first cache. The processor further determines a fourth portion of the first table that the user accesses at a frequency greater than a set frequency and copies the fourth portion into the second cache.

Abstract: Methods and apparatus are provided for determining a navigational state of a vehicle, the vehicle having at least one pivotable wheel and a plurality of front wheels. The apparatus comprises a steering angle sensor coupled to the at least one pivotable wheel for determining a steering angle, a plurality of wheel speed sensors each coupled to a different one of the plurality of pivotable wheels for determining an angular velocity of each of the plurality of pivotable wheels, a GPS receiver coupled to the vehicle for receiving GPS positioning data, and a processor coupled to the steering angle sensor, the plurality of wheel speed sensors, and the GPS receiver. The processor is configured to determine a yaw rate for the vehicle based on the positioning data, the steering angle, and the longitudinal angular velocity of each of the plurality of front wheels, and integrate the yaw rate to determine a heading for the vehicle.