Abstract: A method for generating schemas to compress storage of denormalized data includes receiving user selection of an initial data table and constructing a snowflake schema to decompose the initial data table into a plurality of tables. Tables in the plurality of tables that are related to one another are connected by foreign keys. The snowflake schema is constructed according to a cost model that balances the cost of storing the plurality of tables and the cost of storing the foreign keys. The method also includes transforming data in the initial data table into the plurality of tables according to the snowflake schema.

Abstract: A method selects a federation engine to process a query. The system receives a query that specifies retrieval of data from multiple data sources at distinct physical locations. The system generates multiple federated query plans. Each federated query plan specifies execution of the query using a respective data source as the federation engine. For each federated query plan, the system forms a feature vector including two or more of: (1) a remote cost estimate for executing query operators on data sources distinct from the respective data source; (2) a local cost estimate for executing query operators on the respective data source; and (3) a data transfer cost estimate for transferring partial results from external data sources to the respective data source. The system predicts runtime for each federated query plan, selects a preferred federated query plan using the predicted runtimes, and the preferred federated query plan specifies the federation engine.

Abstract: A system for determining and merging differentially private approximate distinct-counting sketches is disclosed. A first non-private probabilistic cardinality estimator for a first dataset is determined. The first non-private probabilistic cardinality estimator is converted to a first private probabilistic cardinality estimator for the first dataset with a first noise level. The first private probabilistic cardinality estimator for the first dataset is merged with a second probabilistic cardinality estimator for a second dataset with a second noise level to produce a merged probabilistic cardinality estimator for the first dataset and the second dataset combined together based at least in part on the first noise level and the second noise level. A number of unique elements in the first dataset and the second dataset combined together is estimated based on the merged probabilistic cardinality estimator for the first dataset and the second dataset combined together.

Abstract: A method selects a federation engine to process a query. The system receives a query that specifies retrieval of data from multiple data sources at distinct physical locations. The system generates multiple federated query plans. Each federated query plan specifies execution of the query using a respective data source as the federation engine. For each federated query plan, the system forms a feature vector including two or more of: (1) a remote cost estimate for executing query operators on data sources distinct from the respective data source; (2) a local cost estimate for executing query operators on the respective data source; and (3) a data transfer cost estimate for transferring partial results from external data sources to the respective data source. The system predicts runtime for each federated query plan, selects a preferred federated query plan using the predicted runtimes, and the preferred federated query plan specifies the federation engine.

Abstract: A method builds a regression model for predicting processing times for federated queries using a variety of data sources. The method includes obtaining federated queries (e.g., from benchmarks), and generates a plurality of federated query plans for each federated query. Each federated query plan corresponds to executing a respective federated query using a respective data source as the federation engine. The method includes forming feature vectors for each federated query plan based on cost estimations for executing the respective federated query plan and cost estimations for data transfer. The method further includes training a regression model, using the feature vectors for the plurality of federated query plans, to predict runtimes for executing federated queries using the variety of data sources as a federation engine. Some implementations use the trained regression model to determine a suitable federation engine for a given federated query.

Abstract: A method samples a stream of data items. Each data item has an associated timestamp. The method assigns a priority value to each data item. Each data item is represented as a point on a two-dimensional graph whose axes are time and priority. A sliding window covers a predetermined length of time tspan and uses a backward probability decay curve to specify what priority values are included in the sliding window. This defines, for a current time tc, a current data sample consisting of data items whose timestamps t fall within the time span tc?tspan?t?tc and have priority values below the decay curve. The data sample is stored in a buffer. The process iteratively moves the sliding window forward by a time increment, creating a provisional data sample. When the size of the provisional data sample is too large or too small, the process scales the decay curve.

Abstract: Data items are aggregated in a quantile sketch according to a compactor hierarchy. A first number of data items are stored in a first buffer corresponding to a first (i.e., lowest) compaction level of the compactor hierarchy, and a top level number of data items are stored in a top buffer corresponding to a top compaction level of the compactor hierarchy. For a first subset of the first number of data items, each data item is already stored in the top buffer, and a weight of the respective data item stored in the top buffer is increased by 1. For a second subset of the first number of data items, each data item does not match any data item stored in the top buffer, and a second number of data items are selected from the second subset and stored in a second buffer of a second compaction level.

Abstract: A method of modifying a retina fundus image for a deep learning model is disclosed. In a described embodiment, the method includes converting a retina fundus image to a binary image by converting pixels of the retina fundus image to low intensity modified pixels and high intensity modified pixels of the binary image, and determining a first boundary between the low intensity modified pixels and high intensity modified pixels of the binary image. The method further includes removing outlier boundary values from the first boundary, constructing a second boundary from remaining boundary values, identifying the pixels of the retina fundus image that are within the second boundary, and constructing a modified retina fundus image containing the identified pixels for a deep learning model.

Abstract: A method estimates counts of distinct items. A system allocates a matrix B in memory and receives a data stream including many data items. Each item includes a set of query fields and a set of identifier fields. For each item, the system computes a row for the data item by hashing data values of the identifier fields. The system also computes a column for the data item by hashing a combination of data values of the query fields and the row. The system computes a bin value for the item by hashing data values of the identifier fields. The system updates the matrix B by computing B[r,c]=max(v, B[r,c]). Subsequently, the system receives a query key value. For each row in the matrix B, the system computes a column, and estimates the number of data items from the data stream matching the query key.

Abstract: Methods, non-transitory computer readable media, and computing devices that accelerate data access requests. With this technology, a hierarchy of a plurality of objects is inserted into a location database. Each of at least a subset of the plurality of objects comprises a physical storage location for data stored in a filesystem. One or more of the plurality of objects includes an object version number and a parent version number of a parent one of the plurality of objects. A determination is made when an invalidation event has occurred in the filesystem. The invalidation event is associated with one of the plurality of objects. The object version number for the one of the plurality of objects is modified to invalidate one or more of the subset of the objects, when the determining indicates that the invalidation event has occurred in the filesystem.

Abstract: This application is directed to modifying display of data visualizations. A computer system visualizes on its display a set of data in a first data space. A focus point is identified in the first data space and used as a starting point for constructing a set of geodesics, each of which is configured to connect the focus point to a respective data point in the first data space. A plurality of primary eigenvectors are determined in the first data space and they are used to create a second data space having fewer dimensions than the first data space. For each data point, the computer system determines a plurality of projection values associated with the primary eigenvectors. The set of data is re-visualized in the second data space created using the projection values of each data point.

Abstract: A method samples a stream of data items. Each data item has an associated timestamp. The method assigns a priority value to each data item. Each data item is represented as a point on a two-dimensional graph whose axes are time and priority. A sliding window covers a predetermined length of time tspan and uses a backward probability decay curve to specify what priority values are included in the sliding window. This defines, for a current time tc, a current data sample consisting of data items whose timestamps t fall within the time span tc?tspan?t?tc and have priority values below the decay curve. The data sample is stored in a buffer. The process iteratively moves the sliding window forward by a time increment, creating a provisional data sample. When the size of the provisional data sample is too large or too small, the process scales the decay curve.

Abstract: A method of modifying a retina fundus image for a deep learning model is disclosed. In a described embodiment, the method includes converting a retina fundus image to a binary image by converting pixels of the retina fundus image to low intensity modified pixels and high intensity modified pixels of the binary image, and determining a first boundary between the low intensity modified pixels and high intensity modified pixels of the binary image. The method further includes removing outlier boundary values from the first boundary, constructing a second boundary from remaining boundary values, identifying the pixels of the retina fundus image that are within the second boundary, and constructing a modified retina fundus image containing the identified pixels for a deep learning model.

Abstract: A process creates a data sketch for a data set with many rows. A user selects data fields from the data source. The process allocates storage for N bins, where each bin has storage space for a key value and an associated counter value (which is initialized to zero). The process sequentially accesses the rows from the data source (e.g., as a stream). For each row, the process computes a respective key value using data values for the selected data fields. When the respective key value matches a key value for a respective bin, the process increments the counter value for the respective bin. Otherwise, the process identifies a respective bin with a smallest counter value c. The process increments the counter value of the respective bin, and with probability 1/(1+c), replaces the key value of the respective bin with the respective key value.

Abstract: A method samples a stream of data items. Each data item has an associated timestamp. The method assigns a priority value to each data item. Each data item is represented as a point on a two-dimensional graph whose axes are time and priority. A sliding window covers a predetermined length of time tspan and uses a backward probability decay curve to specify what priority values are included in the sliding window. This defines, for a current time tc, a current data sample consisting of data items whose timestamps t fall within the time span tc?tspan?t?tc and have priority values below the decay curve. The data sample is stored in a buffer. The process iteratively moves the sliding window forward by a time increment, creating a provisional data sample. When the size of the provisional data sample is too large or too small, the process scales the decay curve.

Abstract: Methods, non-transitory computer readable media, and computing devices that accelerate data access requests. With this technology, a hierarchy of a objects is inserted into a location database. Each of at least a subset of the objects comprises a physical storage location for data stored in a filesystem. One or more of the objects includes an object version number and a parent version number of a parent one of the objects. A determination is made when an invalidation event has occurred in the filesystem. The invalidation event is associated with one of the objects. The object version number for the one of the objects is modified to invalidate one or more of the subset of the objects, when the determining indicates that the invalidation event has occurred in the filesystem.

Abstract: A process creates a data sketch for a data set with many rows. A user selects data fields from the data source. The process allocates storage for N bins, where each bin has storage space for a key value and an associated counter value (which is initialized to zero). The process sequentially accesses the rows from the data source (e.g., as a stream). For each row, the process computes a respective key value using data values for the selected data fields. When the respective key value matches a key value for a respective bin, the process increments the counter value for the respective bin. Otherwise, the process identifies a respective bin with a smallest counter value c. The process increments the counter value of the respective bin, and with probability 1/(1+c), replaces the key value of the respective bin with the respective key value.

Abstract: Broadband reflectors include a UV-reflective multilayer optical film and a VIS/IR-reflective layer. In various embodiments, the VIS/IR reflective layer may be a reflective metal layer or a multilayer optical film. Concentrated solar power systems and methods of harnessing solar energy using the broadband reflectors and optionally comprising a celestial tracking mechanism are also disclosed.

Abstract: A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI's image of the state of the file system to a deduplicated state.

Abstract: A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI's image of the state of the file system to a deduplicated state.