Abstract: Implementations detailed herein include description of a computer-implemented method to migrate a machine learning model from one accelerator portion (such as a portion of a graphical processor unit (GPU)) to a different accelerator portion. In some instances, a state of the first accelerator portion is persisted, the second accelerator portion is configured, the first accelerator portion is then detached from a client application instance, and at least a portion of an inference request is performed using the loaded at least a portion of the machine learning model on the second accelerator portion that had been configured.

Abstract: Described is using flash memory (or other secondary storage), RAM-based data structures and mechanisms to access key-value pairs stored in the flash memory using only a low RAM space footprint. A mapping (e.g. hash) function maps key-value pairs to a slot in a RAM-based index. The slot includes a pointer that points to a bucket of records on flash memory that each had keys that mapped to the slot. The bucket of records is arranged as a linear-chained linked list, e.g., with pointers from the most-recently written record to the earliest written record. Also described are compacting non-contiguous records of a bucket onto a single flash page, and garbage collection. Still further described is load balancing to reduce variation in bucket sizes, using a bloom filter per slot to avoid unnecessary searching, and splitting a slot into sub-slots.

Abstract: Described is using flash memory (or other secondary storage), RAM-based data structures and mechanisms to access key-value pairs stored in the flash memory using only a low RAM space footprint. A mapping (e.g. hash) function maps key-value pairs to a slot in a RAM-based index. The slot includes a pointer that points to a bucket of records on flash memory that each had keys that mapped to the slot. The bucket of records is arranged as a linear-chained linked list, e.g., with pointers from the most-recently written record to the earliest written record. Also described are compacting non-contiguous records of a bucket onto a single flash page, and garbage collection. Still further described is load balancing to reduce variation in bucket sizes, using a bloom filter per slot to avoid unnecessary searching, and splitting a slot into sub-slots.

Abstract: Implementations detailed herein include description of a computer-implemented method.

Abstract: Implementations detailed herein include description of a computer-implemented method.

Abstract: Implementations detailed herein include description of a computer-implemented method.

Abstract: Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including: an indication of the central processing unit (CPU) capability to be used, an arithmetic precision of the machine learning model to be used, an indication of the accelerator capability to be used, a storage location of the application, and an indication of an amount of random access memory to use.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Methods and systems enable a database reorganization to occur without a database outage. In one aspect, the method includes pausing transactions directed to the database, keeping a logical view of the database online. The method may also include taking individual partitions offline, changing the names of datasets associated with the individual partitions in a database schema, and bringing the partitions online, all while the logical view of the database remains online. The database schema may be changed to reflect the name of datasets associated with a shadow copy of the database that has been reorganized.

Abstract: Systems and methods that allow secure application-driven arbitrary compute in storage devices in a cloud-based computing system are provided. A computing system including a compute controller configured to: (1) provide access to host compute resources, and (2) operate in at least one of a first mode or a second mode is provided. The computing system may further include a storage controller configured to provide access to storage systems including storage components, at least one compute component, and at least one cryptographic component. In the first mode, the host compute resources may be configured to execute at least a first operation on at least a first set of data stored in at least one of the storage components. In the second mode, the at least one compute component may be configured to execute at least a second operation on at least a second set of data.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Deduplication is integrated with software building and chunk storing. A dedup module includes dedup software, a build graph interface, and a chunk store interface. A dedup graph includes a portion of the build graph, and a portion that represents build artifact file chunks. The dedup software queries whether chunks are present in the chunk store, submits a chunk for storage when the chunk is not already present, and avoids submitting the chunk when it is present. Queries may use hash comparisons, a hash tree dedup graph, chunk expiration dates, content addressable chunk store memory, inference of a child node's presence, recursion, and a local cache of node hashes and node expiration dates, for example. A change caused by the build impacts fewer dedup graph nodes than directory graph nodes, resulting in fewer storage operations to update the chunk storage with new or changed build artifacts.

Abstract: Deduplication is integrated with software building and chunk storing. A dedup module includes dedup software, a build graph interface, and a chunk store interface. A dedup graph includes a portion of the build graph, and a portion that represents build artifact file chunks. The dedup software queries whether chunks are present in the chunk store, submits a chunk for storage when the chunk is not already present, and avoids submitting the chunk when it is present. Queries may use hash comparisons, a hash tree dedup graph, chunk expiration dates, content addressable chunk store memory, inference of a child node's presence, recursion, and a local cache of node hashes and node expiration dates, for example. A change caused by the build impacts fewer dedup graph nodes than directory graph nodes, resulting in fewer storage operations to update the chunk storage with new or changed build artifacts.

Abstract: A transaction engine includes a multi-version concurrency control (MVCC) module that accesses a latch-free hash table that includes respective hash table entries that include respective buckets of respective bucket items. The bucket items represent respective records, the respective bucket items each including a value indicating a temporal most recent read time of the item and a version list of descriptions that describe respective versions of the respective records, the MVCC module performing timestamp order concurrency control, using the latch-free hash table. Recovery log buffers may be used as cache storage for the transaction engine.

Abstract: Systems and methods that allow secure application-driven arbitrary compute in storage devices in a cloud-based computing system are provided. A computing system including a compute controller configured to: (1) provide access to host compute resources, and (2) operate in at least one of a first mode or a second mode is provided. The computing system may further include a storage controller configured to provide access to storage systems including storage components, at least one compute component, and at least one cryptographic component. In the first mode, the host compute resources may be configured to execute at least a first operation on at least a first set of data stored in at least one of the storage components. In the second mode, the at least one compute component may be configured to execute at least a second operation on at least a second set of data.

Abstract: The subject disclosure is directed towards a data deduplication technology in which a hash index service's index is partitioned into subspace indexes, with less than the entire hash index service's index cached to save memory. The subspace index is accessed to determine whether a data chunk already exists or needs to be indexed and stored. The index may be divided into subspaces based on criteria associated with the data to index, such as file type, data type, time of last usage, and so on. Also described is subspace reconciliation, in which duplicate entries in subspaces are detected so as to remove entries and chunks from the deduplication system. Subspace reconciliation may be performed at off-peak time, when more system resources are available, and may be interrupted if resources are needed. Subspaces to reconcile may be based on similarity, including via similarity of signatures that each compactly represents the subspace's hashes.

Abstract: The subject disclosure is directed towards using primary data deduplication concepts for more efficient access of data via content addressable caches. Chunks of data, such as deduplicated data chunks, are maintained in a fast access client-side cache, such as containing chunks based upon access patterns. The chunked content is content addressable via a hash or other unique identifier of that content in the system. When a chunk is needed, the client-side cache (or caches) is checked for the chunk before going to a file server for the chunk. The file server may likewise maintain content addressable (chunk) caches. Also described are cache maintenance, management and organization, including pre-populating caches with chunks, as well as using RAM and/or solid-state storage device caches.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Methods and systems enable a database reorganization to occur without a database outage. In one aspect, the method includes pausing transactions directed to the database, keeping a logical view of the database online. The method may also include taking individual partitions offline, changing the names of datasets associated with the individual partitions in a database schema, and bringing the partitions online, all while the logical view of the database remains online. The database schema may be changed to reflect the name of datasets associated with a shadow copy of the database that has been reorganized.

Abstract: The subject disclosure is directed towards a data deduplication technology in which a hash index service's index and/or indexing operations are adaptable to balance deduplication performance savings, throughput and resource consumption. The indexing service may employ hierarchical chunking using different levels of granularity corresponding to chunk size, a sampled compact index table that contains compact signatures for less than all of the hash index's (or subspace's) hash values, and/or selective subspace indexing based on similarity of a subspace's data to another subspace's data and/or to incoming data chunks.