Abstract: Implementations detailed herein include description of a computer-implemented method.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Systems and methods that allow secure application-driven arbitrary compute in storage devices in a cloud-based computing system are provided. A computing system including a compute controller configured to: (1) provide access to host compute resources, and (2) operate in at least one of a first mode or a second mode is provided. The computing system may further include a storage controller configured to provide access to storage systems including storage components, at least one compute component, and at least one cryptographic component. In the first mode, the host compute resources may be configured to execute at least a first operation on at least a first set of data stored in at least one of the storage components. In the second mode, the at least one compute component may be configured to execute at least a second operation on at least a second set of data.

Abstract: Methods and systems enable a database reorganization to occur without a database outage. In one aspect, the method includes pausing transactions directed to the database, keeping a logical view of the database online. The method may also include taking individual partitions offline, changing the names of datasets associated with the individual partitions in a database schema, and bringing the partitions online, all while the logical view of the database remains online. The database schema may be changed to reflect the name of datasets associated with a shadow copy of the database that has been reorganized.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Deduplication is integrated with software building and chunk storing. A dedup module includes dedup software, a build graph interface, and a chunk store interface. A dedup graph includes a portion of the build graph, and a portion that represents build artifact file chunks. The dedup software queries whether chunks are present in the chunk store, submits a chunk for storage when the chunk is not already present, and avoids submitting the chunk when it is present. Queries may use hash comparisons, a hash tree dedup graph, chunk expiration dates, content addressable chunk store memory, inference of a child node's presence, recursion, and a local cache of node hashes and node expiration dates, for example. A change caused by the build impacts fewer dedup graph nodes than directory graph nodes, resulting in fewer storage operations to update the chunk storage with new or changed build artifacts.

Abstract: Deduplication is integrated with software building and chunk storing. A dedup module includes dedup software, a build graph interface, and a chunk store interface. A dedup graph includes a portion of the build graph, and a portion that represents build artifact file chunks. The dedup software queries whether chunks are present in the chunk store, submits a chunk for storage when the chunk is not already present, and avoids submitting the chunk when it is present. Queries may use hash comparisons, a hash tree dedup graph, chunk expiration dates, content addressable chunk store memory, inference of a child node's presence, recursion, and a local cache of node hashes and node expiration dates, for example. A change caused by the build impacts fewer dedup graph nodes than directory graph nodes, resulting in fewer storage operations to update the chunk storage with new or changed build artifacts.

Abstract: A transaction engine includes a multi-version concurrency control (MVCC) module that accesses a latch-free hash table that includes respective hash table entries that include respective buckets of respective bucket items. The bucket items represent respective records, the respective bucket items each including a value indicating a temporal most recent read time of the item and a version list of descriptions that describe respective versions of the respective records, the MVCC module performing timestamp order concurrency control, using the latch-free hash table. Recovery log buffers may be used as cache storage for the transaction engine.

Abstract: Systems and methods that allow secure application-driven arbitrary compute in storage devices in a cloud-based computing system are provided. A computing system including a compute controller configured to: (1) provide access to host compute resources, and (2) operate in at least one of a first mode or a second mode is provided. The computing system may further include a storage controller configured to provide access to storage systems including storage components, at least one compute component, and at least one cryptographic component. In the first mode, the host compute resources may be configured to execute at least a first operation on at least a first set of data stored in at least one of the storage components. In the second mode, the at least one compute component may be configured to execute at least a second operation on at least a second set of data.

Abstract: The subject disclosure is directed towards a data deduplication technology in which a hash index service's index is partitioned into subspace indexes, with less than the entire hash index service's index cached to save memory. The subspace index is accessed to determine whether a data chunk already exists or needs to be indexed and stored. The index may be divided into subspaces based on criteria associated with the data to index, such as file type, data type, time of last usage, and so on. Also described is subspace reconciliation, in which duplicate entries in subspaces are detected so as to remove entries and chunks from the deduplication system. Subspace reconciliation may be performed at off-peak time, when more system resources are available, and may be interrupted if resources are needed. Subspaces to reconcile may be based on similarity, including via similarity of signatures that each compactly represents the subspace's hashes.

Abstract: The subject disclosure is directed towards using primary data deduplication concepts for more efficient access of data via content addressable caches. Chunks of data, such as deduplicated data chunks, are maintained in a fast access client-side cache, such as containing chunks based upon access patterns. The chunked content is content addressable via a hash or other unique identifier of that content in the system. When a chunk is needed, the client-side cache (or caches) is checked for the chunk before going to a file server for the chunk. The file server may likewise maintain content addressable (chunk) caches. Also described are cache maintenance, management and organization, including pre-populating caches with chunks, as well as using RAM and/or solid-state storage device caches.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Methods and systems enable a database reorganization to occur without a database outage. In one aspect, the method includes pausing transactions directed to the database, keeping a logical view of the database online. The method may also include taking individual partitions offline, changing the names of datasets associated with the individual partitions in a database schema, and bringing the partitions online, all while the logical view of the database remains online. The database schema may be changed to reflect the name of datasets associated with a shadow copy of the database that has been reorganized.

Abstract: The subject disclosure is directed towards a data deduplication technology in which a hash index service's index and/or indexing operations are adaptable to balance deduplication performance savings, throughput and resource consumption. The indexing service may employ hierarchical chunking using different levels of granularity corresponding to chunk size, a sampled compact index table that contains compact signatures for less than all of the hash index's (or subspace's) hash values, and/or selective subspace indexing based on similarity of a subspace's data to another subspace's data and/or to incoming data chunks.

Abstract: Methods and systems enable a database reorganization to occur without a database outage. In one aspect, the method includes pausing transactions directed to the database, keeping a logical view of the database online. The method may also include taking individual partitions offline, changing the names of datasets associated with the individual partitions in a database schema, and bringing the partitions online, all while the logical view of the database remains online. The database schema may be changed to reflect the name of datasets associated with a shadow copy of the database that has been reorganized.

Abstract: A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records.

Abstract: Update requests that specify updates to a logical page associated with a key-value store are obtained. Updates to the logical page are posted using the obtained plurality of update requests, without accessing the logical page via a read operation.

Abstract: The subject disclosure is directed towards encryption and deduplication integration between computing devices and a network resource. Files are partitioned into data blocks and deduplicated via removal of duplicate data blocks. Using multiple cryptographic keys, each data block is encrypted and stored at the network resource but can only be decrypted by an authorized user, such as domain entity having an appropriate deduplication domain-based cryptographic key. Another cryptographic key referred to as a content-derived cryptographic key ensures that duplicate data blocks encrypt to substantially equivalent encrypted data.

Abstract: This patent application relates to an agile network architecture that can be employed in data centers, among others. One implementation provides a virtual layer-2 network connecting machines of a layer-3 infrastructure.

Abstract: In various embodiments, methods and systems are disclosed for a hybrid rate plus window based congestion protocol that controls the rate of packet transmission into the network and provides low queuing delay, practically zero packet loss, fair allocation of network resources amongst multiple flows, and full link utilization. In one embodiment, a congestion window may be used to control the maximum number of outstanding bits, a transmission rate may be used to control the rate of packets entering the network (packet pacing), a queuing delay based rate update may be used to control queuing delay within tolerated bounds and minimize packet loss, and aggressive ramp-up/graceful back-off may be used to fully utilize the link capacity and additive-increase, multiplicative-decrease (AIMD) rate control may be used to provide fairness amongst multiple flows.