Abstract: A system can apply file placement rules to dynamically place files and directories within file system views backed by objects in an object storage system. After detection of an update to a first file system view that causes an update of an object in a storage grid, an object manager begins evaluation of file placement rules against metadata of the object. For each file placement rule that is triggered, the object manager determines identifies gateways that export the first file system view. The object manager then instructs the gateways to update their representations of the first file system view. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

Abstract: A system can apply file placement rules to dynamically place files and directories within file system views backed by objects in an object storage system. After detection of an update to a first file system view that causes an update of an object in a storage grid, an object manager begins evaluation of file placement rules against metadata of the object. For each file placement rule that is triggered, the object manager determines identifies gateways that export the first file system view. The object manager then instructs the gateways to update their representations of the first file system view. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

Abstract: Methods, non-transitory computer readable media, and devices that more effectively manage snapshots by creating a namespace including files described by stored file recipe objects and each comprising fragments, directories described by metadata in the file recipe objects and each comprising one or more of the files, and snapshots described by stored snapshot request objects. Content of one of the directories are identified based on an identification of corresponding ones of the file recipe objects that existed within the namespace at a specified time. At least one of the files, included in the identified content and existing within the namespace at the specified time, is accessed in response to one or more received requests. A garbage collection operation is periodically performed to delete the recipe objects that are marked for deletion by tombstone objects and are unreferenced by any of the snapshots as determined based on the snapshot request objects.

Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: Methods and systems for a networked computing system are provided. One method includes generating, based on a first topology, a first proxy endpoint by a first device of a first pluggable compute module; establishing a communication tunnel between the first proxy endpoint and a non-volatile memory express (NVMe) storage device for peer-to-peer communication between the first proxy endpoint of the first device and a controller of the NVMe storage device. An NVMe translation module receives a request for the NVMe storage device from the first proxy endpoint and the NVMe translation module translates the request to an NVMe request for the NVMe storage device for accessing storage space at the NVMe storage device. The method further includes de-allocating the first proxy endpoint, when the first topology is deactivated making the first pluggable compute module and the NVMe storage device available for a second topology.

Abstract: Methods, non-transitory computer readable media, and computing devices that facilitate secure cloud compute environments are disclosed. A secure application package (SAP) is encrypted with an SAP encryption key. The encrypted SAP is stored on cloud storage. A profiling bitstream is sent to a cloud provider. The profiling bitstream is configured to, when implemented by the HLD, generate and return a profile response, including a bitstream encryption key, which is encrypted with a public key. The profile response is decrypted using a private key and the bitstream encryption key is extracted. An application bitstream is sent to the cloud provider. The application bitstream is encrypted with the bitstream encryption key, includes the SAP encryption key, and is configured to, when implemented by the HLD, obtain the SAP from the cloud storage, decrypt the SAP using the SAP encryption key, and execute an application in a softcore included in the application bitstream.

Abstract: Methods and systems for a networked computing system are provided. One method includes generating, based on a first topology, a first proxy endpoint by a first device of a first pluggable compute module; establishing a communication tunnel between the first proxy endpoint and a non-volatile memory express (NVMe) storage device for peer-to-peer communication between the first proxy endpoint of the first device and a controller of the NVMe storage device. An NVMe translation module receives a request for the NVMe storage device from the first proxy endpoint and the NVMe translation module translates the request to an NVMe request for the NVMe storage device for accessing storage space at the NVMe storage device. The method further includes de-allocating the first proxy endpoint, when the first topology is deactivated making the first pluggable compute module and the NVMe storage device available for a second topology.

Abstract: Methods and systems for a networked computing system are provided. One method includes generating a first proxy endpoint by a non-transparent bridge (NTB) of a first pluggable compute module and a second proxy endpoint at a second pluggable module having a second NTB, based on a user defined topology; establishing a transaction layer packet (TLP) tunnel between the first proxy endpoint and the second proxy endpoint for peer to peer communication using a first stub endpoint of the first NTB and a second stub endpoint of the second NTB; and de-allocating the first proxy endpoint and the second proxy endpoint, when the topology is deactivated such that the first pluggable compute module and the second pluggable module are available for another user defined topology.

Abstract: Methods, non-transitory computer readable media, and computing devices that facilitate secure cloud compute environments are disclosed. A secure application package (SAP) is encrypted with an SAP encryption key. The encrypted SAP is stored on cloud storage. A profiling bitstream is sent to a cloud provider. The profiling bitstream is configured to, when implemented by the HLD, generate and return a profile response, including a bitstream encryption key, which is encrypted with a public key. The profile response is decrypted using a private key and the bitstream encryption key is extracted. An application bitstream is sent to the cloud provider. The application bitstream is encrypted with the bitstream encryption key, includes the SAP encryption key, and is configured to, when implemented by the HLD, obtain the SAP from the cloud storage, decrypt the SAP using the SAP encryption key, and execute an application in a softcore included in the application bitstream.

Abstract: Methods and systems for a networked computing system are provided. One method includes generating a first proxy endpoint by a non-transparent bridge (NTB) of a first pluggable compute module and a second proxy endpoint at a second pluggable module having a second NTB, based on a user defined topology; establishing a transaction layer packet (TLP) tunnel between the first proxy endpoint and the second proxy endpoint for peer to peer communication using a first stub endpoint of the first NTB and a second stub endpoint of the second NTB; and de-allocating the first proxy endpoint and the second proxy endpoint, when the topology is deactivated such that the first pluggable compute module and the second pluggable module are available for another user defined topology.

Abstract: A method, non-transitory computer readable medium, and device that assists with managing storage in a distributed deduplication system includes receiving an object to be stored from a client computing device. The received object is divided into a plurality of fragments. A plaintext hash value and a ciphertext hash value is determined for each of the plurality of fragments, wherein each of the plurality of fragments is renamed with the corresponding determined ciphertext hash value. Each of the renamed plurality of fragments are stored in a plurality of storage repositories.

Abstract: A storage system includes data storage devices, bus conductors, and mobile reader/writer devices. Each of the storage devices is positioned between insulators, is at one of a plurality of locations on one of one or more shelf assemblies, and comprises a processor coupled to a memory and an interface device. One of the bus conductors is adjacent each of the insulators. Each of the reader/writer devices includes a transport apparatus, a processor and a memory. The transport apparatus is configured to move one of the reader/writer devices to one or more of the locations when engaged. The processor is coupled to the transport apparatus and the memory and is configured to execute machine executable code to: engage the transport apparatus to position one of the reader/writer devices to one of the locations in response to a received operation; couple power to one of the storage devices; and execute the operation.

Abstract: A storage system includes data storage devices, bus conductors, and mobile reader/writer devices. Each of the storage devices is positioned between insulators, is at one of a plurality of locations on one of one or more shelf assemblies, and comprises a processor coupled to a memory and an interface device. One of the bus conductors is adjacent each of the insulators. Each of the reader/writer devices includes a transport apparatus, a processor and a memory. The transport apparatus is configured to move one of the reader/writer devices to one or more of the locations when engaged. The processor is coupled to the transport apparatus and the memory and is configured to execute machine executable code to: engage the transport apparatus to position one of the reader/writer devices to one of the locations in response to a received operation; couple power to one of the storage devices; and execute the operation.

Abstract: Embodiments use data shuttle devices to restore erasure-coded data in a distributed storage environment. In some embodiments, a first data shuttle is communicatively coupled to a first node of the storage environment. On the data shuttle, first restoration data is generated from a first erasure-coded data portion stored on the first node. The first data shuttle or a second data shuttle is communicatively coupled to a second node of the storage environment. On the data shuttle at the second node, second restoration data is generated from a second erasure-coded data portion stored on the second node. Subsequent to transporting the first or second data shuttle from at least one of the other nodes to a third node, a third erasure-coded data portion is restored at the third node. The third erasure-coded data portion is generated via an erasure-coding process from one or more of the first or second restoration data.

Abstract: Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes and at least one latent storage. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

Abstract: Embodiments use data shuttle devices to restore erasure-coded data in a distributed storage environment. In some embodiments, a first data shuttle is communicatively coupled to a first node of the storage environment. On the data shuttle, first restoration data is generated from a first erasure-coded data portion stored on the first node. The first data shuttle or a second data shuttle is communicatively coupled to a second node of the storage environment. On the data shuttle at the second node, second restoration data is generated from a second erasure-coded data portion stored on the second node. Subsequent to transporting the first or second data shuttle from at least one of the other nodes to a third node, a third erasure-coded data portion is restored at the third node. The third erasure-coded data portion is generated via an erasure-coding process from one or more of the first or second restoration data.

Abstract: A flash-based storage warehouse system includes flash memory devices and reader/writer devices that are moveable to different locations in a structure. Each of the flash memory devices are at one of the different locations in the structure. Each of the reader/writer devices includes a locomotion apparatus configured to move the corresponding one of the reader/writer devices to a different location in the structure and a processor coupled to a memory and the locomotion apparatus that is configured to execute machine executable code to: engage the locomotion apparatus to adjustably position one of reader/writer devices to one of the locations in the structure in response to a received operation; couple power via the corresponding one of the reader/writer devices to one of the flash memory devices at the one of the locations in the structure; and execute the operation on the flash memory device at the location in the structure.

Abstract: Some embodiments involve a method of managing power for a first multiple-data-storage-devices enclosure. The method can include: checking out a first token over a network connection from a token pool shared by multiple-data-storage-devices enclosures including the first enclosure, wherein each token of the token pool is available to be checked out by a single device and wherein the enclosures share power drawn from a power supply; after checking out the first token, initiating activation of a data storage device within the first enclosure; monitoring power consumption within the first enclosure; and releasing the first token back to the token pool when the power consumption in the first enclosure substantially reaches a steady-state after the activation of the data storage device is initiated.

Abstract: Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.