Abstract: A method includes obtaining, by a data transform accelerator, a first command and first source data associated with the first command. The method also includes generating, by the data transform accelerator, one or more containers using the first command. Each of the one or more containers may have a container size and may be operable to store compressed source data. The method further includes obtaining a mode of operation for a data compression operation. The method also includes performing the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data. The method further includes storing the compressed source data in a first container of the one or more containers.

Abstract: A method includes determining an address associated with a data transform command in a container data structure which is in a host computing unit. The host computing unit is in communication with the data transform accelerator. In response to a determination that the address is in the container data structure, the method includes accessing the data transform command based on the address. The method also includes obtaining metadata based on information in the data transform command. The metadata is in the host computing unit. The metadata can be shared in its entirety or partially by multiple data transform commands grouped together. The method further includes configuring a data transform pipeline based on the metadata.

Abstract: A method may include identifying an application operable to submit one or more commands to a data transform accelerator. The method may also include determining one or more classes of service utilized with at least one bank of data transform engines in the data transform accelerator. The method may further include estimating a workload to be transmitted to the data transform accelerator. In response to the workload satisfying a threshold and interrupt control being enabled in the at least one bank of data transform engines, the method may also include configuring interrupt control for the one or more classes of service.

Abstract: A method may include obtaining input data to be compressed by a compression operation. The method may also include obtaining metadata associated with the input data to be compressed by the compression operation. The method may further include determining a data threshold of the input data and the metadata to be compressed by the compression operation. The method may also include preprocessing the metadata. The method may further include arranging the input data and the metadata based on the data threshold. The method may also include performing the compression operation on the arranged input data and the arranged metadata.

Abstract: A method includes obtaining, by a data transform accelerator, a first command and first source data associated with the first command. The method also includes generating, by the data transform accelerator, one or more containers using the first command. Each of the one or more containers may have a container size and may be operable to store compressed source data. The method further includes obtaining a mode of operation for a data compression operation. The method also includes performing the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data. The method further includes storing the compressed source data in a first container of the one or more containers.

Abstract: A method may include obtaining multiple tunable parameters associated with a data transform accelerator operable to perform data transform operations. The method may also include configuring a resource configuration vector based on the multiple tunable parameters. The method may further include obtaining a target performance metric. The method may also include measuring one or more performance metrics associated with the data transform accelerator. The method may further include automatically tuning at least one tunable parameter of the multiple tunable parameters to obtain tuned parameters in response to a performance metric of the one or more performance metrics failing to satisfy the target performance metric. The method may also include updating the resource configuration vector in view of the tuned parameters.

Abstract: A method includes determining an address associated with a data transform command in a container data structure which is in the data transform accelerator. The data transform accelerator is in communication with a host computing unit. In response to a determination that the address is in the container data structure, the method includes accessing the data transform command based on the address. The data transform command is in the host computing unit. The method includes obtaining metadata based on information in the data transform command. The metadata is in the data transform accelerator or spread out in the host computing unit memory and in the memory of data transform accelerator. The method includes configuring a data transform pipeline based on the metadata. The metadata can be shared in its entirety or partially by multiple data transform commands grouped together.

Abstract: A method includes determining an address associated with a data transform command in a container data structure which is in the data transform accelerator. The data transform accelerator is in communication with a host computing unit. In response to a determination that the address is in the container data structure, the method includes accessing the data transform command based on the address. The data transform command is in the host computing unit. The method includes obtaining metadata based on information in the data transform command. The metadata is in the data transform accelerator or spread out in the host computing unit memory and in the memory of data transform accelerator. The method includes configuring a data transform pipeline based on the metadata. The metadata can be shared in its entirety or partially by multiple data transform commands grouped together.

Abstract: A system for calculating a fingerprint across a data set by identifying a data set to hash, the data set comprising a set of data blocks, identifying data within the data set to skip, generating, by a hash engine, a hash for each data block in the set of data blocks within the data set except for the data within the data set to skip, and compressing the data.

Abstract: A method includes obtaining multiple command requests, each including a command address. The method also includes performing a load balancing operation to select a first container of multiple containers to store a first command address. The method further includes storing the first command address in the first data container. The method also includes transmitting the first command address from the first data container to a data transform accelerator. The method further includes obtaining transformed data from the data transform accelerator.

Abstract: A method includes obtaining, by a data transform accelerator, a first command and first source data associated with the first command. The method also includes generating, by the data transform accelerator, one or more containers using the first command. Each of the one or more containers may have a container size and may be operable to store compressed source data. The method further includes obtaining a mode of operation for a data compression operation. The method also includes performing the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data. The method further includes storing the compressed source data in a first container of the one or more containers.

Abstract: A system for calculating a fingerprint across a data set by identifying a data set to hash, the data set comprising a set of data blocks, identifying data within the data set to skip, generating, by a hash engine, a hash for each data block in the set of data blocks within the data set except for the data within the data set to skip, and compressing the data.

Abstract: A method includes obtaining, by a data transform accelerator, first metadata. The method also includes configuring a first pipeline in the data transform accelerator using the first metadata. The method further includes obtaining input data to be transformed by the data transform accelerator. The method also includes generating encoded data and second metadata in the first pipeline using the input data and the first metadata. The encoded data and the second metadata may be stored together for later operations, such as decode operations. The method further includes configuring a second pipeline in the data transform accelerator using the second metadata.

Abstract: A system for calculating a fingerprint across a data set by identifying a data set to hash, the data set comprising a set of data blocks, generating, by a first hash engine, a first hash for each data block in the set of data blocks within the data set, and generating, by a second hash engine, a second hash for each data block in the set of data blocks within the data set.

Abstract: A method includes obtaining an address associated with a data transform command. The address may be disposed in a container located in a first memory. The method also includes obtaining metadata associated with the data transform command. The data transform command may be disposed in the first memory and may be pointed to by the address. A first portion of the metadata may be public data and a second portion of the metadata may be sensitive data. The method further includes storing the public data in the first memory and the sensitive data in a second memory disposed internally to a data transform accelerator. The method also includes configuring a data transform pipeline in the data transform accelerator based on the public data in the first memory and the sensitive data in the second memory.

Abstract: A method includes initializing a virtual machine including a virtual machine memory disposed in memory of a data transform accelerator. The method also includes obtaining an address associated with a data transform command. The address may be disposed in a container located in a first partition of the virtual machine memory. The method also includes obtaining metadata associated with the data transform command. A first portion of the metadata may be public data and a second portion of the metadata may be sensitive data. The method further includes storing the public data in the first partition and the sensitive data in a second partition of the virtual machine memory. The method also includes configuring a data transform pipeline in the data transform accelerator based on the public data in the first partition and the sensitive data in the second partition.

Abstract: A method includes determining a communication interface between a host computing unit and a data transform accelerator, hosting one or more virtual machines on a host operating system of the host computing unit; partitioning a collection of container data structures into multiple sets; partitioning a memory of the data transform accelerator into multiple partitions; submitting one or more data transform commands for processing by the data transform accelerator device; in response to a determination that an address is in a particular container data structure, accessing the data transform command based on the address; obtaining metadata based on information in the data transform command; and configuring a data transform pipeline based on the metadata.

Abstract: A method includes determining an address associated with a data transform command in a container data structure which is in a host computing unit. The host computing unit is in communication with the data transform accelerator. In response to a determination that the address is in the container data structure, the method includes accessing the data transform command based on the address. The method also includes obtaining metadata based on information in the data transform command. The metadata is in the host computing unit. The metadata can be shared in its entirety or partially by multiple data transform commands grouped together. The method further includes configuring a data transform pipeline based on the metadata.

Abstract: A method includes determining an address associated with a data transform command in a container data structure which is in the data transform accelerator. The data transform accelerator is in communication with a host computing unit. In response to a determination that the address is in the container data structure, the method includes accessing the data transform command based on the address. The data transform command is in the host computing unit. The method includes obtaining metadata based on information in the data transform command. The metadata is in the data transform accelerator or spread out in the host computing unit memory and in the memory of data transform accelerator. The method includes configuring a data transform pipeline based on the metadata. The metadata can be shared in its entirety or partially by multiple data transform commands grouped together.

Abstract: A system for calculating a fingerprint across a data set by identifying a data set to hash, the data set comprising a set of data blocks, generating, by a first hash engine, a first hash for each data block in the set of data blocks within the data set, and generating, by a second hash engine, a second hash for each data block in the set of data blocks within the data set.