Abstract: A system and method for executing user-provided code securely on a solid state drive (SSD) to perform data processing on the SSD. In one embodiment, a user uses a security-oriented cross-compiler to compile user-provided source code for a data processing task on a host computer containing, or otherwise connected to, an SSD. The resulting binary is combined with lists of input and output file identifiers and sent to the SSD. A central processing unit (CPU) on the SSD extracts the binary and the lists of file identifiers. The CPU obtains from the host file system the addresses of storage areas in the SSD containing the data in the input files, reads the input data, executes the binary using a container, and writes the results of the data processing task back to the SSD, in areas corresponding to the output file identifiers.

Abstract: A system and method for combining the execution of a query with other operations, such as a data retention scan, in a storage device, when the execution of the query is not time-sensitive. The storage device may be connected to a host, and may operate during intervals of time in a power save mode. When a query is received by the host that is not time-sensitive, the query may be stored in the host or in the storage device until such time as the device would otherwise return to a normal operating mode, and then the query may be executed. Such delayed execution may enable the sharing of read operations for the query with read operations used, for example, for the execution of other queries or for a data retention scan.

Abstract: A system and method for providing consistent performance in a storage device, such as a solid state drive. A threshold value for command execution time for a command in a category of command (e.g., a read command or a write command) and a command size, is stored in the storage device. When a host command in the category (e.g., a read command) and corresponding size is received, the storage device executes the command, and if it completes execution of the command in a time that is less than the threshold value, the solid state drive waits until an amount of time equal to the threshold value has elapsed before sending the command completion.

Abstract: A system and method for executing user-provided code securely on a solid state drive (SSD) to perform data processing on the SSD. In one embodiment, a user uses a security-oriented cross-compiler to compile user-provided source code for a data processing task on a host computer containing, or otherwise connected to, an SSD. The resulting binary is combined with lists of input and output file identifiers and sent to the SSD. A central processing unit (CPU) on the SSD extracts the binary and the lists of file identifiers. The CPU obtains from the host file system the addresses of storage areas in the SSD containing the data in the input files, reads the input data, executes the binary using a container, and writes the results of the data processing task back to the SSD, in areas corresponding to the output file identifiers.

Abstract: A method of writing multi-stream host data to a storage device comprising a CPU, a multi-stream fast release buffer (FRB), and a non-volatile memory (NVM), includes: receiving a command to write the multi-stream host data to the NVM, the multi-stream host data being associated with a logical block number (LBN) and a new stream ID, recording a status of the active stream ID and retrieving a status of the new stream ID to determine a physical address in the NVM for storing one or more codewords (CWs) corresponding to the multi-stream host data, allocating space in a buffer of the FRB for storage of the multi-stream host data, organizing the multi-stream host data into the one or more CWs, and storing the one or more CWs into the allocated space in the buffer, transferring the one or more CWs from the buffer to the physical address in the NVM.

Abstract: A system and method of providing enhanced data processing and analysis in an infrastructure for distributed computing and large-scale data processing. This infrastructure uses the Apache Spark framework to divide an application into a large number of small fragments of work, each of which may be performed on one of a large number of compute nodes. The work may involve Spark transformations, operations, and actions, which may be used to categorize and analyze large amounts of data in distributed systems. This infrastructure includes a cluster with a driver node and a plurality of worker nodes. The worker nodes may be, or may include, intelligent solid state drives capable of executing data processing functions under the Apache Spark framework. The use of intelligent solid state drives reduces the need to exchange data with a central processing unit (CPU) in a server.

Abstract: A system and method for executing user-provided code securely on a solid state drive (SSD) to perform data processing on the SSD. In one embodiment, a user uses a security-oriented cross-compiler to compile user-provided source code for a data processing task on a host computer containing, or otherwise connected to, an SSD. The resulting binary is combined with lists of input and output file identifiers and sent to the SSD. A central processing unit (CPU) on the SSD extracts the binary and the lists of file identifiers. The CPU obtains from the host file system the addresses of storage areas in the SSD containing the data in the input files, reads the input data, executes the binary using a container, and writes the results of the data processing task back to the SSD, in areas corresponding to the output file identifiers.

Abstract: A method for memory-efficient backup of a logical-to-physical (LtoP) table in a storage device having a processor, a volatile memory, and a non-volatile memory, includes retrieving and subsequently processing, by the processor, a plurality of flushed zone segments stored in a first block stripe of the non-volatile memory in an order opposite to an order of storage of the plurality of flushed zone segments in the first block stripe in the non-volatile memory, wherein the processing of the plurality of flushed zone segments includes identifying a retrieved flushed zone segment of the plurality of flushed zone segments differing from all previously retrieved plurality of flushed zone segments, and writing the identified retrieved flushed zone segment of the plurality of flushed zone segments to a second block stripe of the non-volatile memory.

Abstract: A system and method for providing consistent performance in a storage device, such as a solid state drive. A threshold value for command execution time for a command in a category of command (e.g., a read command or a write command) and a command size, is stored in the storage device. When a host command in the category (e.g., a read command) and corresponding size is received, the storage device executes the command, and if it completes execution of the command in a time that is less than the threshold value, the solid state drive waits until an amount of time equal to the threshold value has elapsed before sending the command completion.

Abstract: A system and method for providing consistent performance in a storage device, such as a solid state drive. A threshold value for command execution time for a command in a category of command (e.g., a read command or a write command) and a command size, is stored in the storage device. When a host command in the category (e.g., a read command) and corresponding size is received, the storage device executes the command, and if it completes execution of the command in a time that is less than the threshold value, the solid state drive waits until an amount of time equal to the threshold value has elapsed before sending the command completion.

Abstract: A system and method of providing enhanced data processing and analysis in an infrastructure for distributed computing and large-scale data processing. This infrastructure uses the Apache Spark framework to divide an application into a large number of small fragments of work, each of which may be performed on one of a large number of compute nodes. The work may involve Spark transformations, operations, and actions, which may be used to categorize and analyze large amounts of data in distributed systems. This infrastructure includes a cluster with a driver node and a plurality of worker nodes. The worker nodes may be, or may include, intelligent solid state drives capable of executing data processing functions under the Apache Spark framework. The use of intelligent solid state drives reduces the need to exchange data with a central processing unit (CPU) in a server.

Abstract: A system and method for combining the execution of a query with other operations, such as a data retention scan, in a storage device, when the execution of the query is not time-sensitive. The storage device may be connected to a host, and may operate during intervals of time in a power save mode. When a query is received by the host that is not time-sensitive, the query may be stored in the host or in the storage device until such time as the device would otherwise return to a normal operating mode, and then the query may be executed. Such delayed execution may enable the sharing of read operations for the query with read operations used, for example, for the execution of other queries or for a data retention scan.

Abstract: A method of writing multi-stream host data to a storage device comprising a CPU, a multi-stream fast release buffer (FRB), and a non-volatile memory (NVM), includes: receiving a command to write the multi-stream host data to the NVM, the multi-stream host data being associated with a logical block number (LBN) and a new stream ID, recording a status of the active stream ID and retrieving a status of the new stream ID to determine a physical address in the NVM for storing one or more codewords (CWs) corresponding to the multi-stream host data, allocating space in a buffer of the FRB for storage of the multi-stream host data, organizing the multi-stream host data into the one or more CWs, and storing the one or more CWs into the allocated space in the buffer, transferring the one or more CWs from the buffer to the physical address in the NVM.

Abstract: A method of maintaining and updating a logical-to-physical (LtoP) table in a storage device including a processor, a volatile memory, and a non-volatile memory, the storage device being in communication with a host utilizing atomic writes, the method including receiving, by the processor, data for storing at a plurality of physical addresses in the non-volatile memory, the data being associated with a plurality of logical addresses of the host, storing, by the processor, the plurality of physical addresses in an atomic segment in the volatile memory, storing, by the processor, one or more of zones of the LtoP table in the non-volatile memory, the one or more zones of the LtoP table corresponding in size to the atomic segment, and updating the one or more zones of the LtoP table with the plurality of physical addresses in the atomic segment.

Abstract: A system and method for combining the execution of a query with other operations, such as a data retention scan, in a storage device, when the execution of the query is not time-sensitive. The storage device may be connected to a host, and may operate during intervals of time in a power save mode. When a query is received by the host that is not time-sensitive, the query may be stored in the host or in the storage device until such time as the device would otherwise return to a normal operating mode, and then the query may be executed. Such delayed execution may enable the sharing of read operations for the query with read operations used, for example, for the execution of other queries or for a data retention scan.

Abstract: Managing data returns to a host in response to read commands, an operation monitor of a solid-state drive (SSD) manages counters used to hold metrics that characterize the estimated time to complete a read operation on a corresponding flash die. A timer generates a periodic event which decrements the counters over time. The value stored in each counter is generated for flash operations submitted to the corresponding die and is, generally, based on the operational history and the physical location of the operation. Whenever a read command is scheduled for submission to a particular die, the time estimate for that particular read operation is retrieved and, based on this information, the optimum order in which to return data to the host is determined. This order is used to schedule and program data transfers to the host so that a minimum number of read commands get blocked by other read commands.

Abstract: A system and method for comparing a character from a search space simultaneously to each of a set of search characters. The set of search characters may correspond to a regular expression. In one embodiment, the search space character is encoded to a short binary presentation (e.g., to an 8-bit representation), which is then converted to a long binary representation one bit of which is set, at a first position in the long binary representation corresponding to the value of the short representation. Each character of the set of search characters is similarly encoded and converted to a respective long binary representation. If the bit in one of the long binary representations corresponding to the set of search characters is set, it indicates that the search character matches the corresponding character of the set of search characters.

Abstract: a method for memory-efficient backup of a logical-to-physical (LtoP) table in a storage device having a processor, a volatile memory, and a non-volatile memory, includes retrieving and subsequently processing, by the processor, a plurality of flushed zone segments stored in a first block stripe of the non-volatile memory in an order opposite to an order of storage of the plurality of flushed zone segments in the first block stripe in the non-volatile memory, wherein the processing of the plurality of flushed zone segments includes identifying a retrieved flushed zone segment of the plurality of flushed zone segments differing from all previously retrieved plurality of flushed zone segments, and writing the identified retrieved flushed zone segment of the plurality of flushed zone segments to a second block stripe of the non-volatile memory.

Abstract: A system and method for combining the execution of a query with other operations, such as a data retention scan, in a storage device, when the execution of the query is not time-sensitive. The storage device may be connected to a host, and may operate during intervals of time in a power save mode. When a query is received by the host that is not time-sensitive, the query may be stored in the host or in the storage device until such time as the device would otherwise return to a normal operating mode, and then the query may be executed. Such delayed execution may enable the sharing of read operations for the query with read operations used, for example, for the execution of other queries or for a data retention scan.

Abstract: A system and method for comparing a character from a search space simultaneously to each of a set of search characters. The set of search characters may correspond to a regular expression. In one embodiment, the search space character is encoded to a short binary presentation (e.g., to an 8-bit representation), which is then converted to a long binary representation one bit of which is set, at a first position in the long binary representation corresponding to the value of the short representation. Each character of the set of search characters is similarly encoded and converted to a respective long binary representation. If the bit in one of the long binary representations corresponding to the set of search characters is set, it indicates that the search character matches the corresponding character of the set of search characters.