Abstract: Various devices, such as storage devices or storage systems are configured to perform on device semantic searching. The device includes a processor, a plurality of memory devices, a controller coupled to the memory devices, and an intelligent memory array logic. The intelligent memory array logic is configured to receive a query, extract contextual data from the query, determine a machine learning model for processing the query based on the extracted contextual data, generate a query vector based on the query and determined machine learning model, determine one or more relevant memory structures associated with the generated query vector, and pass in the query vector to the one or more determined relevant memory structures. The one or more relevant memory structures include feature data and one or more machine learning processing units configured to process the query vector and feature data to generate a comparison value.

Abstract: Methods and apparatus for processing data in a distributed inference scheme based on sparse inputs. An example method includes receiving an input at a first node. A first sparsified input is generated for a second node based on a set of features associated with the second node, which are identified based on a weight mask having non-zero values for weights associated with features upon which processing by the second node depends and zeroed values for weights associated with other features. The first sparsified input is transmitted to the second node for generating an output of the second node. A second sparsified input is received from the second node and combined into a combined input. The combined input is processed into an output of the first node. The neural network is configured to generate an inference based on processing the outputs of the first node and the second node.

Abstract: A system includes at least one memory controller that partitions at least one memory into a plurality of nodes. Blast zones are formed that each include a predetermined number of nodes. Cache lines are erasure encoded to be stored in one or more blast zones with at least two nodes in a blast zone storing respective portions of a cache line and at least one node in the blast zone storing a parity portion. In one aspect, it is determined that data stored in one or more nodes of a blast zone needs to be reconstructed and stored in one or more spare nodes designated to replace the one or more nodes. Erasure decoding is performed using data from one or more other nodes in the blast zone to reconstruct the data for storage in the one or more spare nodes.

Abstract: A Virtual Switching (VS) kernel module manages different flows of packets between at least one Virtual Machine (VM) running at a node and one or more other VMs running at the node or at one or more other nodes in a network. A packet is received from a first VM using the VS kernel module and is parsed to identify a memory message and an address for at least one memory block stored in a shared memory. At least one entry for the at least one memory block is updated in a directory in a kernel space using the VS kernel module based on the memory message. According to another aspect, a state for the at least one memory block is determined from the directory and the VS kernel module is used to respond to the memory request based on the determined state.

Abstract: A node includes a shared memory for a distributed memory system. A Virtual Switch (VS) controller establishes different flows of packets between at least one Virtual Machine (VM) running at the node and one or more other VMs running at the node or at another node. Requests to access the shared memory are queued in submission queues in a kernel space and processed requests are queued in completion queues in the kernel space. Indications of queue occupancy are determined for at least one queue and one or more memory request rates are set for at least one application based at least in part on the determined indications of queue occupancy. In another aspect, flow metadata is generated for each flow and at least one of the set one or more respective memory request rates and one or more respective resource allocations is adjusted for the at least one application.

Abstract: A node includes a shared memory for a distributed memory system on a network. A Non-Volatile Memory express (NVMe) request is received from a user space application executed by a Virtual Machine (VM) to send an NVMe command to a different node in the network. If a data size for the NVMe request exceeds a maximum segment size of an NVMe over Fabric (NVMe-oF) connection, packets are created to be sent for the NVMe request and an order is determined for sending the packets with one or more packets including data for the NVMe command being sent before a last packet that includes the NVMe command. In another aspect, Virtual Switching (VS) queues are created in a kernel space with each VS queue corresponding to a different respective user space application initiating requests and at least one user space application being executed by one or more other nodes.

Abstract: A programmable network switch includes at least one pipeline including a packet parser configured to parse packets, and a plurality of ports for communication with network devices including a plurality of Data Storage Devices (DSDs). A packet comprising a write command is received to store data in a DSD of the plurality of DSDs, and an identifier generated for the data is compared to a plurality of identifiers generated for data stored in the plurality of DSDs. It is determined whether to send the write command to store the data to the DSD based on whether the generated identifier matches an identifier of the plurality of identifiers. In one aspect, the data to be stored for the write command is extracted from the packet using a pipeline of the programmable network switch, and at least a portion of the extracted data is used to generate the identifier for the data.

Abstract: A node includes at least one memory for use as a shared cache in a distributed cache. One or more other nodes on a network each provide a respective shared cache for the distributed cache. A request is received by a kernel of the node to access data in the shared cache and an Input/Output (I/O) queue is identified from among a plurality of I/O queues in a kernel space of the at least one memory for queuing the received request based on at least one of a priority indicated by the received request and an application that initiated the request. In another aspect, each I/O queue of the plurality of I/O queues corresponds to at least one of different respective priorities for requests to access data in the shared cache and different respective applications initiating requests to access data in the shared cache.

Abstract: A Non-Volatile Memory express (NVMe) node includes a memory used at least in part as a shared cache in a distributed cache. At least one processor of the NVMe node executes a kernel of an Operating System (OS). A request is received from another NVMe node to read data stored in the shared cache or to write data in the shared cache and an error detection operation is performed on the data for the request using the kernel. In another aspect, the kernel is used to perform Erasure Coding (EC) on data to be stored in the distributed cache. A network controller determines different EC ratios based at least in part on indications received from NVMe nodes of frequencies of access of different data and/or usage of the distributed cache by different applications. The network controller sends the determined EC ratios to the NVMe nodes.

Abstract: A mesh network-on-a-chip (NOC) with heterogenous routers for use with homogenous processing elements. Some of the routers are configured differently from other routers to interface more efficiently with particular physical resources that the processing elements require, such as particular input/output or memory devices. For example, one router is configured for use with the Peripheral Component Interconnect Express (PCIe) protocol, whereas another router is configured for use with the InterLaken communication protocol. Still further, the overall system is configured so that the various physical resources are physically adjacent to the particular router that is designed to access the resource to help ensure fair access by each processing element of the NOC to the particular resources that are required. The NOC may be part of a large manycore system on field programmable gate array (FPGA). The methods and apparatus described herein are generally applicable to all system-on-a-chip (SOC) designs.

Abstract: Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply a plurality of analog weights to data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the plurality of analog weights. The device also comprises a device controller configured to program the plurality of analog weights to the CIM module based on the digital weight references and determine degradation of one or more analog weights. The digital weight references in the digital weight storage unit are populated with values from a host device. Degraded analog weights in the CIM module are replaced with corresponding digital weight references from the digital weight storage unit without reference to the host device.

Abstract: A programmable switch includes ports, and circuitry to receive cache messages for a distributed cache from client devices. The cache messages are queued for sending to memory devices from the ports. Queue occupancy information is generated and sent to a controller that determines, based at least in part on the queue occupancy information, at least one of a cache message transmission rate for a client device, and one or more weights for the queues used by the programmable switch. In another aspect, the programmable switch extracts cache request information from a cache message. The cache request information indicates a cache usage and is sent to the controller, which determines, based at least in part on the extracted cache request information, at least one of a cache message transmission rate for a client device, and one or more weights for queues used in determining an order for sending cache messages.

Abstract: A node includes a memory configured to be used at least in part as a shared cache in a distributed cache. A network interface of the node is configured to communicate with one or more other nodes in a network. Each of the one or more other nodes is configured to provide a respective shared cache for the distributed cache. At least one processor of the node is configured to execute a kernel of an Operating System (OS) for allocating resources of the node. The kernel is used to collect cache access information for the shared cache for identifying malicious operations in the distributed cache.

Abstract: A programmable switch includes a plurality of ports for communicating with devices on a network. Circuitry of the programmable switch is configured to receive a series of related messages from a first device on the network via at least one port, and determine whether one or more messages of the series of related messages have been received out-of-order based at least in part on a sequence number included in the one or more messages. The series of related messages are sent by the programmable switch to a second device via one or more ports in an order indicated by sequence numbers included in the series of related messages by delaying at least one message. According to one aspect, a network controller selects a programmable switch between the first device and the second device to serve as a message sequencer for reordering out-of-order messages using a stored network topology.

Abstract: A Non-Volatile Memory express (NVMe) node includes a memory used at least in part as a shared cache in a distributed cache. At least one processor of the NVMe node executes a kernel of an Operating System (OS). A request is received from another NVMe node to read data stored in the shared cache or to write data in the shared cache and an error detection operation is performed on the data for the request using the kernel. In another aspect, the kernel is used to perform Erasure Coding (EC) on data to be stored in the distributed cache. A network controller determines different EC ratios based at least in part on indications received from NVMe nodes of frequencies of access of different data and/or usage of the distributed cache by different applications. The network controller sends the determined EC ratios to the NVMe nodes.

Abstract: An apparatus is disclosed having a parity buffer having a plurality of parity pages and one or more dies, each die having a plurality of layers in which data may be written. The apparatus also includes a storage controller configured to write a stripe of data across two or more layers of the one or more dies, the stripe having one or more data values and a parity value. When a first data value of the stripe is written, it is stored as a current value in a parity page of the parity buffer, the parity page corresponding to the stripe. For each subsequent data value that is written, an XOR operation is performed with the subsequent data value and the current value of the corresponding parity page and the result of the XOR operation is stored as the current value of the corresponding parity page.

Abstract: A programmable switch includes at least one memory configured to store a cache directory for a distributed cache, and circuitry configured to receive a cache line request from a client device to obtain a cache line. The cache directory is updated based on the received cache line request, and the cache line request is sent to a memory device to obtain the requested cache line. An indication of the cache directory update is sent to a controller for the distributed cache to update a global cache directory. In one aspect, the controller sends at least one additional indication of the update to at least one other programmable switch to update at least one backup cache directory stored at the at least one other programmable switch.

Abstract: A programmable switch receives a cache line request from a client of a plurality of clients on a network to obtain a cache line. One or more additional cache lines are identified based on the received cache line request and prefetch information. The cache line and the one or more additional cache lines are requested from one or more memory devices on the network. The requested cache line and the one or more additional cache lines are received from the one or more memory devices, and are sent to the client.

Abstract: In some implementations, the present disclosure relates to a method. The method includes obtaining a set of weights for a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. The method also includes identifying a first subset of weights and a second subset of weights based on the set of weights. The first subset of weights comprises weights that used by the neural network. The second subset of weights comprises weights that are prunable. The method further includes storing the first subset of weights in a first portion of a memory. A first error correction code is used for the first portion of the memory. The method further includes storing the second subset of weights in a second portion of the memory. A second error correction code is used for the second portion of the memory. The second error correction code is weaker than the first error correction code.

Abstract: A network device includes at least one control path port and data path ports configured to communicate on a network. A connection request is received from a host via a control path port, and a resource of the network device is allocated to the host. A data path port is determined from among the plurality of data path ports for communication between the host and the allocated resource. An indication of the determined data path port is sent to the host via the control path port for communication on a data path between the host and the allocated resource. In one aspect, a network interface includes at least one control path port and a first plurality of data path ports configured to communicate on a network. A connection request is received from a host via a control path port, and a locally connected device is allocated to the host.