Abstract: An apparatus that operates in a first mode of operation to enable performance of a first predetermined task to transfer data via a transmitter device to a receiver device across a first communication channel using a first artificial neural network, wherein the first artificial neural network is partitioned to the transmitter device and the receiver device, and operate in a second mode of operation to enable performance of a second predetermined task to transfer data via the transmitter device to the receiver device across a second communication channel using a second artificial neural network, wherein the second artificial neural network is partitioned to the transmitter device and the receiver device, and determine to operate the apparatus in the first mode or the second mode.

Abstract: A computing system and a compressing method for neural network parameters are provided. In the method, multiple neural network parameters are obtained. The neural network parameters are used for a neural network algorithm. Every at least two neural network parameters are grouped into an encoding combination. The number of neural network parameters in each encoding combination is the same. The encoding combinations are compressed with the same compression target bit number. Each encoding combination is compressed independently. The compression target bit number is not larger than a bit number of each encoding combination. Thereby, the storage space can be saved and excessive power consumption for accessing the parameters can be prevented.

Abstract: A data compression method includes inputting a data sequence that includes first data strings and second data strings alternating with each other, each of the first data strings containing one or more first data all of which have a same predetermined value, each of the second data strings containing one or more second data different from the predetermined value, determining first numbers and second numbers from a beginning of the input data sequence sequentially, each of the first numbers being a number of the first data in a respective one of the first data strings, and each of the second numbers being a number of the second data in a respective one of the second data strings, and arranging the first numbers, the second numbers, and the second data strings in a cyclic manner to generate compressed data.

Abstract: Methods and apparatus to parallelize data decompression are disclosed. An example method selecting initial starting positions in a compressed data bitstream; adjusting a first one of the initial starting positions to determine a first adjusted starting position by decoding the bitstream starting at a training position in the bitstream, the decoding including traversing the bitstream from the training position as though first data located at the training position is a valid token; outputting first decoded data generated by decoding a first segment of the bitstream starting from the first adjusted starting position; and merging the first decoded data with second decoded data generated by decoding a second segment of the bitstream, the decoding of the second segment starting from a second position in the bitstream and being performed in parallel with the decoding of the first segment, and the second segment preceding the first segment in the bitstream.

Abstract: A method for managing file based backups (FBBs) includes obtaining, by a backup agent, a backup request for a FBB, in response to the backup request, generating a FBB, generating a FBB metadata file corresponding to the FBB, wherein the FBB metadata file comprises a set of attribute regions, performing, using the set of attribute regions, a deduplication on the FBB metadata file to obtain a deduplicated FBB metadata file, and storing the deduplicated FBB metadata file in a backup storage system.

Abstract: Systems and methods are described for providing secure storage of data sets while enabling efficient deduplication of data. Each data set can be divided into fixed-length blocks. The plaintext of each block can be convergently encrypted, such as by using a hash of the plaintext as an encryption key, to result in block-level ciphertext that can be stored. If two data sets share blocks, the resulting block-level ciphertext can be expected to overlap, and thus duplicative block-level ciphertexts need not be stored. A manifest can be created to facilitate re-creation of the data set, which manifest identifies the block-level ciphertexts of the data set and a key by which each block-level ciphertext was encrypted. By use of block-level encryption, nearly identical data sets can be largely deduplicated, even if they are not perfectly identical.

Abstract: There is provided a method of encoding a video signal, the method comprising: receiving an input frame; processing the input frame to generate at least one set of residuals data, the residuals data enabling a decoder to reconstruct the input frame from a reference reconstructed frame; and, applying a run-length coding operation to the set of residuals data, wherein the run-length coding operation comprises generating a run-length encoded bytestream comprising a set of symbols representing non-zero data values of the residuals data set and counts of consecutive zero values of the residuals data set. In certain embodiments the method comprises apply a Huffman coding operation to the set of symbols. A method of decoding is also provided as well as apparatuses and a computer readable medium.

Abstract: Partition information associated with one or more partitions that divide a range of values into at least a higher and lower set of values is received. An uncompressed value that falls within the range of values is received and a compressed value that includes a set indicator and intra-set information is generated using the uncompressed value. This includes generating the set indicator based at least in part on whether the uncompressed value falls in the higher or lower set of values, determining whether the uncompressed value includes an extraneous bit where it is necessary but not sufficient that the uncompressed value fall in the higher set of values for the uncompressed value to include the extraneous bit, and generating the intra-set information, including by: excluding the extraneous bit in the uncompressed value from the intra-set information if it is determined to be included. The compressed value is output.

Abstract: A network system that facilitates financial transactions. A software defined network may operate to provide a variety of trading related services to a variety of customers with a low latency. Core or processor affinity for routing processes may improve speeds of routing. Data capture through a shared memory space may allow for a variety of analytics without introducing unacceptable delay.

Abstract: A method of identifying errors or manipulations of data or software, includes receiving a first hash value stored in a first block of the memory, receiving a second hash value from a reference memory, and comparing the hash values. If different, error correction information and the content of the first block is received. The content of the first block is reconstructed by in accordance with the error correction information, generating a hash value and comparing the hash value of the modified content with the received first hash value, until the modified content and the received hash values are identical. The content of the first block received from the reference memory and the content of the reconstructed first block stored in the memory of the device are compared for identifying the differences in the content.

Abstract: A computer-implemented method for content-agnostic referencing of a binary data file, the method comprising: determining a length of the binary data file, the length comprising the number of bits of the binary data file; for the determined length, generating all permutations of data of the determined length; locating an index within the generated permutations, wherein the index is the starting position of the binary data file within the generated permutations; and using the length and the index to indicate the binary data file.

Abstract: In one implementation, a method includes identifying a first content-dependent feature associated with a data sector. The method further includes determining a baseline data sector associated with the data sector. The method further includes determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector. The method further includes providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.

Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.

Abstract: Methods, systems, and apparatus for, for data compression based on a key-value store. In one aspect, a method includes generating, at a server, a current dictionary based on a plurality of key-values stored in a storage system of the server; receiving a key-value pair transmitted by a client device; and performing, at the server, data compression on a key-value in the key-value pair by using the current dictionary; and storing the key-value in the storage system of the server.

Abstract: An information handling system for compressing data includes a data storage device and a processor. The data storage device stores a dictionary and an uncompressed data block. The processor prepends the dictionary to the uncompressed data block, determines, from the uncompressed data block, a literal data string and a match data string where the match data string is a matching entry of the dictionary, and compresses the uncompressed data block into a compressed data block that includes the literal data string and an offset pointer that points to the matching entry.

Abstract: A technique for managing data storage includes generating entropy of blocks on a per-block basis and selectively performing inline compression on blocks based at least in part on their entropy. Entropy of a block provides a rough measure of the block's compressibility. Thus, using per-block entropy enables a storage system to steer compression decisions, e.g., whether to compress and/or how much to compress, flexibly and with high granularity, striking a balance between throughput and storage efficiency.

Abstract: The present invention aims to encode and decode a sequence of integer values by substantially assigning the number of bits of a decimal fraction value per sample. An integer converter 11 selects M selected integer values from L input integer values for a set of the L input integer values and obtains J-value selection information that specifies which of the L input integer values the M selected integer values are. Furthermore, the integer converter 11 obtains one converted integer value by reversibly converting the M selected integer value and an integer value corresponding to the J-value selection information. An integer encoder 12 encodes the converted integer value to obtain a code.

Abstract: Techniques to provide for improved (i.e., reduced) power consumption in an exemplary neural network (NN) and/or Deep Neural Network (DNN) environment using data management. Improved power consumption in the NN/DNN may be achieved by reducing a number of bit flips needed to process operands associated with one or more storages. Reducing the number bit flips associated with the NN/DNN may be achieved by multiplying an operand associated with a first storage with a plurality of individual operands associated with a plurality of kernels of the NN/DNN. The operand associated with the first storage may be neuron input data and the plurality of individual operands associated with the second storage may be weight values for multiplication with the neuron input data. The plurality of kernels may be arranged or sorted and subsequently processed in a manner that improves power consumption in the NN/DNN.

Abstract: In embodiments, secure compression algorithms are provided that may be employed as a single operation on raw data to produce compressed and encrypted data. In embodiments, the algorithms described herein may be performed using any type of dictionary based encryption. In one embodiment, upon adding a new prefix to a dictionary table, the dictionary table may be permuted to randomize the entries into the table. The randomization may be based upon a permutation value generated by a deterministic pseudo-random generator and/or pseudo-random function. Other embodiments of randomization may be employed to provide secure compression. For example, instead of permuting the entire table upon adding a prefix, the prefix may be randomly added to the table.

Abstract: In the data compression method, a raw data block in raw data is processed based on a compression algorithm to obtain a standard compressed data block that has a length of L2 and that corresponds to the raw data block, and the raw data is further compressed into one or more standard compressed data blocks that each have the length of L2 and that are to be decompressed in parallel by a decompression apparatus, where the decompression apparatus includes a plurality of decompression engines, and each decompression engine is capable of decompressing one standard compressed data block within one processing cycle. According to the data compression method, a standard compressed data block with a fixed length can be obtained through compression.

Abstract: A memory system including a history buffer, a hash calculator, a read pointer table, a history buffer writing circuit, a read pointer writing circuit, a read pointer reading circuit, a history buffer reading circuit, a matching circuit replacing the input data string with a reference information referring the matching candidate data string in the case where at least a part of the input data string and a part of the matching candidate data string match. Reading of the read pointer by the read pointer reading circuit and reading of the stored input data string by the history buffer reading circuit are executed after writing of the read pointer by the read pointer writing circuit and writing of the input data string by the history buffer writing circuit are finished.

Abstract: Continuity-based data protection may be implemented by systems and methods described herein for collecting a set of data that corresponds to a graphical representation of a computing environment, determining a plurality of subsets of the set of data, wherein a subset of the plurality has mathematical continuity, compressing at least the subset of the plurality, thereby generating one or more compressed subsets, and providing the one or more compressed subset to another computing entity, the other computing entity being able to determine the graphical representation of the computing environment, wherein the graphical representation is presentable to a user of the other computing entity.

Abstract: According to one general aspect, an apparatus may include a host interface circuit configured to receive a memory access request, wherein the memory access request is associated with a data set. The apparatus may include a non-volatile memory storage circuit configured to create and store a transformed data set. The apparatus may include a translation circuit comprising at least one machine learning circuit. The translation circuit may be configured to: in response to a write memory access, convert an original version of the data set to the transformed data set, and in response to a read memory access, convert the transformed data set to a reconstructed data set.

Abstract: Techniques involve storing data. In particular, such techniques involve: obtaining first data to be stored; determining whether the first data is able to be compressed in a compression ratio exceeding a predetermined threshold; and storing, based on the determined result, the first data into a storage device. Accordingly, such techniques can execute corresponding processing for data in a predicted compression ratio, so as to store the data into a storage device. In this manner, such techniques can significantly cut down the overheads for processing data while minimizing a storage space required for storing data.

Abstract: Techniques are disclosed for identifying fixed bits of a bitstring format. One or more processors are configured to generate a first bitstring having respective first bit values that have a first satisfiability state and generate a second bitstring having respective second bit values that have a second satisfiability state. The one or more processors are configured to identify first potential free bits having respective first common values and generate a third bitstring having first potential free bits with the respective first common values and third remaining bits. The one or more processors are configured to identify second potential free bits having respective second common values and identify a fixed bit that is not included in the first potential free bits and is not included in the second potential free bits.

Abstract: A modeling system may generate a function model and a corresponding functional complexity score (FCS) for display on a graphical user interface (GUI). The FCS may be calculated by looping through each level of the function model and determining a score based on both hierarchical relationships and functional relationships within the function model at each level. The modeling system may constantly monitor for additional inputs representative of an additional function or changes to current functions and automatically update the FCS displayed on the GUI based on the additional inputs.

Abstract: A data compression technique involves: selecting, from a data block, a plurality of bit strings as sample data. The technique further involves: determining a set of characters included in the sample data. Each character in the set of characters is represented by at least one bit string among the plurality of bit strings. The technique further involves: compressing the data block if statistical characteristics of the set of characters match predetermined statistical characteristics. Accordingly, incompressible data can be filtered out efficiently and accurately, so as to utilize computing resources more efficiently.

Abstract: A device may receive, from a source device, original time series data to be stored in a data structure associated with the device, and may sort the original time series data to generate sorted time series data. The device may identify an index for the original time series data based on the sorted time series data. The device may process the sorted time series data, with a regression model, to generate compressed time series data and parameters associated with the compressed time series data. The device may encode the index to generate an encoded index, and may store the encoded index, the compressed time series data, and the parameters in the data structure.

Abstract: In various examples, metadata may be generated corresponding to compressed data streams that are compressed according to serial compression algorithms—such as arithmetic encoding, entropy encoding, etc.—in order to allow for parallel decompression of the compressed data. As a result, modification to the compressed data stream itself may not be required, and bandwidth and storage requirements of the system may be minimally impacted. In addition, by parallelizing the decompression, the system may benefit from faster decompression times while also reducing or entirely removing the adoption cycle for systems using the metadata for parallel decompression.

Abstract: A memory system including a storage device and a memory controller controlling the storage device and decoding an encoded data. The memory controller including: a history buffer storing a decoded data string; a history buffer read controller executing a read request to the history buffer; a decode executing section generating a first shaped data string based on the decoded data string read from the history buffer, generating a second shaped data string by referring the first shaped data string before the first shaped data string being written back to the history buffer in response to the read request, and generating a decoded result using the first shaped data string and the second shaped data string.

Abstract: A method of scheduling instructions in a processing system comprising a processing unit and one or more co-processors comprises dispatching a plurality of instructions from a master processor to a co-processor of the one or more co-processors, wherein each instruction of the plurality of instructions comprises one or more additional fields, wherein at least one field comprises grouping information operable to consolidate the plurality of instructions for decomposition, and wherein at least one field comprises control information. The method also comprises decomposing the plurality of instructions into a plurality of fine-grained instructions, wherein the control information comprises rules associated with decomposing the plurality of instructions into the plurality of fine-grained instructions. Further, the method comprises scheduling the plurality of fine-grained instructions to execute on the co-processor, wherein the scheduling is performed in a non-sequential order.

Abstract: An apparatus and method for predicting a brand name of a product are disclosed herein. A product identification number for the product is converted into a normalized global trade item number (GTIN). For each of a plurality of GTIN prefixes corresponding to the normalized GTIN, brand names and counts of each of the brand names using product information stored in a product catalog are identified. A probability distribution of the brand names is determined in accordance with the brand names and the counts of the brand names for the plurality of the GTIN prefixes. A predicted brand name for the product is identified from among the brand names for the plurality of the GTIN prefixes, the predicted brand name having a highest probability score in the probability distribution of the brand names.

Abstract: A method and apparatus for encoding and decoding of floating-point number is provided. The method for encoding is used to convert at least one original floating-point number to at least one encoded floating-point number. The method for encoding includes: determining a number of exponent bits of the at least one encoded floating-point number and calculating an exponent bias according to at least one original exponent value of the at least one original floating-point number; and converting an original exponent value of a current original floating-point number of the at least one original floating-point number to an encoded exponent value of a current encoded floating-point number of the at least one encoded floating-point number according to the exponent bias.

Abstract: A system and method for encoding data using a plurality of encoding libraries. Portions of the data are encoded by different encoding libraries, depending on which library provides the greatest compaction for a given portion of the data. This methodology not only provides substantial improvements in data compaction over use of a single data compaction algorithm with the highest average compaction, but provides substantial additional security in that multiple decoding libraries must be used to decode the data. In some embodiments, each portion of data may further be encoded using different sourceblock sizes, providing further security enhancements as decoding requires multiple decoding libraries and knowledge of the sourceblock size used for each portion of the data. In some embodiments, encoding libraries may be randomly or pseudo-randomly rotated to provide additional security.

Abstract: Provided are an electronic device, a signal processing method thereof, a biological signal measurement system, and a non-transitory computer readable recording medium. The electronic device, according to one embodiment of the present disclosure, comprises: a sensor for measuring a biological signal of a user; and a processor for determining the periodicity of the measured biological signal, and selectively compressing the measured biological signal according to the determined periodicity.

Abstract: Systems, methods, and software are disclosed herein for compressing audio data. In an implementation, sampled values of an audio signal have a dynamic range. A division of the dynamic range, into at least a lower range and an upper range, is identified based on a fixed mapping of a lower portion of the sampled values to a subset of quanta in a set of quanta having a depth less than a depth of the sampled values. Then an adaptive mapping of an upper portion of the sampled values to a remaining subset of quanta in the set of quanta is also identified, based at least on a dimension of the upper range. The fixed mapping is used to encode the lower portion of the sampled values based, while the adaptive mapping is used to encode the upper portion of the sampled values based on the adaptive mapping.

Abstract: A system and method stores a database file into Flash memory or other write-constrained storage. The system and method can decompress the data to use to process a request by only decompressing data it determines, via metadata, might correspond to a criteria in the request.

Abstract: A compression device includes a dictionary based encoder, a second buffer, a comparator, and a compression data generator. The dictionary based encoder searches for second data at least partially matching first data from a first buffer, and acquires a first match position indicating a position of the second data in the first buffer and a match length indicating a matched length of the first and second data. The second buffer stores the previously acquired second match position with an index. The compression data generator generates first compressed data that includes the index assigned to the second match position in the second buffer and the match length when the first match position matches the second match position in the second buffer.

Abstract: A data encoding method comprises generating an ordered array of data values representing an image region and having an array order; deriving a quantization parameter indicating a quantization degree so that the ordered array of data values, when encoded using that quantization degree, meets one or more predetermined criteria; quantizing the data values using the derived quantization parameter to generate respective quantized values; detecting a number of non-zero quantized values; if no more than a threshold number of first values in the array order are non-zero, selecting a different quantization parameter and quantizing the data values using a quantization degree indicated by the selected different quantization parameter; and entropy encoding at least the non-zero quantized values.

Abstract: The DATA LEARNING AND ANALYTICS APPARATUSES, METHODS AND SYSTEMS (“DLA”) provides a data learning platform that analyzes and incorporates data knowledge from new data event updates (e.g., in a real-time stream, in a batch, etc.). In one implementation, the DLA provides a real-time data mining mechanism that accommodates an ever increasing data load instantaneously with reduced complexity, e.g., without re-building the data analytical model in its entirety whenever new data is received. In this way, the DLA provides an incremental learning mechanism for data analytics and thus the latency for real-time data assessment is reduced and the data processing efficiency is improved.

Abstract: For join acceleration, a computer stores local encoding dictionaries (ED), including a build ED that contains a plurality of distinct build dictionary codes (DC) and a probe ED that contains a plurality of distinct probe DCs that is not identical to the plurality of distinct build DCs (BDC). Build data rows (DR) that contain a build key that contains BDCs from the plurality of distinct BDCs is stored. Probe DRs that contain a probe key that contains probe DCs from the plurality of distinct probe DCs is stored. A request for a relational join of the build DRs with the probe DRs is received. The BDCs from the build key and the probe DCs from the probe key are transcoded to global DCs (GDC) of a global ED. Based on GDCs for the build key, a build array whose offsets are respective GDCs of the global ED is populated. Based on GDCs for the probe key, offsets of the build array are accessed. A response to the request for the relational join that is based on accessing offsets of the build array is sent.

Abstract: A non-transitory computer-readable recording medium storing a program that causes a computer to execute a process, the process includes comparing first data previously acquired from a data transmission source device and second data currently acquired from the device to specify a difference portion of the second data as compared with the first data; determining a dividing position of the second data so that the difference portion is made into a single block; dividing the second data into first blocks at the dividing position; transmitting, to the device, a first hash value for each of the first blocks and information on the dividing position; and receiving, from the device, the second block corresponding to a second hash value different from the first hash value for each of the first blocks, among second blocks obtained by dividing third data held by the device based on the information on the dividing position.

Abstract: One example method includes receiving ‘n’ data streams, where ‘n’ is ?2, receiving a commonality parameter ‘F’, creating a mixed data stream having a commonality of ‘F’ by mixing data from the ‘n’ data streams together, and outputting the mixed data stream. The mixed data stream may be provided to a deduplication engine for deduplication of stream data that is common to one or more other data streams.

Abstract: This document describes example methods, devices, and computing systems to rank a data string into a ranking value, and de-rank a second value representative of a variation of the ranking value (e.g., cipher value) into a second data string (e.g., cipher string), using a tree data-structure. The tree data-structure defines a domain of values representative of all variations of a set of characters that satisfy a format within which the data string is defined. The ranking value is identified with respect to a first offset value calculated for individual characters of the data string relative to the domain of values of the tree data-structure. The second value is de-ranked into a second data string of individual characters identified with respect to a second offset value calculated in the domain of values of the tree data-structure relative to the second value.

Abstract: An image signal processor, comprising an input module for obtaining input data from a camera, whereby the camera is arranged to capture a representation of a real-world environment. The image signal processor further comprises at least one adjustment module for compressing the input data and producing compressed input data, and a localization and mapping module arranged to generate one or more data points from the compressed input data. The image signal processor also comprises an output module for outputting at least the one or more data points.

Abstract: A data encoding method comprises generating an ordered array of data values representing an image region and having an array order; deriving a quantization parameter indicating a quantization degree so that the ordered array of data values, when encoded using that quantization degree, meets one or more predetermined criteria; quantizing the data values using the derived quantization parameter to generate respective quantized values; detecting a number of non-zero quantized values; if no more than a threshold number of first values in the array order are non-zero, selecting a different quantization parameter and quantizing the data values using a quantization degree indicated by the selected different quantization parameter; and entropy encoding at least the non-zero quantized values.

Abstract: A method includes receiving a data set that includes a plurality of data records, where a data record includes a first data field containing a first fixed length data value and a second data field containing a first variable length data value. The method further includes accessing a compression dictionary for the second data field, where a first entry of the compression dictionary includes a key field storing a first fixed length index value and a value field storing the first variable length data value, and where the key field has a smaller data size than the value field. The method further includes creating a storage data set based on the compression dictionary and sending the storage data set to a storage sub-system for storage, where the first variable length data value of the second data field of the data record is replaced with the first fixed length index value.

Abstract: Methods and apparatus to parallelize data decompression are disclosed. An example method selecting initial starting positions in a compressed data bitstream; adjusting a first one of the initial starting positions to determine a first adjusted starting position by decoding the bitstream starting at a training position in the bitstream, the decoding including traversing the bitstream from the training position as though first data located at the training position is a valid token; outputting first decoded data generated by decoding a first segment of the bitstream starting from the first adjusted starting position; and merging the first decoded data with second decoded data generated by decoding a second segment of the bitstream, the decoding of the second segment starting from a second position in the bitstream and being performed in parallel with the decoding of the first segment, and the second segment preceding the first segment in the bitstream.

Abstract: There are provided a data archiving method and system, which can process primary archiving capable of rapidly searching for compressed and stored data by compressing and storing, in a table of a database or another database, at least some of data stored in the database included in a target system, and can process secondary archiving for data for which search is no longer requested or for which a search request is low by storing at least some of the data compressed and stored in the table in the form of a file according to a preset condition.

Abstract: Embodiments of the present disclosure relate to verifying compressed data. Compressed data files can be read from a global cache for a storage device into a local buffer. A data verification level of a plurality of data verification levels can be selected to perform on the compressed data files. An amount of data blocks of each data file can be decompressed based on the determined data verification level. An integrity of the compressed data files verified using the decompressed data blocks.