Abstract: A search apparatus encodes a plurality of elements, each of which is a unit of encoding, stored in a plurality of items, in a file that includes a plurality of records each including the plurality of items separated by delimiters, with use of an encoding dictionary indicating corresponding relationships between the elements and codes. Further, the search apparatus generates an index that associates the elements with positions of the respective elements in the file. Then, the search apparatus generates mask information indicating a range of a specific search target item among the plurality of items, in the file. The search apparatus searches for one or more element strings satisfying a search condition, from among search target elements that are indicated by the index as being within the range of the search target item indicated in the mask information, based on the search condition.

Abstract: In a system in which a database process among processes of a computer is offloaded to one or more hardware circuits each having a memory, the computer divides a column store database made up of a plurality of records and a plurality of columns into a plurality of data segments. One or more columns among the plurality of columns are each one or more compression target columns. Each of the plurality of data segments includes: a column store database portion made up of one or more records among the plurality of records; and a compression dictionary which is a dictionary for decompressing one or more compression target column portions within the column store database portion. The size of the compression dictionary in each of the plurality of data segments is equal to or smaller than the size of the memory.

Abstract: A method of compressing data is described in which the compressed data is generated by either or both of a primary compression unit or a reserve compression unit in order that a target compression threshold is satisfied. If a compressed data block generated by the primary compression unit satisfies the compression threshold, that block is output. However, if the compressed data block generated by the primary compression unit is too large, such that the compression threshold is not satisfied, a compressed data block generated by the reserve compression unit using a lossy compression technique, is output.

Abstract: A method of data nibble-histogram compression can include determining a first amount of space freed by compressing the input data using a first compression technique, determining a second amount of space freed by compressing the input data using a second, different compression technique, compressing the input data using the compression technique of the first and second compression techniques determined to free up more space to create compressed input data, and inserting into the compressed input data, security data including one of a message authentication control (MAC) and an inventory control tag (ICT).

Abstract: A method and apparatus for decoding a video sequence using a discrete sine transform (DST) type-VII transform core includes generating a set of tuples of transform core elements associated with an n-point DST-VII transform core. A first sum of a first subset of transform core elements of a first tuple is equal to a second sum of a second subset of remaining transform core elements of the first tuple. The n-point DST-VII transform core is generated based on generating the set of tuples of transform core elements. A transform on a block is performed using the n-point DST-VII transform core.

Abstract: A compression method, system, and computer program product include creating compressed data via a first system from input data, sending information to a second system detailing a compression strategy for the compressed data, and learning, via the second system, from the information how to recreate the input to the first system using the compressed data.

Abstract: One embodiment provides a computer implemented method of data compression using a hardware accelerator. The method includes communicating, using a hardware accelerator, to a client device a set of instructions that compressed data be sent in LZ compressed format; receiving LZ compressed data at the hardware accelerator in LZ compressed format; decompressing the LZ compressed data using the hardware accelerator to create decompressed data; recompressing the decompressed data into GZ format, using the hardware accelerator, to create GZ compressed data; and sending the GZ compressed data to a storage device.

Abstract: An information processing apparatus includes a data storage, and a processor coupled to the data storage and that executes a process including dividing data including a plurality of blocks, performing, for each portion generated by dividing the data, first clustering of blocks included in each of the portions based on similarities between the blocks included in each of the portions, determining, among the plurality of blocks, blocks to be collectively compressed based on results of the first clustering performed for each of the portions, and collectively compressing data of the determined blocks before storing the data into the data storage.

Abstract: In an embodiment, a method includes receiving, from a data source, time-series data of a time-series data stream produced by the data source. The method further includes identifying a target compression algorithm for the time-series data, wherein the target compression algorithm is linked to the data source in memory pursuant to a dynamically-variable assignment. The method also includes compressing the time-series data using the target compression algorithm and transmitting the compressed time-series data to a destination. Furthermore the method includes periodically optimizing the dynamically-variable assignment in real-time as the time-series data is received.

Abstract: Methods, devices and systems enhance compression and decompression of data blocks of data values by selecting the best suited compression method and device among two or a plurality of compression methods and devices, which are combined together and which said compression methods and devices compress effectively data values of particular data types; said best suited compression method and device is selected using as main selection criterion the dominating data type in a data block by predicting the data types within said data block.

Abstract: In an example, data from a source location is merged into a target location containing existing data, in which the source location and the target location contain tuples of data. For each tuple in the source location, during a single operation, a determination is made as to whether there is a matched tuple in the target location that satisfies a predetermined condition. For each matched tuple that satisfies the predetermined condition, the matched tuple in the target location is updated with a count value that is equal to a count of the matched tuple in the source location and the target location. In addition, for each tuple that does not have a matched tuple that satisfies the predetermined condition, the unmatched tuple is inserted into the target location.

Abstract: The present disclosure relates to a data clearing method, apparatus, and storage medium. The method includes: determining whether a difference between a latest access time of data stored in a currently scanned node in a data structure, and a current time is greater than a predetermined threshold, where the data structure includes a plurality of nodes and each node store a latest access time and file information of respective data to be cleared; if the difference is greater than the predetermined threshold, determining whether the data to be cleared is in a white list; if the data to be cleared corresponding to the currently scanned node is in the white list, continuing to scan a next node; and if the data to be cleared corresponding to the currently scanned node is not in the white list, clearing the data to be cleared corresponding to the currently scanned node.

Abstract: A highly programmable device, referred to generally as a data processing unit, having multiple processing units for processing streams of information, such as network packets or storage packets, is described. The data processing unit includes one or more specialized hardware accelerators configured to perform acceleration for various data-processing functions. This disclosure describes a hardware-based programmable data compression accelerator for the data processing unit including a pipeline for performing string substitution. The disclosed string substitution pipeline, referred to herein as a “search block,” is configured to perform string search and replacement functions to compress an input data stream. In some examples, the search block is a part of a compression process performed by the data compression accelerator. The search block may support single and multi-thread processing, and multiple levels of compression effort.

Abstract: Provided is a method for transmitting a packet in a communication system, comprising the steps of: identifying the packet according to a packet identification criterion; reallocating frames included in the identified packet and encrypting the packet in which the frames have been rearranged; and transmitting the encrypted packet.

Abstract: According to one embodiment, a memory system includes a first decoder that decodes read information read from a nonvolatile memory that records therein a multidimensional error-correcting code to output hard decision decoding information of each symbol; a second decoder that performs soft decision decoding in units of component codes for the read information using a soft-input value to output soft decision decoding information of each symbol; a soft-decision-decoding information memory that retains the soft decision decoding information of each symbol; and a soft-input-value specifying unit that obtains the soft-input value of each symbol using the read information and the hard decision decoding information or the soft decision decoding information, and the soft-input-value specifying unit obtains an initial value of the soft-input value using the read information and the hard decision decoding information, and outputs an output decode word obtained as a result of the soft decision decoding when the output de

Abstract: Genomic data is written to disk in a compact format by dividing the data into segments and encoding each segment with the smallest number of bits per character necessary for whatever alphabet of characters appears in that segment. A computer system dynamically chooses the segment boundaries for maximum space savings. A first one of the segments may use a different number of bits per character than a second one of the segments. In one embodiment, dividing the data into segments comprises scanning the data and keeping track of a number of unique characters, noting positions in the sequence where the number increases to a power of two, calculating a compression that would be obtained by dividing the genomic data into one of the plurality of segments at ones of the noted positions, and dividing the genomic data into the plurality of segments at the positions that yield the best compression.

Abstract: A computer-implemented method according to one embodiment includes dividing data into a plurality of partitions, creating a plurality of groups of different data types within each of the plurality of partitions, independently compressing, within each of the plurality of partitions, each of the plurality of groups of different data types to create a plurality of independently compressed partitions, and validating each of the plurality of independently compressed partitions to create a plurality of validated independently compressed partitions.

Abstract: A mechanism is described for facilitating smart compression/decompression schemes at computing devices. A method of embodiments, as described herein, includes unifying a first compression scheme relating to three-dimensional (3D) content and a second compression scheme relating to media content into a unified compression scheme to perform compression of one or more of the 3D content and the media content relating to a processor including a graphics processor.

Abstract: The invention refers to an apparatus for testing a circuit, including: an interrupter configured for interrupting based on a circuit model describing at least a part of the circuit a connection between two components of the circuit, wherein the circuit model describes the two components connected by the connection, an inserter configured for inserting based on the circuit model a test element model into the interrupted connection, and an evaluator configured for evaluating based on the circuit model and the test element model a response of the circuit model to the inserted test element model. The invention also refers to a corresponding method.

Abstract: The present disclosure describes methods and systems that provide for content based dynamic data compression. During a training stage a plurality of training sets are compressed using multiple data compression techniques, and each training data set is associated with a most effective compression technique. Statistical features can be extracted from each of the training data sets with consideration of the most effective compression techniques and are saved as a compression parameters file. During a working stage, individual data blocks within a telemetry data set are compressed using a data compression technique selected for the data block in light of the statistical feature presented in the compression parameters file, and the compressed data blocks are written to a compressed data set, along with encoding tags that identify the compression technique used on each data block. During a consumption stage, the compressed data blocks are decompressed using techniques identified by the encoding tags.

Abstract: A data compression apparatus is described which has an encoder configured to receive an input data item and to compress the data item into an encoding comprising a plurality of numerical values. The numerical values are grouped at least according to whether they relate to content of the input data item or style of the input data item. The encoder has been trained using a plurality of groups of training data items grouped according to the content and where training data items within individual ones of the groups vary with respect to the style. The encoder has been trained using a training objective which takes into account the groups.

Abstract: A data compression system includes a memory to store a plurality of predetermined prefixes corresponding to a plurality of classes of data. A classifying module is configured to receive data, receive a class of the data, and select a prefix to compress the data from the plurality of predetermined prefixes based on the data and the class of the data. A compressing module is configured to compress the data using the prefix. A header generating module is configured to generate a header including an indication of the prefix used to compress the data, and to output the header and the compressed data for storage or transmission. Using the prefix from the predetermined prefixes to compress the data eliminates an overhead of fetching the prefix from outside the data compression system.

Abstract: Methods, non-transitory computer readable media, computing devices and systems for persistent indexing and space management for flat directory include creating, using at least one of said at least one processors, an index file to store mapping information, computing, using at least one of said at least one processor, a hash based on a lookup filename, searching, using at least one of said at least one processor, the index file to find all matching directory cookies based on the computed hash, selecting, using at least one of said at least one processor, the directory entity associated with the lookup filename from among the matched directory cookies, and returning, using at least one of said at least one processor, the determined directory entity.

Abstract: A system for disambiguation of noisy text includes an interface and processor. The interface is configured to receive a set of words. The processor is configured to determine a token from the set of words, wherein the token includes a subset of the set of words; determine candidates for the token; determine a ranking for the candidates based at least in part on a semantic similarity between the token and each of the candidates; determine whether to select a top ranked candidate or to abstain; and in response to a select determination to select the top ranked candidate, indicate the top ranked candidate is selected.

Abstract: A compression coding apparatus for artificial neural network, including memory interface unit, instruction cache, controller unit and computing unit, wherein the computing unit is configured to perform corresponding operation to data from the memory interface unit according to instructions of controller unit; the computing unit mainly performs three steps operation: step one is to multiply input neuron by weight data; step two is to perform adder tree computing and add the weighted output neuron obtained in step one level-by-level via adder tree, or add bias to output neuron to get biased output neuron; step three is to perform activation function operation to get final output neuron. The present disclosure also provides a method for compression coding of multi-layer neural network.

Abstract: A method includes receiving, by a processing device, a plurality of genome files. Each genome file corresponds to a different sample and defining a genetic sequence. The method also includes generating, by the processing device, a two-dimensional alignment file based on the genome files and a reference sequence. A first dimension of the alignment file corresponds to individual genetic sequences and each of the genetic sequences is aligned with respect to the reference sequence along a second dimension of the alignment file. The method includes separating, by the processing device, the alignment file into a plurality of groups and storing the groups in a non-transitory genome data store. Each group contains segments of the genetic sequences of two or more of the genomic files.

Abstract: Described herein are several embodiments which provide for enhanced data caching in combination with adaptive and dynamic compression to increase the storage efficiency and reduce the transmission bandwidth of data during input and output from a GPU. The techniques described herein can reduce the need to access off-chip memory, resulting in improved performance and reduced power for GPU operations. One embodiment provides for a graphics processing apparatus comprising a shader engine; one or more cache memories; cache control logic to control at least one of the one or more cache memories; and a codec unit coupled with the one or more cache memories, the codec unit configurable to perform lossless compression of read-only surface data upon storage to or eviction from the one or more cache memories.

Abstract: An image coding apparatus comprising a coding unit generating coded data by executing prediction coding of an image of a coding target frame, a compression/decompression unit compressing a decoded image obtained by decoding the coded data and store it in a memory, and decompressing a compressed decoded image and output the decompressed decoded image as a reference image in the prediction coding, and a compression ratio calculation unit calculating a compression ratio of the compressed decoded image from the decoded image before compression and the compressed decoded image, wherein in accordance with the compression ratio of the compressed decoded image corresponding to a reference image used in inter-prediction coding processing among the stored compressed decoded images, execution of the bidirectional prediction using the reference image is controlled.

Abstract: The present invention relates to a method for performing channel encoding by a transmitting end in a wireless communication system. Particularly, the method comprises the steps of: transmitting, to a receiving end, a configuration indicating a plurality of channel coding configurations; performing channel encoding using a first channel coding configuration among the plurality of channel coding configurations; and performing reconfiguration from the first channel coding configuration to a second channel coding configuration according to a change in system requirements, wherein the plurality of channel coding configurations comprise channel coding configurations, each comprising at least one channel code concatenated differently according to the system requirements.

Abstract: An electronic device for decompressing compressed data to recreate original data includes a first string copy engine and a second string copy engine. The first string copy engine processes a first string copy command by acquiring a first string from recreated original data and appending the first string to the recreated original data. The second string copy engine processes a second string copy command by checking the second string copy command for a dependency on the first string and, when the dependency is found, stalling further processing of the second string copy command until the first string copy engine has appended a corresponding portion of the first string to the recreated original data. The second string copy engine processes the second string copy command by acquiring a second string from the recreated original data and appending the second string to the recreated original data.

Abstract: The present disclosure discloses a method and device in user equipment and a base station for wireless communication. The base station equipment sequentially generates a first information block, performs first channel coding and transmits a first radio signal, wherein the first information block includes bits in a first sub-information block and padding bits, the first information block is used to generate a first check bit block, the first information block and the first check bit block are interleaved to generate the first bit block, and the first bit block is used as the input of the first channel coding; the bits in the first sub-information block are discontinuous in the first information block, or the padding bits are discontinuous in the first information block.

Abstract: A method for transmitting and updating program data in a highly compact format using recursive encoding, wherein data is deconstructed into chunklets, and is processed through a series of reference code libraries that reduce the data to a sequence of reference codes, and where the output of each reference library is used as the input to the next.

Abstract: An apparatus and a computer-implemented method for processing data are disclosed. The apparatus comprises a memory device that stores first history data, and at least one processor configured to receive input data, determine a relationship between the first history data and one or more portions of the input data, generate one or more reference tokens reflecting the relationship, and transmit the one or more reference tokens to a recipient device.

Abstract: A method of storing data is provided. The method includes receiving a first set of data provided over a network session, and compressing the first set of data to form a second set of data. As further provided, the second set of data includes a number of bytes smaller than the first set of data. Further, the second set of data includes a portion of compressed data that is common to other network sessions. The method further includes compressing further the portion of the compressed data common to other network sessions to obtain a third set of data, and storing the third set of data.

Abstract: A method includes generating data at a first device. The method includes determining one or more parameters corresponding to a statistical distribution of values based on the data. The method includes encoding the data based on a frequency-based code that is based on the statistical distribution to generate compressed data. The method further includes sending the one or more parameters and the compressed data from the first device to the second device.

Abstract: A computing device may select a plurality of data sets, determine a set of strings that are included in at least two data sets of the plurality of data sets, and select a particular string of the set of strings. The computing device may replace each occurrence of the particular string in the plurality of data sets to create a modified plurality of data sets such that the modified plurality of data sets is smaller in size than the plurality of data sets. The computing device may assign a reference to the particular string and replace each occurrence of the particular string in the plurality of data sets to create a plurality of modified data sets. The computing device may replace may store the reference and the particular string in a table.

Abstract: A system and method for data storage, transfer, synchronization, and security using recursive encoding, wherein data is deconstructed into chunklets, and is processed through a series of reference code libraries that reduce the data to a sequence of reference codes, and where the output of each reference library is used as the input to the next.

Abstract: A system and method for data storage, transfer, synchronization, and security using recursive encoding, wherein data is deconstructed into chunklets, and is processed through a series of reference code libraries that reduce the data to a sequence of reference codes, and where the output of each reference library is used as the input to the next.

Abstract: Systems and methods are provided for transcoding media content in accordance with thematic and/or cinematic characteristics of scenes. In particular, compression techniques may be selected to optimize compression of scenes based on their thematic and/cinematic characteristics. Scene boundary identifiers, such as scene in and out points may be identified, and the corresponding thematic and/or cinematic characteristic meta-data can be sent to a transcoder for transcoding. In this way, the specific characteristics of each scene in a movie can be accounted for during transcoding. This can result in transcoded versions of the movie that have better audio/visual qualities, and that can be encoded at a lower overall bit rate.

Abstract: An aspect includes a system architecture that includes a processing unit, an accelerator, a main source buffer, a main target buffer, and a memory block. The main source buffer stores a first part of a source symbol received from an external source. The main target buffer stores an output symbol received from the accelerator. The memory block includes an overflow source buffer that stores the first part of the source symbol received from the main source buffer. The accelerator fetches the first part of the source symbol stored in the overflow source buffer and a second part of the source symbol stored in the main source buffer, and converts the first and second parts of the source symbol together into the output symbol. The second part of the source symbol includes a part of the source symbol not included in the first part of the source symbol.

Abstract: A transmitting apparatus is provided for transmitting data to a receiving apparatus, the transmitting apparatus comprising: a first network interface configured to transmit data to the receiving apparatus over a first communications path; a transmit buffer forming part of the first network interface, the transmit buffer being configured to store a series of packets of data for transmission over the first communications path; a data compression module configured to compress at least some of the packets of data stored for transmission; a second network interface configured to receive a decompression rate value from the receiving apparatus over a second communications path; and wherein the apparatus is configured to select data packets stored in the transmit buffer for compression based on a compression rate value and a transmission rate value of the transmitting apparatus and a decompression rate value received from the receiving apparatus via the second network interface.

Abstract: A decoding device is implemented on an integrated circuit, for decoding a market data input stream received in a given data representation format. The decoding device comprises an engine built around a finite state machine, the engine being generated from at least one description file and configured to perform the following steps, in a current state of the finite state machine: i) dividing the market data input stream into a number of tokens and reading a set of tokens, ii) accumulating the set of read tokens in internal registers, iii) generating output commands from the tokens accumulated in the internal registers depending on a condition related to the tokens accumulated in the internal registers, and iv) selecting the next state of the Finite State Machine state based on a triggering condition.

Abstract: Embodiments of the present invention are directed to a computer-implemented method for data compression. The method includes monitoring data, from a data stream, stored in an input buffer and system memory of a data compression system. The method further includes choosing an encoding scheme based in part upon the amount of data in the input buffer. The method further includes encoding data using the encoding scheme to compress the data from the data stream. The method further includes reevaluating, during the data stream, an encoding scheme choice based in part upon the amount of data in the input buffer.

Abstract: The described technology is generally directed towards reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression. According to an embodiment, a system can comprise a memory that can store computer executable components, and a processor that can execute the components stored in the memory. The components can comprise a data block identifier that can identify, for a sequence of data blocks, a first data block that corresponds to a first data, resulting in a first identified data block, and a deduplication component that can identify a second data block that corresponds to the first data, resulting in a second identified data block, wherein the deduplication component can replace the second identified data block with a key value corresponding to the first identified data block. Further, a compression component can compress the first identified data block, resulting in a compressed data block.

Abstract: A processing device receives a representation of an image, wherein the image has a first size and the representation has a second size that is smaller than the first size, the representation having been generated from the image by a first portion of a first trained machine learning model. The processing device processes the representation of the image using a second portion of the trained machine learning model to generate a reconstruction of the image and then outputs the reconstruction of the image.

Abstract: A software update architecture, system, apparatus, and methodology are provided for performing block-based swapping of OTA software stored as a plurality of compressed blocks in a first, smaller NVM with the system software stored as a plurality of decompressed blocks in a second, larger NVM by using a first decompressor circuit and first scratch memory to sequentially decompress each compressed code block of OTA software for storage in decompressed form as updated system software in the second, larger NVM while using a first compressor circuit and second scratch memory to sequentially compress each decompressed code block of system software for storage in compressed form as backup system software in the first, smaller NVM.

Abstract: An accelerated compression method and apparatus are provided. The accelerated compression apparatus includes a look-ahead buffer, a string matching processing pipeline and a control circuit. A string to be compressed extracted from the data register is stored to the look-ahead buffer. P instances are issued in parallel from the look-ahead buffer. When P substrings corresponding to the P instances issued in a first issue cycle are identical, the control circuit sends a first instance and a second instance of the P instances to the string matching processing pipeline for a matching operation and does not send the remaining instances of these P instances to the string matching processing pipeline. In consecutive issue cycles after the first issue cycle, the control circuit does not send any of the P instances to the string matching processing pipeline until the P substrings corresponding to the P instances are not identical.

Abstract: An accelerated compression method and apparatus are provided. The accelerated compression apparatus includes a look-ahead buffer, a string matching processing pipeline and a control circuit. The string to be compressed extracted from the data register is stored to the look-ahead buffer. P instances are issued in parallel from the look-ahead buffer in each issue cycle. When P substrings corresponding to the instances are identical to each other, one of the P instances is sent to the string matching processing pipeline for a matching operation by the control circuit, and the remaining instances of the P instances are prevent from being sent to the string matching processing pipeline.

Abstract: Technologies for high-ratio compression with heterogeneous history buffers include a computing device having an accelerator complex with a large history buffer and a small history buffer. The large history buffer has a larger size than the small history buffer. For example, the small history buffer may be 32 kilobytes and the large history buffer may be 64 kilobytes, 1 megabyte, or larger. The large history buffer is coupled to a large-buffer compare core that searches for matches in the large history buffer, finds a best match, and forwards the best match to a small-buffer compare core. The small-buffer compare core searches the small history buffer for matches, receives the match forwarded from the large-buffer compare core, and determines a best match from the matches in the small history buffer and the forwarded match. Other embodiments are described and claimed.

Abstract: A computer-implemented method of integrated circuit design comprises: using a computer, detecting data communication paths and associated data traffic requirements between a set of data handling nodes in an integrated circuit layout, the data handling nodes acting as routing nodes or either source nodes or sink nodes for a given data communication path, each source node and each sink node having a respective provisional data width, each data communication path having at least one routing node between the source node and the sink node; using the computer, assigning a provisional data width to each routing node so that, for each of the detected data communication paths, the one or more routing nodes in that data communication path have a provisional data width sufficient to handle the data traffic requirement associated with that communication path; using the computer, performing one or more iterations of modifying the integrated circuit topology by: (i) detecting two or more connected groups of the data han