Abstract: A method for storing data includes: generating data for current data based on historical data and a change rule of the historical data; obtaining a delta between the data and the current data; storing the delta instead of the current data.

Abstract: The instant disclosure provides a data structure store system and a method of managing data in the store. The method includes receiving, by a data structure store management system, a request for storing data from a client. In the created data structure, each data element includes a portion of the data. On receiving a read request for at least part of the data, the data structure store management system provides at least part of the data to a recipient device. The data elements are stored in persistent memory in the form of one or more non-volatile random access devices, wherein during a time interval between receiving the storing request and providing the at least part of the data by the data structure store management system to the recipient device, the data structure store management system provides no portion of the data for writing to a hard disk drive.

Abstract: A system implements data compression for a plurality of data objects each having a respective fixed data constituent and a variable data constituent. The data compression includes selecting a first subset of the fixed data constituents and a second subset of the variable data constituents. The second subset of the variable data constituents having an end date in common and event timing in common. The system compresses the first subset of the fixed data constituents and the second subset of the variable data constituents by defining a representative data object for the fixed data constituent subset and the variable data constituent subset.

Abstract: A system for committing event data includes an interface and a hardware processor. The interface is configured to receive modified input data for desired events. The hardware processor is configured to retrieve the desired events from a desired aggregate; select requested event to modify from the desired aggregate; replace input data of the requested event with the modified input data to create a modified aggregate; replay events of the modified aggregate N to generate a new state of aggregate N; and provide the new state of aggregate N.

Abstract: Embodiments include systems and methods for managing a network of nodes with delta processing. A network with a plurality of connected nodes can be stored, the nodes representing entities of the network. One or more deltas to the network that indicate updates to the connections among the plurality of nodes can be received. The deltas can be added to a queue. The connections between the nodes of the network can be updated using a delta from the queue. The network can be processed after updating the connections between the nodes to generate a vector for a given node. One or more paths for the given node can be identified based on the generated vector.

Abstract: One example method includes, in a data buffer that includes one or more words and whitespaces, calculating a hash value of data in a window that is movable within the data buffer, comparing the hash value to a mask, and when the hash value matches the mask, identifying a position of the window in the data buffer as a chunk anchor position, searching for a whitespace nearest the chunk anchor position, and designating an offset of the whitespace as a segment boundary.

Abstract: A memory swapping method and apparatus are provided. The method includes: selecting n to-be-swapped-out pages; compressing the n to-be-swapped-out pages into n compressed blocks, and buffering the n compressed blocks in a compressed data buffer area; organizing at least one of the n compressed blocks into m to-be-written units; and writing the m to-be-written units into a swap area of a non-volatile storage device in a maximum of m batches, where at least one of the m to-be-written units is stored in a segment of continuous space in the swap area. The method reduces a quantity of write times during memory swapping, thereby prolonging a service life of the non-volatile storage device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for transformation for machine learning pre-processing. In some implementations, an instruction to create a model is obtained. A determination is made whether the instruction specifies a transform. In response to determining that the instruction specifies a transform, a determination is made as to whether the transform requires statistics on the training data. The training data is accessed. In response to determining that the transform requires statistics on the training data, transformed training data is generated from both the training data and the statistics. A model is generated with the transformed training data. A representation of the transform and the statistics is stored as metadata for the model.

Abstract: Deduplicating data based on recently reading the data, including: determining whether a calculated signature for write data matches a particular signature corresponding to data that was recently read from the storage device, wherein the signature is calculated using the write data as input; and after determining that the calculated signature for the write data matches the particular signature, obtaining the data that was recently read and comparing the data that was recently read to the write data.

Abstract: A computer system hosting a column-store database engine is responsive to database requests for the update and retrieval of data from within a stable data table and providing for the storage of database tuples within a column-store organized database structure. A positional delta tree data structure is implemented in the memory space of the database engine and is operatively coupled in an update data transfer path between a database engine interface and the stable data table. The positional delta tree data structure includes a differential data storage layer operative to store differential update data values in positionally defined relative reference to database tuples stored by the stable data table.

Abstract: The linkability of trajectory data based on similarities to candidate trajectory data is measured and quantified as describer herein. Methods may include: receiving a set of probe data points defining a target trajectory from a probe apparatus; characterizing the trajectory based on features of the target trajectory; identifying a plurality of candidate trajectories sharing at least some features with the target trajectory; calculating, for each of the plurality of candidate trajectories, a similarity score with respect to the target trajectory; calculating a privacy score representing a likelihood of identifying the probe apparatus from the target trajectory based on a number of trajectories in the plurality of candidate trajectories and their respective similarity score; and providing information associated with the target trajectory for location-based services in response to the privacy score satisfying a predetermined value.

Abstract: A method for controlling compression of data includes accessing genomic annotation data in one of a plurality of first file formats, extracting attributes from the genomic annotation data, dividing the genomic annotation data into multiple chunks, and processing the extracted attributes and chunks into correlated information. The method also includes selecting different compressors for the attributes and chunks identified in the correlated information and generating a file in a second file format that includes the correlated information and information indicative of the different compressors for the chunks and attributes indicated in the correlated information. The information indicative of the different compressors is processed into the second file format to allow selective decompression of the attributes and chunks indicated in correlated information.

Abstract: In an approach, a processor segments a data structure into a plurality of initial chunks. A processor, for each of the plurality of initial chunks, determines a chunk key based on identifying a sequence of bits within the respective chunk that is unique within the data structure. A processor re-segments the data structure into a plurality of final chunks, wherein divisions correspond to positions of the identified sequences of bits.

Abstract: A compression device includes an analyzer circuit, a control circuit, a compressor circuit, and a selector circuit. The analyzer circuit is configured to analyze first data that is input thereto and generate one or more parameter values regarding data compression and/or decompression. The control circuit is configured to generate at least one compression mode information indicating whether or not compression is to be performed, based on the one or more parameter values. The compressor circuit is configured to compress the first data into second data according to the compression mode information. The selector circuit is configured to output the first data if not compressed or the second data if the first data is compressed, together with the compression mode information.

Abstract: Systems and methods including one or more processors and one or more non-transitory media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform: storing one or more notifications in a data store; receiving a new notification; determining a respective number of notifications in each respective segment of a plurality of approximately equal segments; using the respective number of notifications in each respective segment of the plurality of approximately equal segments to determine a number of the one or more notifications; when the number of the one or more notifications is equal to or greater than a maximum number of notifications, removing at least one notification of the one or more notifications; and before or after removing the at least one notification, storing the new notification in the data store.

Abstract: A file management system includes a communication device and a control device. The control device acts as a file manager, a file outputter, an information file generator, and a file merger. The file manager manages various files. The file outputter outputs a file based on a target file among the various files, to outside of the computer via the communication device. The information file generator generates an information file which is a document file containing information linked with the target file. The file merger generates a merged target file, by merging the information file with the target file, when the target file is of a specific file format. The file outputter outputs, when the file merger generates the merged target file, the merged target file to outside of the computer via the communication device, as the file based on the target file.

Abstract: A processing unit of a processing system compiles a priority queue listing of a plurality of processor cores to run a workload based on a cost of running the workload on each of the processor cores. The cost is based on at least one of a system usage policy, characteristics of the workload, and one or more physical constraints of each processor core. The processing unit selects a processor core based on the cost to run the workload and communicates an identifier of the selected processor core to an operating system of the processing system.

Abstract: Quality communication can be maintained for integrated channels in transaction systems. For example, a system can receive, by a transaction management layer, a transaction request from a transaction channel of a plurality of transaction channels, the transaction request being in a channel-specific format associated with the transaction channel. The system can, in response to a downstream transaction processing system processing the transaction request, receive a status indicator of the processing of the transaction request. The system can transform the status indicator into the channel-specific format associated with the transaction channel. The system can send the status indicator in the channel-specific format to the transaction channel. The transaction channel can be configured to perform an action in response to receiving the status indicator.

Abstract: A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

Abstract: A hardware-implemented file reader includes an interface, multiple hardware-implemented column readers and a hardware-implemented record reconstructor. The interface is configured to access a file including multiple records. The records store values in accordance with a nested structure that supports optional values and repeated values. The file is stored in a columnar format having multiple columns, each column storing (i) compressed values and (ii) corresponding compressed structure information that associates the values in the column to the nested structure of the records. Each column reader is configured to be assigned to a respective selected column, and to read and decompress both the values and the structure information from at least a portion of the selected column. The record reconstructor is configured to reconstruct one or more of the records from at least portions of the columns that are read by the column readers, and to output the reconstructed records.

Abstract: A remediation system for threshold leaks in a datacenter liquid cooling system is disclosed. The system includes a fluid controller and a power controller that are adapted to receive input from a learning subsystem that can determine that a threshold leak has occurred even though a computing component is functioning normally, so that a change in power state to reduce reliance on the coolant and so that a change of flow of the coolant may be effected.

Abstract: A system and method for faster communication between blockchain mining nodes and faster block validation. The system uses machine learning on data chunks to generate codebooks which compact the data to be stored, processed, or sent with a smaller data profile than uncompacted data. The system uses a data compaction in an existing blockchain fork or implemented in a new blockchain protocol from which nodes that wish to or need to use the blockchain can do so with a reduced storage requirement. The system uses network data compaction across all nodes to increase the speed of and decrease the size of a blockchain's data packets. The system uses data compaction firmware to increase the efficiency at which mining rigs can computationally validate new blocks on the blockchain. The system can be implemented using any combination of the three data compaction services to meet the needs of the desired blockchain technology.

Abstract: A method for compressing columnar data may include generating, for a data column included in a data chunk, a dictionary enumerating, in a sorted order, a first set of unique values included in the first data column. A compression technique for generated a compressed representation of the data column having a fewest quantity of bytes may be identified based at least on the dictionary. The compression technique including a dictionary compression applying the dictionary and/or another compression technique. A compressed data chunk may be generated by applying the compression technique to compress the data column included in the data chunk. The compressed data chunk may be stored at a database in a variable-size persistent page whose size is allocated based on the size of the compressed representation of the data column. Related systems and articles of manufacture are also provided.

Abstract: One embodiment provides for a method of transmitting data between multiple compute nodes of a distributed compute system, the method comprising creating a global view of communication operations to be performed between the multiple compute nodes of the distributed compute system, the global view created using information specific to a machine learning model associated with the distributed compute system; using the global view to determine a communication cost of the communication operations; and automatically determining a number of network endpoints for use in transmitting the data between the multiple compute nodes of the distributed compute system.

Abstract: An operation method of a server for updating firmware includes: generating a first delta file including a plurality of blocks based on a plurality of update areas included in a first version firmware; generating a second delta file by repositioning the plurality of blocks included in the first delta file such that a plurality of unit blocks are generated by grouping control blocks, difference blocks, and extra blocks, each of which corresponds to the plurality of update areas, respectively; generating a plurality of swap blocks based on extra blocks among the plurality of blocks; and generating a third delta file by adding the generated plurality of swap blocks to the second delta file.

Abstract: A method and apparatus for compression units in an index block is provided. A particular leaf block of a database index for a table is generated. The particular leaf block corresponds to a particular range of key values. A compression scheme is determined for a plurality of index entries with key values within the particular range. A compression unit that includes a plurality of compressed index entries is generated by applying the compression scheme to the plurality of index entries. Compression metadata describing the compression scheme is generated. The compression unit and the compression metadata is stored in the particular leaf block. In response to a request, at least a portion of the compression unit is decompressed to access one or more index entries.

Abstract: Database systems may dynamically management concurrency levels for performing queries. A query may be received at a database system and a memory usage for the query may be predicted. A determination may be made as to whether available memory is enough to satisfy the predicted memory usage for the query. If the available memory is enough to satisfy the predicted memory usage for the query, then an increase in a concurrency level for performing queries at the database system may be made. The query may be allowed to execute concurrently with other queries according to the increased concurrency level.

Abstract: The invention relates to a method for packaging media and data for transmission over a broadband communications network link for display at a terminal destination. Features of the method included creating desired media segments for transmission to the terminal destination and for creating targeting information, for example in whole or in part in the form of metadata, for transmission to the terminal destination. Other features include packaging the created media segments and the created targeting information in a single package and unpacking the single package for display of the targeted media at the terminal destination.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An intelligent automated assistant receives a user input and generates a set of one or more token sequences based on the user input. The set of one or more token sequences are interpreted to generate a plurality of candidate interpretations each including a corresponding action and metadata associated with the action. A top candidate interpretation is selected from the plurality of candidate interpretations, and the corresponding action is performed based on the associated metadata.

Abstract: A system is provided for electronic data compression by automated time-dependent compression algorithm. In particular, the system may track instances in which a particular dataset is used, copied, or accessed over time. For certain datasets (e.g., datasets that have not been accessed for a threshold amount of time), the system may use a time-based compression algorithm that progressively removes the least significant bits of such datasets as time passes. The compression of the datasets may continue until the system detects that further compression would cause the dataset to be unreadable or unrecoverable. In this way, the system may minimize the computing resources allocated to storing datasets that are not frequently accessed.

Abstract: A method provides for classifying data fields of a dataset. A classifier configured for determining confidence values for a plurality of data classes for the data fields may be applied. Using the confidence values, data class candidates may be identified. Data fields may be determined for which a plurality of data class candidates is identifiable. Using previous user-selected data class assignments, a probability may be determined for the data class candidates that the respective data class candidate is a data class to which the respective data field is to be assigned. The data fields may be classified using the probabilities to select for the data fields a data class from the data class candidates. The dataset may be provided with metadata identifying for the data fields the data classes to which the respective data fields are assigned.

Abstract: The present invention relates to a method and a system for maintaining consistency of data between data-sets stored in a master database of a master computing node and corresponding data-sets stored in a replication database of at least one replication computing node. Each time an updated version of a stored data set is received, the master computing node is configured for updating a corresponding data-set stored in the master database and transmitting replication data relating to the updated data-set version to the at least one replication computing node for replication. The replication data comprising a bit-vector storing data values extracted from data fields of the data-set classified as volatile data fields. The data values stored in the bit-vector representing the changes in the data-set to be replicated in the at least one replication node.

Abstract: Systems and methods are disclosed for scalable bucket merging in a data intake and query system. Various components of a bucket manager can be used to monitor recently-created buckets of data in common storage that are associated with a particular tenant and a particular index, apply a comprehensive bucket merge policy to determine groups of buckets that qualify for merging, merge those group of buckets into merged buckets to be stored in the common storage, and update any information associated with the merged buckets and pre-merged buckets. These components may be shared across multiple tenants, and some of these components may be dynamically scalable based on need. This approach may also provide many additional benefits, including improved search performance from merged buckets, efficient resource utilization associated with discriminate merging, and redundancy in case of component failure.

Abstract: The invention discloses an aggregation optimized processing method for time-series data, characterized by comprising the following steps: writing a time-series data record into a database, forming a time-series database file, wherein the time-series database file comprises a data file and an index file, the data file comprises multiple data blocks, the index file comprises index blocks, and each index block correspond to one data block; by scanning an index file according to a start time period and a stop time period, extracting all index blocks of the time series that need to be aggregated that meet the time period conditions, and then sorting the index blocks according to the data block offset recorded in the index block; and by scanning the data file according to a data block offset order recorded in sorted index blocks, performing specified reading and calculating on each data block, and aggregating calculation results.

Abstract: Methods for creating emails with complex graphics and customized web fonts that are ensured to render consistently across multiple platforms and email client applications and systems that ensure consistent rendering of created emails with complex graphics and customized web fonts by automatically transforming, completely transparently, the contents of each created email to a form that renders properly across the platforms and email clients are disclosed.

Abstract: A columnar storage method and a query method and system for time series data. The storage method includes: dividing a column of time series data into a plurality of pages, wherein each page stores a part of data points of the column of time series data and the sum of the data points stored in all the pages is all the data points in the column of time series data (S1); and setting two parts, i.e., a page header and a page body, for each page, storing summary index information of all the data points in the page in the page header of the page and storing data value information of all the data points in the page in the page body of the page (S2).

Abstract: A system and method for highly efficient encoding of data that includes extended functionality for asymmetric encoding/decoding and network policy enforcement. In the case of asymmetric encoding/decoding the original data is encoded by an encoder according to a codebook and sent to a decoder, but the output of the decoder depends on data manipulation rules applied at the decoding stage to transform the decoded data into a different data set from the original data. In the case of network policy enforcement, a behavior appendix into the codebook, such that the encoder and/or decoder at each node of the network comply with network behavioral rules, limits, and policies during encoding and decoding.

Abstract: A method for automatically identifying applications that circumvent permissions. The method includes logging network traffic transmitted by one or more computing devices while the one or more computing devices execute one or more applications, identifying, based on analyzing the logged network traffic and permissions granted to the one or more applications, those of the one or more applications that caused permission-protected data to be transmitted without having permission to access that data as circumventing permissions, and generating a report indicating the applications that were identified as circumventing permissions.

Abstract: In some examples, a system computes respective values for corresponding data value indicators added to and removed from a deduplication data store in which duplicated data values have been eliminated, where each respective data value indicator of the data value indicators represents presence of a unique data value in the deduplication data store. The system updates an estimator based on the respective values, to reflect an addition of a first data value indicator to the deduplication data store and a removal of a second data value indicator from the deduplication data store. The system computes, using the updated estimator, a parameter relating to data deduplication at the deduplication data store.

Abstract: A computer system implements a file system migration by determining a destination namespace of a destination file system based at least in part on a namespace of a source file system. At least a portion of the destination namespace includes an organization that is altered as compared to a corresponding portion of the source namespace.

Abstract: Embodiments are disclosed for a method for private transfer learning. The method includes generating a machine learning model comprising a training application programming interface (API) and an inferencing API. The method further includes encrypting the machine learning model using a predetermined encryption mechanism. The method additionally includes copying the encrypted machine learning model to a trusted execution environment. The method also includes executing the machine learning model in the trusted execution environment using the inferencing API.

Abstract: A primary storage system reads a plurality of journal data, and performs collective compression that is compression of data that is at least a part of the plurality of pieces of journal data in the plurality of journals and is larger than a size of one journal data. The collectively compressed data, which is a plurality of pieces of journal data subjected to collective compression, is a transfer target from the primary storage system to the secondary storage system. The journal is journal data and metadata including a write order of the journal data and associated with the journal data. The journal data is a copy of data written in the primary volume. The secondary storage system acquires a plurality of pieces of journal data by decompressing one or more pieces of collectively compressed data, and writes the plurality of pieces of journal data to the secondary volume.

Abstract: The present disclosure involves systems, software, and computer implemented methods for compressed columnar data search using fingerprints. One example method includes compressing columnar data that includes dividing the columnar data into multiple data blocks and generating a fingerprint for each data block, storing the compressed columnar data and the generated fingerprints in an in-memory database, receiving a query for the columnar data, for each in-memory data block stored in the in-memory database, determining whether the in-memory data block satisfies the query and in response to a determination that the in-memory data block does not satisfy the query, pruning the in-memory data block from the multiple data blocks to generate an unpruned set of data blocks, decompressing the unpruned set of data blocks, and performing a query search on the decompressed unpruned set of data blocks for the received query.

Abstract: Systems and method for providing tier selection for data based on a weighted flash fragmentation factor. A weighted flash fragmentation factor is determined indicating a severity of fragmentation in a non-volatile storage based on a logical block address range in a logical-to-physical mapping table for data from a host device to be stored in the tiered data storage system. The factor is shared with the host device to determine a tier selection. The data is stored according to the tier selection based on the factor.

Abstract: A system and method for generating a predicted vehicle trajectory includes a generative adversarial network configured to receive a trajectory vector of a target vehicle and generate a set of latent state vectors using the received trajectory vector and an artificial neural network. The latent state vectors each comprise a high-level sub-vector, ZH. The GAN enforces ZH to be correlated to an annotation coding representing semantic categories of vehicle trajectories. The GAN selects a subset, from the set of latent state vectors, using farthest point sampling and generates a predicted vehicle trajectory based on the selected subset of latent state vectors.

Abstract: Systems, apparatuses, and methods related to a computing tile are described. The computing tile may perform operations on received data to extract some of the received data. The computing tile may perform operations without intervening commands. The computing tile may perform operations on data streamed through the computing tile to extract relevant data from data received by the computing tile. In an example, the computing tile is configured to receive a command to initiate an operation to reduce a size of a block of data from a first size to a second size. The computing tile can then receive a block of data from a memory device coupled to the apparatus. The computing tile can then perform an operation on the block of data to extract predetermined data from the block of data to reduce a size of the block of data from a first size to a second size.

Abstract: Efficient aggregation of time series data is disclosed, including: obtaining a first entry value corresponding to an item, wherein the first entry value comprises a first recorded data point that is associated with a first time interval; generating a compressed block based at least in part on compressing the first entry value with at least a second entry value; storing the compressed block in a document corresponding to the item; determining that the item matches an aggregation search query; decompressing the compressed block from the document corresponding to the item to obtain the first entry value and the second entry value; and generating an aggregation result in response to the aggregation search query based on at least a portion of the first entry value and the second entry value.

Abstract: A system and method for standardizing user interface elements are presented. A first application is identified having a higher use metric than a second application, the first application including one or more user interface elements that have one or more respective parameters. The second application has one or more user interface elements that are similar to the user interface elements of the first application and has one or more respective parameters that are different than the respective parameters of the user interface elements of the first application. A determination of similarity is made between the user interface elements applications based upon at least one predetermined criterion. Based on the determination, one or more parameters of the user interface elements of the second application are modified to match one or more parameters of the first application.

Abstract: A method of encoding input data includes dividing the input data into a plurality of data packets, an input packet of the plurality of data packets including a plurality of digits in a first base system, base-converting the input packet from the first base system to generate a base-converted packet including a plurality of converted digits in a second base system, the second base system having a base value lower than that of the first base system, and incrementing the converted digits to generate a coded packet for transmission through a communication channel.

Abstract: A method for data reduction may comprise computing (i) a first sketch of a first segment and (ii) a second sketch of a second segment. The first sketch and the second sketch may each comprise a set of features that are representative of or unique to the corresponding first and second segments. The method also comprise processing the first sketch and the second sketch to generate a similarity metric indicative of whether the second segment is similar to the first segment. The method may further comprise (1) performing a differencing operation on the second segment relative to the first segment when the similarity metric is greater than or equal to a similarity threshold, or (2) storing the first segment and the second segment in a database without performing the differencing operation when the similarity metric is less than the similarity threshold.