Abstract: A system and method for encrypted data compression, which uses frequency analysis on data blocks within an input data stream to produce a prefix table, representing a first layer of transformation, and which applies a Burrow's-Wheeler transform (BWT) to the data inside the prefix table, representing a second layer of transformation, and which compresses the transformed data. In some implementations, the system and method may further include applying the BWT to a conditioned stream of genomic data, wherein the conditioned stream of data is accompanied by an error stream comprising the differences between the original data and the encrypted data.

Abstract: A system and method for efficient data storage, transfer, synchronization, and security using automated model monitoring and training. The system analyzes test datasets to detect data drift, retraining encoding and decoding algorithms as needed. New data sourceblocks are created and assigned codewords, compiling an updated codebook for distribution to connected devices. A novel dyadic distribution subsystem simultaneously compresses and encrypts data by transforming input streams into a dyadic distribution. This process generates a compressed main data stream and a secondary stream of transformation information, which are combined into a secure output. The system includes a network device manager for optimizing codebook distribution based on device resource usage. Operating in both lossless and lossy modes, the system offers flexible, efficient, and secure data handling across various network configurations.

Abstract: A system and methods for multi-type data compression or decompression with a virtual management layer, comprising. It incorporates a virtual management layer to organize incoming data types and select a compression or decompression system that utilizes a technique best suited for a particular data type. Associated data sets may be flagged prior to compression or decompression so that associated types may be preserved together after the compression or decompression process is complete. This approach allows each data type to be compressed or decompressed using a technique that is the most efficient for a particular data type. Additionally, the approach allows all information associated with a particular data set to be compressed or decompressed in some way.

Abstract: A system and method for lossy precompression for data compaction using automated model monitoring and training, wherein statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data sourceblocks, and pre-compression of data prior to processing and statistical analysis allows for the compaction of already compressed data into highly dense formats. The new data sourceblocks may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

Abstract: A system and method for lossy precompression for data compaction using automated model monitoring and training, wherein statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data sourceblocks, and pre-compression of data prior to processing and statistical analysis allows for the compaction of already compressed data into highly dense formats. The new data sourceblocks may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

Abstract: A system and method for highly efficient encoding of data that includes extended functionality for asymmetric encoding/decoding and network policy enforcement. In the case of asymmetric encoding/decoding the original data is encoded by an encoder according to a codebook and sent to a decoder, but the output of the decoder depends on data manipulation rules applied at the decoding stage to transform the decoded data into a different data set from the original data. In the case of network policy enforcement, a behavior appendix into the codebook, such that the encoder and/or decoder at each node of the network comply with network behavioral rules, limits, and policies during encoding and decoding.

Abstract: A system and method for enhancing lossy compressed data. The system receives a compressed data stream, decompresses it, and enhances the decompressed data using adaptive neural network models. Key features include data characteristic analysis, dynamic model selection from multiple specialized neural networks, and quality estimation with feedback-driven optimization. The system adapts to various data types and compression levels, recovering lost information without detailed knowledge of the compression process. It implements online learning for continuous improvement and includes security measures to ensure data integrity. The method is applicable to diverse data types, including financial time-series, images, and audio.

Abstract: Compression of medical imaging data using codebooks and entropy encoding. Medical imaging data such as tomosynthesis imagery data may be compressed using codewords based on frequency analysis. In an implementation sequential registration technique may be applied to the medical imaging data to create a plurality transformation matrices. The plurality of transformation matrices may be compressed using a matrix codebook. The compressed medical imaging data may be represented as an image codebook and the matrix codebook, providing secure storage and lossless compression of sensitive medical information.

Abstract: A system and method for highly efficient encoding of data that includes extended functionality for asymmetric encoding/decoding and network policy enforcement. In the case of asymmetric encoding/decoding the original data is encoded by an encoder according to a codebook and sent to a decoder, but the output of the decoder depends on data manipulation rules applied at the decoding stage to transform the decoded data into a different data set from the original data. In the case of network policy enforcement, a behavior appendix into the codebook, such that the encoder and/or decoder at each node of the network comply with network behavioral rules, limits, and policies during encoding and decoding.

Abstract: A system and method for intrusion detection with prediction and validation subsystems powered by machine learning. The system analyzes real-time codeword streams and historical data to predict potential intrusions before they fully manifest. It employs various machine learning models to extract features, identify patterns, and validate detected anomalies. The system continuously learns from validated events and false positives, improving its accuracy over time. An integrated encryption module secures sensitive data using a dyadic distribution-based algorithm, combining compression and encryption. This approach significantly reduces false positives, enhances threat detection capabilities, and provides robust data protection for cybersecurity applications.

Abstract: Data storage, transfer, synchronization, and security using automated system efficacy monitoring and model training is disclosed. Statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data sourceblocks. The new data sourceblocks may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

Abstract: A system and method for optimizing intrachip communication using machine learning-based codebook refinement is presented. The system employs an on-chip machine learning model to continuously analyze data patterns and update a codebook used for data compression in intrachip communication. Key aspects may comprise real-time data collection, feature extraction, performance monitoring, and gradual codebook updates. The system adapts to evolving data patterns, improving compression efficiency over time. A fallback mechanism ensures system stability by reverting to a conservative codebook if performance degrades. Security measures, including cryptographic signatures for updates and anomaly detection, are integrated. The system optimizes power consumption by adjusting operations based on the chip's power state. This adaptive approach significantly enhances intrachip communication efficiency, potentially improving overall chip performance and energy efficiency.

Abstract: A system and method for data storage, transfer, synchronization, and security using automated system efficacy monitoring and model training, wherein statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data chunklets. The new data chunklets may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

Abstract: A system and method for random-access manipulation of compacted data files, utilizing a reference codebook, a random-access engine, a data deconstruction engine, and a data deconstruction engine. The system may receive a data query pertaining to a data read or data write request, wherein the data file to be read from or written to is a compacted data file. A random-access engine may facilitate data manipulation processes by transforming the codebook into a hierarchical representation and then traversing the representation scanning for specific codewords associated with a data query request. In an embodiment, an estimator module is present and configured to utilize cardinality estimation to determine a starting codeword to begin searching the compacted data file for the data associated with the data query. The random-access engine may encode the data to be written, insert the encoded data into a compacted data file, and update the codebook as needed.

Abstract: A system and method for event-driven data communication using codebooks with protocol prediction and translation. This invention presents an advanced adaptive communication system that dynamically optimizes network protocols using machine learning-driven prediction and translation modules. The system analyzes real-time traffic patterns and historical data to anticipate communication needs, proactively switching to optimal protocols when beneficial. A sophisticated translation module, powered by large language models, enables seamless communication between systems using different protocols, including legacy systems. This approach enhances network efficiency, ensures backward compatibility, and future-proofs communication infrastructures, making it particularly valuable in complex, heterogeneous network environments.

Abstract: A system and method for data compression with intrusion detection, that measures in real-time the probability distribution of an encoded data stream, compares the probability distribution to a reference probability distribution, and uses one or more statistical algorithms to determine the divergence between the two sets of probability distributions to determine if an unusual distribution is the result of a data intrusion. The system further comprises a signature generating component which correlates anomalous event data with known vulnerabilities and exploits to create a signature based on statistical information of the anomalous event. Computed statistics may be compared against a signature database to determine if a data intrusion has occurred.

Abstract: A system and method for securing high-speed communications between processing units on computer chips, wherein a training data set is used to find patterns and associated smaller indices, or codewords, which are stored in a reference codebook library, and where reconstruction and deconstruction algorithms are used to encode and decode data as it is received. The codebook and algorithms may be stored in the firmware of a semiconductor which enable reduced resources and cost when transmitting data between or among devices that utilize such semiconductors.

Abstract: A system and method for implementing multilayer security in documents using multiple codebooks. The system encodes different sections of a document using codebooks associated with specific security clearance levels. Users can only access sections of the document for which they have appropriate clearance, with unauthorized sections appearing redacted. The system provides granular, section-level security while also offering data compression benefits. Additional features include dynamic reclassification, hierarchical codebooks, expiring access, multi-factor authentication, and user-specific watermarking.

Abstract: A system and method are disclosed for encoding and decoding QR codes using proprietary compression codebooks to increase information density and provide data security. Public data is encoded using a standard codebook while private data uses a proprietary codebook. The encoded data is combined into a single QR code. Decoding extracts the public and private portions and decompresses them using the appropriate codebooks.

Abstract: A system and method for data compression with encryption, that produces a conditioned data stream by replacing data blocks within an input data stream to bring the frequency of each data block closer to an ideal value, produces an error stream comprising the differences between the original data and the encrypted data, and compresses the conditioned data.