Abstract: A method and system for performing end-to-end encrypted semantic search and retrieval are disclosed. The system is configured to perform semantic searches of encrypted data, using product quantization and distance scores to facilitate efficient and secure retrieval of relevant data. User queries, indexes, document identifiers and document contents remain encrypted throughout the search process, offering a significant enhancement in privacy and security compared to known search methods. The method and system are particularly beneficial for applications that rely on semantic understanding in search queries, such as Generative AI and Retrieval-Augmented Generation (RAG), while maintaining strict data confidentiality.

Abstract: An encrypted search uses hash vectorization (HV) models, which are secure, one-way hash indices that are produced by a compression process, such as a modeling phase of Lempel-Ziv (LZ) parsing. Each HV model includes a hash filter and a chain vector. The hash filter is a Boolean quotient filter similar to Bloom filters, and allows for the quick elimination of negative query assessments. The hash filter is followed by a chain vector, which provides spatial modeling of hashed elements throughout the compressed and encrypted data, thereby providing increased levels of accuracy, efficiency, and query expressiveness as compared with known techniques.

Abstract: An encrypted search uses hash vectorization (HV) models, which are secure, one-way hash indices that are produced by a compression process, such as a modeling phase of Lempel-Ziv (LZ) parsing. Each HV model includes a hash filter and a chain vector. The hash filter is a Boolean quotient filter similar to Bloom filters, and allows for the quick elimination of negative query assessments. The hash filter is followed by a chain vector, which provides spatial modeling of hashed elements throughout the compressed and encrypted data, thereby providing increased levels of accuracy, efficiency, and query expressiveness as compared with known techniques.

Abstract: A method includes receiving an input data stream at a processor, and for each byte sequence from a plurality of byte sequences of the input data stream, a hash is generated and compared to a hash table to determine whether a match exists. If a match exists, that byte sequence is incrementally expanded to include one or more additional adjacent bytes from the input data stream, to produce multiple expanded byte sequences. Each of the expanded byte sequences is compared to the hash table to identify a maximum-length matched byte sequence from a set that includes the byte sequence and the plurality of expanded byte sequences. A representation of the maximum-length matched byte sequence is stored in the memory. If a match does not exist, a representation of that byte sequence is stored as a byte sequence literal in the memory.

Abstract: A data compression system includes an encoder for receiving a binary string of data which is partitioned into one or more binary segments and assigned Variable Length Codewords (VLCs) to be stored to a file or transmitted over a medium, such as a wired network or a wireless network. The system can also include a decoder for receiving the VLCs from a file or network and reproducing a binary string of data substantially identical to that which was input to the encoder. The system can also include a processor, a memory and/or a coder/decoder (CODEC) for compressing/decompressing a raw or compressed data stream received from a file or network.

Abstract: A data encoding system includes a non-transitory memory, a processor, a digital-to-analog converter (DAC) and a transmitter. The non-transitory memory stores a predetermined file size threshold. The processor is in operable communication with the memory, and is configured to receive data. The processor detects a file size associated with the data. When the file size is below the predetermined file size threshold, the processor compresses the data using a variable length codeword (VLC) encoder. When the file size is not below the predetermined file size threshold, the processor compresses the data, using a hash table algorithm. The DAC is configured to receive a digital representation of the compressed data from the processor and convert the digital representation of the compressed data into an analog representation of the compressed data. The transmitter is coupled to the DAC and configured to transmit the analog representation of the compressed data.

Abstract: A method for searching compressed, encrypted data includes receiving uncompressed data and identifying patterns thereof. Each pattern includes a predetermined number of bytes. Each pattern is hashed into a hash value, producing a set of hash values that is stored in a hash table. Each record of the hash table includes a hash value from the set of hash values and an associated position value. A Boolean filter is generated based on the hash table, each bit of the Boolean filter associated with a different hash value. A search string hash value is calculated based on a received search request. A location in the Boolean filter, having an address equal to the search string hash value, is inspected to determine whether a position stored at the location is true or false. If the position is true, at least a portion of the compressed data is flagged as relevant.

Abstract: A data encoding system includes a non-transitory memory, a processor, a digital-to-analog converter (DAC) and a transmitter. The non-transitory memory stores a predetermined file size threshold. The processor is in operable communication with the memory, and is configured to receive data. The processor detects a file size associated with the data. When the file size is below the predetermined file size threshold, the processor compresses the data using a variable length codeword (VLC) encoder. When the file size is not below the predetermined file size threshold, the processor compresses the data, using a hash table algorithm. The DAC is configured to receive a digital representation of the compressed data from the processor and convert the digital representation of the compressed data into an analog representation of the compressed data. The transmitter is coupled to the DAC and configured to transmit the analog representation of the compressed data.