EFFICIENT AND SECURE DATA STORAGE UTILIZING A DISPERSED DATA STORAGE SYSTEM
A method of securely storing data to a dispersed data storage system is disclosed. A data segment is arranged along the columns or rows of an appropriately sized matrix. Data slices are then created based on either the columns or the rows so that no consecutive data is stored in a data slice. Each data slice is then stored in a separate storage node.
Latest CLEVERSAFE, INC. Patents:
The present U.S. Utility patent application claims priority pursuant to 35 U.S.C. §121 as a divisional of U.S. Utility application Ser. No. 12/426,727, entitled “EFFICIENT AND SECURE DATA STORAGE UTILIZING A DISPERSED DATA STORAGE SYSTEM”, filed Apr. 20, 2009, which is hereby incorporated herein by reference in its entirety and made part of the present U.S. Utility patent application for all purposes.
FIELD OF THE INVENTIONThe present invention relates generally to systems, apparatus, and methods for securely storing data, and more particularly to systems, apparatus, and methods for secure distributed data storage using an information dispersal algorithm so that no one location will store an entire copy of stored data.
DESCRIPTION OF THE PRIOR ARTStoring data in digital form is a well-known problem associated with all computer systems, and numerous solutions to this problem are known in the art. The simplest solution involves merely storing digital data in a single location, such as a punch film, hard drive, or FLASH memory device. However, storage of data in a single location is inherently unreliable. The device storing the data can malfunction or be destroyed through natural disasters, such as a flood, or through a malicious act, such as arson. In addition, digital data is generally stored in a usable file, such as a document that can be opened with the appropriate word processing software, or a financial ledger that can be opened with the appropriate spreadsheet software. Storing an entire usable file in a single location is also inherently insecure as a malicious hacker only need compromise that one location to obtain access to the usable file.
To address reliability concerns, digital data is often “backed-up,” i.e., an additional copy of the digital data is made and maintained in a separate physical location. For example, a backup tape of all network drives may be made by a small office and maintained at the home of a trusted employee. When a backup of digital data exists, the destruction of either the original device holding the digital data or the backup will not compromise the digital data. However, the existence of the backup exacerbates the security problem, as a malicious hacker can choose between two locations from which to obtain the digital data. Further, the site where the backup is stored may be far less secure than the original location of the digital data, such as in the case when an employee stores the tape in her home.
Another method used to address reliability and performance concerns is the use of a Redundant Array of Independent Drives (“RAID”). RAID refers to a collection of data storage schemes that divide and replicate data among multiple storage units. Different configurations of RAID provide increased performance, improved reliability, or both increased performance and improved reliability. In certain configurations of RAID, when digital data is stored, it is split into multiple stripes, each of which is stored on a separate drive. Data striping is performed in an algorithmically certain way so that the data can be reconstructed. While certain RAID configurations can improve reliability, RAID does nothing to address security concerns associated with digital data storage.
One method that prior art solutions have addressed security concerns is through the use of encryption. Encrypted data is mathematically coded so that only users with access to a certain key can decrypt and use the data. While modern encryption methods are difficult to break, numerous instances of successful attacks are known, some of which have resulted in valuable data being compromised. Furthermore, if a malicious hacker should gain access to the encryption key associated with the encrypted data, the entirety of the data is recoverable.
While modern encryption tends to utilize block ciphers, such as, for example, 3-way, AES, Anubis, Blowfish, BMGL, CAST, CRYPTON, CS-Cipher, DEAL, DES, DESede, DESX, DFC, DFCv2, Diamond2, E2, FROG, GOST, HPC-1, HPC-2, ICE, IDEA, ISAAC, JEROBOAM, LEVIATHAN, LOKI91, LOKI97, MAGENTA, MARS, MDC, MISTY1, MISTY2, Noekeon, Noekeon Direct, Panama, Rainbow, RC2, RC4, RC4-drop, RC5, Rijndael, SAFER-K, SAFER-SK, SAFER+, SAFER++, SAFER++-64, Sapphire-II, Scream, Scream-F, SEAL-3.0, Serpent, SHARK, SKIPJACK, SNOW, SOBER, SPEED, Square, TEA, Twofish, WAKE-CFB, WiderWake4+1, WiderWake4+3, PBE-PKCS5, PBE-PKCS12, etc., other methods have been used in the past. One early form of encoding is transposition. Transposition involves the deterministic swapping of members within a set. For example, if a five member set X is defined as X={a,b,c,d,e}, a transposition function a may be defined as follows:
σ(0)=a
σ(1)=e
σ(2)=c
σ(3)=d
σ(4)=b
Therefore, the application of the transposition function to the entire set X would yield a new set X′={a, e, c, d, b}.
By transposing information transmitted in a message, the usability of the transposed information is reduced or eliminated. However, transposition schemes are easily broken by modern computers.
In 1979, two researchers independently developed a method for splitting data among multiple recipients called “secret sharing.” One of the characteristics of secret sharing is that a piece of data may be split among n recipients, but cannot be known unless at least t recipients share their data, where n≧t. For example, a trivial form of secret sharing can be implemented by assigning a single random byte to every recipient but one, who would receive the actual data byte after it had been bitwise exclusive orred with the random bytes. In other words, for a group of four recipients, three of the recipients would be given random bytes, and the fourth would be given a byte calculated by the following formula:
s′=s⊕ra⊕rb⊕rc,
where s is the original source data, ra, rb, and rc are random bytes given to three of the four recipients, and s′ is the encoded byte given to the fourth recipient. The original byte s can be recovered by bitwise exclusive-orring all four bytes together.
A cryptosystem, such as secret sharing, is called information-theoretically secure if its security derives purely from information theory; meaning that its security can be proven even if an adversary has access to unlimited computing power. As a secret sharing scheme can guarantee that no usable information can be recovered unless an attacker gains access to a threshold number of shares, secret sharing is information-theoretically secure. However, each data share is of equal size as the original data, so secret sharing makes for an inefficient storage mechanism.
All-or-nothing encryption is a recent development in cryptography, with the property that the entire cyphertext must be known before even a portion of the original data can be recovered. The original motivation behind all-or-nothing encryption was to increase the time required by brute force attacks to successfully compromise an encrypted cyphertext by a factor equal to the number of message blocks within the cyphertext. All-or-nothing encryption is described in “All-Or-Nothing Encryption and the Package Transform,” by Ronald L. Rivest, which is hereby incorporated by reference. Additional properties of all-or-nothing encryption are described in “Exposure-Resilient Functions and All-Or-Nothing Transforms,” by Ran Canetti, Yevgeniy Dodis, Shai Halevi, Eyal Kushilevitz, and Amit Sahai, as well as “On the Security Properties of OAEP as an All-Or-Nothing Transform,” by Victor Boyko, both of which are hereby incorporated by reference.
Dispersed data storage systems involved utilizing an information dispersal algorithm to slice data. Schemes for implementing dispersed data storage systems, such as dispersed data storage networks (“DDSNs”), are also known in the art. For example, U.S. Pat. No. 5,485,474, issued to Michael O. Rabin, describes a system for splitting a segment of digital information into n data slices, which are stored in separate devices. When the data segment must be retrieved, only m of the original data slices are required to reconstruct the data segment, where n>m.
Generally, dispersed data storage systems provide some level of security, as each data slice will contain less information than the original digital information. Furthermore, as each slice is stored on a separate computer, it will generally be harder for a malicious hacker to break into m computers and gather enough data slices to reconstruct the original information. However, depending on the information dispersal algorithm utilized, each data slice will contain up to 1/m part of the original data. Generally, the information will be retained in the data slice as it existed in the original digital information. Accordingly, by compromising a storage node, a malicious hacker could access up to 1/m part of the original data.
OBJECTS OF THE INVENTIONAccordingly, it is an object of this invention to provide a system, apparatus, and method for efficiently implementing a secure dispersed data storage system.
Another object of the invention is to provide a system, apparatus, and method for implementing a secure dispersed data storage system without significantly increasing the amount of stored data.
Another object of the invention is to provide a system, apparatus, and method for implementing a secure dispersed data storage system with a level of security that approaches information theoretic security.
Another object of the invention is to provide a system, apparatus, and method for implementing a secure dispersed data storage system so that an attacker can gain no information about a stored data segment unless the attacker is able to compromise a threshold number (m) data slices.
Other advantages of the disclosed invention will be clear to a person of ordinary skill in the art. It should be understood, however, that a system, method, or apparatus could practice the disclosed invention while not achieving all of the enumerated advantages, and that the protected invention is defined by the claims.
SUMMARY OF THE INVENTIONThe disclosed invention achieves its objectives by providing a method for securely storing data to a multi-node storage system, the method comprising the steps of arranging a data segment comprised of a plurality of data units in accordance with a matrix having j rows and k columns so that each data unit of the data segment is associated with a position of the matrix. The arranged data segment is then sliced in accordance with either the rows or the columns of the matrix to produce a plurality of data slices so that each data slice does not contain consecutive data units as they were arranged in the original data segment. Each of the data slices is then stored to a separate storage node.
In a further embodiment, a data string is padded with pad data prior to the step of arrangement as described above.
In an alternative embodiment, a method for securely storing data using a multi-node storage system is described, the method starting with the step of encrypting a data segment comprised of a plurality of data units using a suitable block cipher. The encrypted data segment is then arranged in accordance with a matrix having j rows and k columns so that each data unit of the encrypted data segment is associated with a position of the matrix. The encrypted arranged data segment is then sliced in accordance with either the rows or the columns of the matrix to produce a plurality of data slices so that each data slice does not contain consecutive data units as they were arranged in the original data segment. Each of the data slices is then stored to a separate storage node.
In a further embodiment of the disclosed invention, a method for securely storing data using a multi-node storage system comprises the steps of applying an all-or-nothing transformation to a data segment, slicing the all-or-nothing encrypted data segment using an information dispersal algorithm, and storing each of the resultant plurality of data slices on a separate storage node.
In a still further embodiment of the disclosed invention, a method of securely storing data using a multi-node storage system comprises the steps of encrypting a data segment using a first block cipher, applying an all-or-nothing transform to the encrypted data segment, slicing the all-or-nothing encrypted data segment using an information dispersal algorithm, and storing each of the resultant data slices on a separate storage node.
The application of an all or nothing transform can comprise the steps of generating an encryption key using, for example, a random key generator, and encrypting a data segment using the generated encryption key. The digest of the encrypted data is then calculated using an appropriate hashing algorithm, and the encryption key is exclusive-orred with the digest to produce an obfuscated encryption key. The obfuscated encryption key is then packaged with the data segment by, for example, appending the encryption key to the end of the data segment.
The disclosed invention can be operated by different computers. For example, in one embodiment of the disclosed invention, an access computer comprises a network port and a processor. The processor receives a data segment from the network port, arranges the data segment in accordance with a matrix having j rows and k columns, and slices the arranged data segment in accordance with the rows or columns of the matrix. The processor then sends each data slice to a separate storage node. In a separate embodiment, a stand-alone client comprises a network port and a processor. The processor retrieves a data segment and arranges the data segment in accordance with a matrix having j rows and k columns. The processor then slices the arranged data segment in accordance with the rows or columns of the matrix, and sends each data slice to a separate storage node.
Although the characteristic features of this invention will be particularly pointed out in the claims, the invention itself, and the manner in which it may be made and used, may be better understood by referring to the following description taken in connection with the accompanying drawings forming a part hereof, wherein like reference numerals refer to like parts throughout the several views and in which:
Turning to the Figures, and to
Where a higher level of security is required, the data segment 202 can be encrypted using an appropriate block cipher, such as DES or AES. While the use of encryption will increase security for the data storage system, it will also increase processor load on computers accessing the storage system. Accordingly, the performance of the system will be lowered, as computers writing data to the dispersed data storage system will need to encrypt data prior to writing it out, and systems reading data will likewise need to decrypt data. For those systems where high performance is more important than high security, the system administrator can elect to disable encryption. In this case, a moderate level of security is still attained by the disclosed system through the use of transposition, as explained below.
Regardless of whether the data segment is encrypted or not, the data within the data segment is arranged in a matrix 206. The matrix 206 is sized so that (1) every element of the data segment is assigned to a particular matrix entry, and (2) the number of data slices created is a multiple of the number of data slices created per data segment. In the depicted example, which assumes that eight data slices are created per data segment, an 8×2 matrix is used to fit the 16 data unit data segment, with the data segment arranged sequentially along the columns.
The data is then dispersed into data slices 208-215, each containing one row of data. As depicted, each data slice 208-215 contains entirely non-consecutive data from the original data segment.
A variety of sizes of matrices can be used to achieve many of the advantages of the disclosed system. For example, for a 16 byte data segment being stored on a system that slices data into 4 data slices, a 4×4 matrix could be used; data could be arranged along either the rows or columns, with the other serving as the basis for the data slices. However, while such an arrangement would increase security for the stored information, as no consecutive data units would be stored in a single slice, the optimal increase in security is achieved by sizing the matrix so that one dimension of the matrix, rows or columns, is equal to the threshold of the dispersed data storage network. For example, in system with a threshold m of eight and the data segment size is set to 16 bytes, an 8×2 matrix could be used as described above. It should be noted that additional, code slices containing redundant data would be generated if the total slice number n for the system is above eight. In this case, if a malicious hacker should recover two consecutive slices, a minimal number of consecutive data units will be recovered, i.e., two strips of data, each two data units in length.
Persons of skill in the art will realize that the decision to arrange data along the columns of the matrix is an arbitrary decision. For example, a 2×8 matrix could be used, and data could be arranged along the rows, instead of the columns. The data slices would then be made from the columns.
In step 708 the encrypted and transposed data segment is dispersed using a suitable information dispersal algorithm, such as Cauchy-Reed Solomon. The slices are then stored to different nodes of the dispersed data storage system, such as, for example, slice servers in step 710.
The first two embodiments of the disclosed invention involved the use of transposition to improve the security of data stored by a dispersed data storage network. The final two embodiments of the disclosed invention involve the use of all-or-nothing encryption to improve the security of data stored by a dispersed data storage network. Specifically, the use of all-or-nothing encryption, when combined with additional encryption, can provide the advantage of an attacker gaining access to no usable information unless the attacker is able to compromise a threshold m number of data slices. This approaches information theoretic security without the disadvantage of greatly increasing storage size.
The foregoing description of the invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the invention to the precise form disclosed. The description was selected to best explain the principles of the invention and practical application of these principles to enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention not be limited by the specification, but be defined by the claims set forth below.
Claims
1. A method operating on a computer, the method comprises:
- applying an all-or-nothing transformation to a data segment to produce an all-or-nothing data segment;
- encoding, using an information dispersal algorithm, the all-or-nothing data segment to produce a set of encoded data slices; and
- generating a set of write command to write the set of encoded data slices to storage units of a dispersed storage network.
2. The method of claim 1 wherein the applying an all-or-nothing transformation comprises:
- generating an encryption key;
- encrypting the data segment with said encryption key to produce an encrypted data segment;
- calculating a digest corresponding to the encrypted data segment;
- obfuscating the encryption key by exclusive ORing said encryption key with the digest to produce an obfuscated encryption key; and
- packaging the obfuscated encryption key with the encrypted data segment to produce the all-or-nothing data segment.
3. The method of claim 1 further comprises:
- encrypting the data segment using a first block cipher prior to the applying of the all-or-nothing transformation.
4. The method of claim 1 further comprises:
- retrieving, by a second computer, at least a decode threshold number of encoded data slices of the set of encoded data slices from the storage units;
- decoding, by the second computer using the information dispersal algorithm, the at least the decode threshold number of encoded data slices to recover the all-or-nothing data segment; and
- reversing, by the second computer, the all-or-nothing transformation on the recovered all-or-nothing data segment to recover the data segment.
5. The method of claim 1 further comprises:
- retrieving at least a decode threshold number of encoded data slices of the set of encoded data slices from the storage units;
- decoding, using the information dispersal algorithm, the at least the decode threshold number of encoded data slices to recover the all-or-nothing data segment; and
- reversing the all-or-nothing transformation on the recovered all-or-nothing data segment to recover the data segment.
6. The method of claim 5, wherein the reversing the all-or-nothing transformation comprises:
- separating the all-or-nothing data segment into an encrypted data segment and an obfuscated encryption key;
- calculating a digest based on the encrypted data segment;
- exclusive ORing the obfuscated encryption key with the digest to recover an encryption key; and
- decrypting the encrypted data segment using the encryption key to recover the data segment.
7. A computer comprising:
- a network port adapted to couple with a network and receive a data segment; and
- a processor coupled to the network port wherein the processor: applies an all-or-nothing transformation to the data segment to produce an all-or-nothing data segment; encodes, using an information dispersal algorithm, the all-or-nothing data segment to produce a set of encoded data slices; and generates a set of write commands to write the set of encoded data slices to storage nodes of a dispersed storage network.
8. The computer of claim 7, wherein the processor further functions to apply the all-or-nothing transformation by:
- generating an encryption key;
- encrypting said data segment with said encryption key to produce an encrypted data segment;
- calculating a digest corresponding to said encrypted data segment;
- obfuscating said encryption key by exclusive-ORing said encryption key with said digest to produce an obfuscated encryption key; and
- packaging said obfuscated encryption key with said encrypted data segment to produce the all-or-nothing encrypted data segment.
9. The computer of claim 7, wherein the processor further functions to:
- encrypt the data segment using a first block cipher prior to the applying of the all-or-nothing transformation.
10. The computer of claim 7, wherein the processor further functions to:
- retrieve, via the network port, at least a decode threshold number of encoded data slices of the set of encoded data slices from the storage nodes;
- decode, using the information dispersal algorithm, the at least the decode threshold number of encoded data slices to recover the all-or-nothing data segment; and
- reverse the all-or-nothing transformation on the recovered all-or-nothing data segment to recover the data segment.
11. The computer of claim 10, wherein the processor further functions to reverse the all-or-nothing transformation by:
- separating the all-or-nothing data segment into an encrypted data segment and an obfuscated encryption key;
- calculating a digest based on the encrypted data segment;
- exclusive ORing the obfuscated encryption key with the digest to recover an encryption key; and
- decrypting the encrypted data segment using the encryption key to recover the data segment.
Type: Application
Filed: Feb 27, 2015
Publication Date: Jun 18, 2015
Applicant: CLEVERSAFE, INC. (CHICAGO, IL)
Inventors: Wesley Leggette (Chicago, IL), Jason K. Resch (Chicago, IL)
Application Number: 14/633,315