Method and system for performing a proactive copy operation for a spare persistent storage
In general, the invention relates to a method for managing data. The method includes detecting a failure of a persistent storage device of a plurality of persistent storage devices, and in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using proactive copy metadata, checkpoint metadata, and slice metadata, wherein the data is a copy of data that was stored in the persistent storage device.
Latest DELL PRODUCTS L.P. Patents:
- SYSTEMS AND METHODS FOR SECURE SECRET PROVISIONING OF REMOTE ACCESS CONTROLLERS
- CONFIGURABLE HOT KEYS FOR AN INPUT DEVICE OF AN INFORMATION HANDLING SYSTEM
- ALLOWING A NETWORK FILE SYSTEM (NFS) CLIENT INFORMATION HANDLING SYSTEM MORE THAN ONE SESSION IN PARALLEL OVER A SAME NETWORK INTERFACE CARD (NIC)
- CLUSTER MANAGEMENT PLUGIN WITH SUPPORT FOR MULTIPLE CLUSTERS AND CLUSTER MANAGERS
- Adaptive Segmentation of Data Center Asset Telemetry Information
Computing devices may include any number of internal components such as processors, memory, and persistent storage. Each of the internal components of a computing device may be used to generate data. The process of generating, storing, and backing-up data may utilize computing resources of the computing devices such as processing and storage. The utilization of the aforementioned computing resources to generate backups may impact the overall performance of the computing resources.
SUMMARYIn general, in one aspect, the invention relates to a method for managing data. The method includes detecting a persistent storage device failure in a persistent storage device of a plurality of persistent storage devices, and in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using checkpoint metadata and slice metadata, wherein the data is a copy of data that was stored in the persistent storage device and wherein the spare persistent storage device is not one of the plurality of persistent storage device.
In general, in one aspect, the invention relates to a system that includes a processor and data processor, which when executed by the processor performs a method. The method comprises detecting a persistent storage device failure in a persistent storage device of a plurality of persistent storage devices, and in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using checkpoint metadata and slice metadata, wherein the data is a copy of data that was stored in the persistent storage device and wherein the spare persistent storage device is not one of the plurality of persistent storage device.
In general, in one aspect, the invention relates to a non-transitory computer readable medium which includes computer readable program code, which when executed by a computer processor enables the computer processor to perform a method. The method includes detecting a persistent storage device failure in a persistent storage device of a plurality of persistent storage devices, and in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using checkpoint metadata and slice metadata, wherein the data is a copy of data that was stored in the persistent storage device and wherein the spare persistent storage device is not one of the plurality of persistent storage device.
Specific embodiments will now be described with reference to the accompanying figures. In the following description, numerous details are set forth as examples of the invention. It will be understood by those skilled in the art that one or more embodiments of the present invention may be practiced without these specific details and that numerous variations or modifications may be possible without departing from the scope of the invention. Certain details known to those of ordinary skill in the art are omitted to avoid obscuring the description.
In the following description of the figures, any component described with regard to a figure, in various embodiments of the invention, may be equivalent to one or more like-named components described with regard to any other figure. For brevity, descriptions of these components will not be repeated with regard to each figure. Thus, each and every embodiment of the components of each figure is incorporated by reference and assumed to be optionally present within every other figure having one or more like-named components. Additionally, in accordance with various embodiments of the invention, any description of the components of a figure is to be interpreted as an optional embodiment, which may be implemented in addition to, in conjunction with, or in place of the embodiments described with regard to a corresponding like-named component in any other figure.
Throughout this application, elements of figures may be labeled as A to N. As used herein, the aforementioned labeling means that the element may include any number of items and does not require that the element include the same number of elements as any other item labeled as A to N. For example, a data structure may include a first element labeled as A and a second element labeled as N. This labeling convention means that the data structure may include any number of the elements. A second data structure, also labeled as A to N, may also include any number of elements. The number of elements of the first data structure and the number of elements of the second data structure may be the same or different.
In general, embodiments of the invention relate to a method and system for storing data and metadata in a data cluster. More specifically, embodiments of the invention relate to proactively protecting data stored in the data cluster using checkpoint zones (see e.g.,
In one or more embodiments of the invention, the host (100) utilizes the data cluster (110) to store data. The data stored may be backups of databases, files, applications, and/or other types of data without departing from the invention.
In one or more embodiments of the invention, the host (100) is implemented as a computing device (see e.g.,
In one or more embodiments of the invention, the host (100) is implemented as a logical device. The logical device may utilize the computing resources of any number of computing devices and thereby provide the functionality of the host (100) described throughout this application.
In one or more embodiments of the invention, the data cluster (110) stores data, metadata, and/or backups of data generated by the host (100). The data and/or backups may be deduplicated versions of data obtained from the host. The data cluster may, via an erasure coding procedure, store portions of the deduplicated data across persistent storage devices operating in the data cluster (110).
As used herein, deduplication refers to methods of storing only portions of files (also referred to as file segments or segments) that are not already stored in persistent storage. For example, when multiple versions of a large file, having only minimal differences between each of the versions, are stored without deduplication, storing each version will require approximately the same amount of storage space of a persistent storage. In contrast, when the multiple versions of the large file are stored with deduplication, only the first version of the multiple versions stored will require a substantial amount of storage. Once the first version is stored in the persistent storage, the subsequent versions of the large file subsequently stored will be de-duplicated before being stored in the persistent storage resulting in much less storage space of the persistent storage being required to store the subsequently stored versions when compared to the amount of storage space of the persistent storage required to store the first stored version.
In one or more embodiments of the invention, an erasure coding procedure includes dividing the obtained data into portions, referred to as data chunks. Each data chunk may include any number of data segments associated with the obtained data. The individual data chunks may then be combined (or otherwise grouped) into data slices (also referred to as Redundant Array of Independent Disks (RAID) slices). One or more parity values are then calculated for each of the aforementioned slices. The number of parity values may vary based on the erasure coding algorithm that is being used as part of the erasure coding procedure. The chunks of a data slice may then be stored across different persistent storage devices in a data cluster. Any chunk within a data slice may be reconstructed using the other chunks in the data slice. Non-limiting examples of erasure coding algorithms are RAID-3, RAID-4, RAID-5, and RAID-6. Other erasing coding algorithms may be used without departing from the invention.
Continuing with the discussion of
In one or more embodiments of the invention, the data processor (120) is a device (physical or logical) that includes the functionality to perform erasure encoding on data obtained from a host (e.g., 100,
In one or more of embodiments of the invention, the data processor (120) is implemented as computer instructions, e.g., computer code, stored on a persistent storage that when executed by a processor of a persistent storage device (e.g., 130A, 130N) cause the persistent storage device to provide the aforementioned functionality of the data processor (120) described throughout this application and/or all, or a portion thereof, of the methods illustrated in
In one or more embodiments of the invention, the data processor (120) is implemented as a computing device (see e.g.,
In one or more embodiments of the invention, the data processor (120) is implemented as a logical device. The logical device may utilize the computing resources of any number of computing devices and thereby provide the functionality of the data processor (120) described throughout this application and/or all, or a portion thereof, of the methods illustrated in
In one or more embodiments of the invention, the persistent storage devices (130A, 130N) store data and rebuild data. The data may be data chunks and/or parity chunks. The persistent storage devices (130A, 130N) may include non-volatile storage (also referred to as persistent storage). In other words, the data stored in the persistent storage devices (130A, 130N) is not lost or removed when the persistent storage devices (130A, 130N) lose power. For additional details regarding the persistent storage devices, see, e.g.,
In one or more embodiments of the invention, the persistent storage devices (130A, 130N) are implemented as computing devices (see e.g.,
In one or more embodiments of the invention, the persistent storage devices (130A, 130N) are implemented as logical devices. The logical devices may utilize the computing resources of any number of computing devices and thereby provide the functionality of the persistent storage devices (130A, 130N) described throughout this application.
In one or more embodiments of the invention, the spare persistent storage device (140) stores data like the persistent storage devices (130A, 130N). The data may be data chunks and/or parity chunks. However, unlike the persistent storage devices (130A, 130N), the spare persistent storage device (140) is a passive persistent storage device. In other words, the spare persistent storage device (140) does not participate in the direct storage of data from the host (e.g., 100,
In one or more embodiments of the invention, the spare persistent storage device (140) is implemented as a computing device (see e.g.,
In one or more embodiments of the invention, the spare persistent storage device (140) is implemented as a logical devices. The logical device may utilize the computing resources of any number of computing devices and thereby provide the functionality of the spare persistent storage device (140) described throughout this application.
The spare persistent storage device (140) may include more storage capacity than the persistent storage devices (130A, 130N) in order to store larger quantities of data (e.g., data chunks and parity chunks) as a result of checkpoint zone operations, proactive copy operations, and persistent storage device rebuild operations. For additional information regarding checkpoint zone operations, proactive copy operations, and persistent storage device rebuild operations, refer to
In one or more embodiments of the invention, the processor (152) is a component that processes data and processes requests. The processor (142) may be, for example, a central processing unit (CPU). The processor (152) may be other types of processors without departing from the invention. The processor (152) may process a request to store data and rebuild data using data stored in persistent storage (160) or from other persistent storage devices (e.g., 130A, 130N,
In one or more embodiments of the invention, the health monitor (154) is a component that monitors the health of persistent storage (160) of the persistent storage device (150). The health monitor (154) may be, for example, computer instructions that when executed by a processor (e.g., 152) of the persistent storage device (150), enables the persistent storage device (150) to perform the functionality of the health monitor (154).
In one or more embodiments of the invention, the health monitor (154) is implemented using a physical device. The physical device may include circuitry. The physical device may be, for example, a field-programmable gate array, application specific integrated circuit, programmable processor, microcontroller, digital signal processor, or other hardware processor. The physical device may be adapted to provide the functionality of the health monitor (154) described throughout this application. The health monitor may be implemented using another type of physical device without departing from the invention.
The health monitor (154) may detect and report various indicators of persistent storage (160) reliability with the intent of anticipating persistent storage (160) failures. The indicators of persistent storage (160) reliability may include attributes that have threshold values that should not be passed under normal operating conditions of the persistent storage device (150). The health monitor (154) may monitor these attributes and send a notification to the data processor (120,
As discussed above, the persistent storage (160) may store data. The data stored in persistent storage 160) may include data chunks (162A, 162M) and parity chunks (164A, 164P). Each of these data structures is discussed below. The persistent storage (160) may store other and/or additional data without departing from the invention.
The persistent storage (160) may be implemented using physical storage devices and/or logical storage devices. The physical storage devices may include any combination of hard disk drives, solid state disk drives, tape drives, and/or any other physical storage mediums for the storage of data.
The logical storage devices (e.g., virtualized storage) may utilize any quantity of hardware storage resources of any number of computing devices for storing data. For example, the persistent storage (160) may utilize portions of any combination of hard disk drives, solid state disk drives, tape drives, and/or any other physical storage medium of any number of computing devices.
In one or more embodiments of the invention, a data chunk (162A, 162M) is a data structure that includes a portion of data that was obtained from a host. The data chunks (162A, 162M) may be (but are not required to be) deduplicated by a data processor and obtained by the persistent storage device (150) from the data processor. Each of the data chunks (162A, 162M) may be used by the persistent storage device (150) (or another persistent storage device) to reconstruct another data chunk or a parity chunk based on an erasure coding algorithm that was applied to the other data chunk or parity chunk.
In one or more embodiments of the invention, a parity chunk (164A, 164P) is a data structure that includes a parity value generated using an erasure coding algorithm. The parity value may be generated by applying the erasure coding algorithm to one or more data chunks stored in the persistent storage device (150) or other data nodes. Each of the parity chunks (164A, 164P) may be used by the persistent storage device (150) (or another persistent storage device) to reconstruct another parity chunk or a data chunk based on an erasure coding algorithm that was applied to the other parity chunk or data chunk.
In one or more embodiments of the invention, slice metadata (210) includes metadata associated with data slices. Each data slice may have an associated metadata slice entry (e.g., 212A, 212N) generated by the data processor (120,
In one or more embodiments of the invention, checkpoint metadata (220) includes metadata associated with checkpoints created during a checkpoint zone operation. The checkpoint metadata (220) may include information regarding data and/or parity chunks within a checkpoint zone that were selected by the data processor (120,
In one or more embodiments of the invention, the proactive copy metadata (230) includes metadata associated with data and/or parity chunks that were copied and stored in the spare persistent storage device during a proactive copy operation. The data processor (120,
In step 300, a determination is made as to whether a checkpoint threshold has been satisfied. In one or more embodiments of the invention, the checkpoint threshold is the minimum amount of data slices stored across the persistent storage devices required in order to generate a checkpoint zone. The data processor may check the number of data slices not already included in previously generated checkpoint zones and compare it with the checkpoint threshold. If the checkpoint threshold is satisfied, the method proceeds to step 302. If the checkpoint threshold is not satisfied, the method performs step 300 again.
In step 302, in response to the determination of step 300 that the checkpoint threshold is satisfied, a checkpoint zone that includes a set of data slices is identified. In one or more embodiments of the invention, the checkpoint zone includes a multiple of data slices. The data slices may include data and parity chunks stored across the persistent storage devices in the data cluster.
The data processor may then generate checkpoint metadata. The checkpoint metadata may include checkpoint zone information associated with the newly identified checkpoint zone. The checkpoint zone information may include, for example, persistent storage identifiers associated with the checkpoint zone, data slice identifiers for data slices included in the checkpoint zone, chunk identifiers of data and parity chunks within the checkpoint zone, and chunk storage locations, as discussed above. The checkpoint zone information may include other and/or additional information regarding the identified checkpoint zone without departing from the invention.
In step 304, a subset of data and parity chunks within the checkpoint zone is selected. In one or more embodiments of the invention, the subset of data and parity chunks within the checkpoint zone may be selected in a rotating fashion (e.g., see
The subset of data and parity chunks within the checkpoint zone may be selected via other and/or additional methods without departing from the invention. The subset of data and parity chunks within the checkpoint zone may include a sufficient number of data and parity chunks to rebuild or replace at least one data or parity chunk associated with each persistent storage device in the checkpoint zone without requiring additional data and/or parity chunks.
After selecting the subset of data and parity chunks within the checkpoint zone, the data processor may update the checkpoint metadata associated with the checkpoint zone to include data and parity chunk information regarding the selected subset of data and parity chunks. The selected subset of data and parity chunk information may include, for example, chunk identifiers and storage locations as discussed above. The selected subset of data and parity chunks may include other and/or additional information regarding the selected subset of data and parity chunks without departing from the invention.
In step 306, the writing of copies of the selected data and parity chunks to a spare persistent storage device to create a checkpoint is initiated. The data processor may send a request to each persistent storage device associated with the checkpoint zone to copy and send the selected data and parity chunk copies to the spare persistent storage device. In response to the request, the persistent storage devices may each copy and send their selected data and/or parity chunks to the spare persistent storage device. Upon receiving the selected data and parity chunk copies from the persistent storage devices, the spare persistent storage device may store the selected data and parity chunk copies in persistent storage and associate the selected data and parity chunk copies with each other to create a checkpoint.
After creating a checkpoint, the spare persistent storage device may send confirmation to the data processor. The confirmation may include checkpoint information. The checkpoint information may include a checkpoint identifier, chunk identifiers, and/or chunk storage locations within the spare persistent storage device as discussed above. The data processor may update the checkpoint metadata to include the checkpoint information.
In step 308, a determination is made as to whether a persistent storage device has failed or will potentially fail. In one or more embodiments of the invention, the data processor may receive a notification from a health monitor of a persistent storage device indicating that the persistent storage device may potentially fail. In such a scenario, the checkpoint zone operation may be stopped and a proactive copy operation may be performed (see e.g.,
The method may end following step 308.
Continuing with the discussion of
Continuing with the discussion of
Upon receiving the request to copy and send data and/or parity chunks not associated with previous checkpoint zones, the potentially failing persistent storage device may copy the specified chunks and send the copies to the spare persistent storage device. After receiving the chunks from the potentially failing persistent storage device, the spare persistent storage device may store the chunks in persistent storage.
The spare persistent storage device may then send confirmation to the data processor. The confirmation may include information regarding the newly stored chunks. The newly stored chunk information may include, for example, chunk identifiers and chunk storage locations. The newly stored chunk information may include other and/or additional information regarding the newly stored chunks without departing from the invention. The data processor may then update the proactive copy metadata with the information included in the confirmation.
In step 314, a determination is made as to whether the potentially failing persistent storage device failed. The data processor may attempt to store or access data within the potentially failing persistent storage device and the persistent storage device may be unavailable due to failure or the data processor may receive an error from a health monitor indicating that the persistent storage device failed. If the potentially failing persistent storage device failed, the method may end following step 314. If the potentially failing persistent storage device has not failed, the method proceeds to step 316.
In step 316, a determination is made as to whether the copying of the chunks not associated with a checkpoint in the potentially failing persistent storage device to the spare persistent storage device is complete. If the copying of the chunks not associated with a checkpoint in the potentially failing persistent storage device to the spare persistent storage device is complete, the method may proceed to step 318. If the copying of the chunks not associated with a checkpoint in the potentially failing persistent storage device to the spare persistent storage device is not complete, the method proceeds to step 314.
Continuing with the discussion of
The method may end following step 318.
Continuing with the discussion of
In step 322, the rebuilding of rebuilding chunks is initiated. The data processor may send a request to the spare persistent storage device to rebuild the rebuilding chunks locally. The request may include which chunks to use to rebuild the rebuilding chunks and the algorithm to use to rebuild the building chunks.
In one embodiment of the invention, the spare persistent storage device may not be able to rebuild the rebuilding chunks using only the data and/or parity chunks already stored in the spare persistent storage device as a result of the checkpoint zone operation and/or the proactive copy operations. As a result, the rebuilding chunks may require data and/or parity chunks not stored on the spare persistent storage device. The data processor may use slice metadata to identify the persistent storage devices in which the data and/or parity chunks required to rebuild the rebuilding chunks are stored. The data processor may then send requests to the persistent storage devices to copy and send the chunks required to rebuild the rebuilding chunks to the spare persistent storage device. The data processor may then send a request to the spare persistent storage device to rebuild the rebuilding chunks. After completing the rebuild of the rebuilding chunks, the spare persistent storage device may delete the chunks that are not associated with the failed persistent storage device.
The method may end following step 322.
Example
The following section describes an example. The example is not intended to limit the invention. The example is illustrated in
The data processor (412) then performs a checkpoint zone operation (the method depicted in
Additionally, each persistent storage device (420A, 420B, 420C) includes a data or parity chunk of each data slice (e.g., 470A, 470B, 470F). For example, data slice A (470A) includes DC1 (432), DC2 (434), and PC1 (436). DC1 (432) is stored in persistent storage device A (420A), DC2 (434) is stored in persistent storage device B (420B), and PC1 (436) is stored in persistent storage device C (420C). Data slice B (470B) includes DC3 (438), PC2 (440), and DC4 (442). DC3 (438) is stored in persistent storage device A (420A), PC2 (440) is stored in persistent storage device B (420B), and DC4 (442) is stored in persistent storage device C (420C). Similarly, each data slice up to and including data slice F (470F) is stored in this configuration as shown in
During the example checkpoint zone operation (which was performed at some point in time before the instant depicted in
Further, as discussed above in
In order to rebuild PC6 (462,
The spare persistent storage device (430) already includes the other chunks (PC2 (440,
End of Example
As discussed above, embodiments of the invention may be implemented using computing devices.
In one embodiment of the invention, the computer processor(s) (502) may be an integrated circuit for processing instructions. For example, the computer processor(s) may be one or more cores or micro-cores of a processor. The computing device (500) may also include one or more input devices (510), such as a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device. Further, the communication interface (512) may include an integrated circuit for connecting the computing device (500) to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, mobile network, or any other type of network) and/or to another device, such as another computing device.
In one embodiment of the invention, the computing device (500) may include one or more output devices (508), such as a screen (e.g., a liquid crystal display (LCD), a plasma display, touchscreen, cathode ray tube (CRT) monitor, projector, or other display device), a printer, external storage, or any other output device. One or more of the output devices may be the same or different from the input device(s). The input and output device(s) may be locally or remotely connected to the computer processor(s) (502), non-persistent storage (504), and persistent storage (506). Many different types of computing devices exist, and the aforementioned input and output device(s) may take other forms.
One or more embodiments of the invention may be implemented using instructions executed by one or more processors of the data management device. Further, such instructions may correspond to computer readable instructions that are stored on one or more non-transitory computer readable mediums.
One or more embodiments of the invention may improve the operation of one or more computing devices. More specifically, embodiments of the invention improve the reliability of storing data and the efficiency of rebuilding data in a data cluster. In one embodiment of the invention, the reliability and efficiency is improved by performing checkpoint zone operations in a manner that copies and sends a subset of data stored in persistent storage devices to a spare persistent storage device of a data cluster in the background of data storage operations with minimal impact on the performance of the data storage operations of the data cluster. The checkpoint zone operations enable a spare persistent storage device to obtain data from persistent storage devices in a data cluster before a persistent storage device fails. Upon a persistent storage device failure, the spare persistent storage device uses the previously obtained data to rebuild and/or replace, at least a portion of, the data stored within the failed persistent storage device.
In other embodiments of the invention, the reliability and efficiency is improved by performing proactive copy operations along with the checkpoint zone operations in a manner that copies and sends data of a potentially failing persistent storage device to a spare persistent storage device of a data cluster in the background of data storage operations with minimal impact on the performance of the data storage operations of the data cluster. More specifically, one or more persistent storage devices may include a health monitor. In these scenarios, the health monitor includes functionality to detect when persistent storage (or a portion thereof) in a persistent storage device may fail. In the event of a potential persistent storage device failure is detected, the checkpoint zone operation is stopped and a proactive copy operation is performed. The proactive copy operation enables a spare persistent storage device to obtain data from a potentially failing persistent storage device before the potentially failing persistent storage device fails. Upon failure of the potentially failing persistent storage device, the spare persistent storage device uses the previously obtained data to rebuild or replace, at least a portion of, the failed persistent storage device.
In traditional data clusters, after a persistent storage device failure, the spare persistent storage device obtains associated data from healthy persistent storage devices in the data cluster and must use the obtained data to rebuild the entirety of the data that was stored on the failed persistent storage device. Embodiments of the invention improve the traditional data clusters by preemptively copying data from persistent storage devices (or a potentially failing persistent storage device) to a spare persistent storage device prior to a persistent storage device failure via checkpoint zone operations and/or proactive copy operations. As a result, the latency and computational efficiency required to rebuild a failed persistent storage device on a spare persistent storage device in a data cluster are improved.
Thus, embodiments of the invention may address the problem of inefficient use of computing resources. This problem arises due to the technological nature of the environment in which data is stored.
The problems discussed above should be understood as being examples of problems solved by embodiments of the invention disclosed herein and the invention should not be limited to solving the same/similar problems. The disclosed invention is broadly applicable to address a range of problems beyond those discussed herein.
While the invention has been described above with respect to a limited number of embodiments, those skilled in the art, having the benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the invention should be limited only by the attached claims.
Claims
1. A method for managing data, the method comprising:
- detecting a failure of a persistent storage device of a plurality of persistent storage devices; and
- in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using proactive copy metadata, checkpoint metadata, and slice metadata,
- wherein the data is a copy of data that was stored in the persistent storage device, and
- wherein the proactive copy metadata specifies: proactive copy data already written into the spare persistent storage device prior to the detecting, and when used in combination with the checkpoint metadata and the slice metadata, a first portion of the data to be rebuilt in the spare persistent storage device.
2. The method of claim 1, wherein the spare persistent storage device is not one of the plurality of persistent storage devices.
3. The method of claim 1, further comprising:
- prior to detecting the failure of the persistent storage device: detecting a potential failure of the persistent storage device; and
- in response to detecting the potential failure of the persistent storage device, initiating writing of the proactive copy data to the spare persistent storage device, wherein the proactive copy metadata is based on the proactive copy data.
4. The method of claim 3, wherein the proactive copy data comprises at least one data chunk and at least one parity chunk.
5. The method of claim 1, further comprising:
- prior to detecting a potential failure of the persistent storage device: identifying a checkpoint zone, wherein the checkpoint zone comprises chunks stored across the plurality of persistent storage devices; selecting a subset of the chunks, wherein the checkpoint metadata is based on the selected subset of the chunks; and initiating writing of copies of the subset of the chunks to the spare persistent storage device;
- after detecting the potential failure of the persistent storage device: ceasing generation of the checkpoint metadata.
6. The method of claim 5,
- wherein the checkpoint zone comprises a plurality of slices,
- wherein each of the chunks is associated with one of the plurality of slices; and
- wherein a cardinality of the plurality of slices is based on a checkpoint threshold.
7. A system, comprising:
- a processor;
- a data processor, which when executed by the processor performs a method, the method comprising: detecting a failure of a persistent storage device of a plurality of persistent storage devices; and in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using proactive copy metadata, checkpoint metadata, and slice metadata, wherein the data is a copy of data that was stored in the persistent storage device: wherein the proactive copy metadata specifies: proactive copy data already written into the spare persistent storage device prior to the detecting, and when used in combination with the checkpoint metadata and the slice metadata, a first portion of the data to be rebuilt in the spare persistent storage device.
8. The system of claim 7, wherein the spare persistent storage device is not one of the plurality of persistent storage devices.
9. The system of claim 7, wherein the method further comprises:
- prior to detecting the failure of the persistent storage device: detecting a potential failure of the persistent storage device; and
- in response to detecting the potential failure of the persistent storage device, initiating writing of the proactive copy data to the spare persistent storage device, wherein the proactive copy metadata is based on the proactive copy data.
10. The system of claim 9, wherein the proactive copy data comprises at least one data chunk and at least one parity chunk.
11. The system of claim 7, wherein the method further comprises:
- prior to detecting a potential failure of the persistent storage device: identifying a checkpoint zone, wherein the checkpoint zone comprises chunks stored across the plurality of persistent storage devices; selecting a subset of the chunks, wherein the checkpoint metadata is based on the selected subset of the chunks; and initiating writing of copies of the subset of the chunks to the spare persistent storage device;
- after detecting the potential failure of the persistent storage device: ceasing generation of the checkpoint metadata.
12. The system of claim 11,
- wherein the checkpoint zone comprises a plurality of slices,
- wherein each of the chunks is associated with one of the plurality of slices; and
- wherein a cardinality of the plurality of slices is based on a checkpoint threshold.
13. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method, the method comprising:
- detecting a failure of a persistent storage device of a plurality of persistent storage devices; and
- in response to the detecting, initiating a rebuilding of data in a spare persistent storage device using proactive copy metadata, checkpoint metadata, and slice metadata,
- wherein the data is a copy of data that was stored in the persistent storage device:
- wherein the proactive copy metadata specifies: proactive copy data already written into the spare persistent storage device prior to the detecting, and when used in combination with the checkpoint metadata and the slice metadata, a first portion of the data to be rebuilt in the spare persistent storage device.
14. The non-transitory computer readable medium of claim 13, wherein the spare persistent storage device is not one of the plurality of persistent storage devices.
15. The non-transitory computer readable medium of claim 13, wherein the method further comprises:
- prior to detecting the failure of the persistent storage device: detecting a potential failure of the persistent storage device; and
- in response to detecting the potential failure of the persistent storage device, initiating writing of the proactive copy data to the spare persistent storage device, wherein the proactive copy metadata is based on the proactive copy data.
16. The non-transitory computer readable medium of claim 15, wherein the proactive copy data comprises at least one data chunk and at least one parity chunk.
17. The non-transitory computer readable medium of claim 13, wherein the method further comprises:
- prior to detecting a potential failure of the persistent storage device: identifying a checkpoint zone, wherein the checkpoint zone comprises chunks stored across the plurality of persistent storage devices; selecting a subset of the chunks, wherein the checkpoint metadata is based on the selected subset of the chunks; and initiating writing of copies of the subset of the chunks to the spare persistent storage device;
- after detecting the potential failure of the persistent storage device: ceasing generation of the checkpoint metadata.
4780809 | October 25, 1988 | Woffinden et al. |
6098098 | August 1, 2000 | Sandahl et al. |
7636814 | December 22, 2009 | Karr et al. |
7987353 | July 26, 2011 | Holdaway et al. |
8099571 | January 17, 2012 | Driscoll et al. |
8161255 | April 17, 2012 | Anglin et al. |
8190835 | May 29, 2012 | Yueh |
8364917 | January 29, 2013 | Bricker et al. |
8386930 | February 26, 2013 | Dillenberger et al. |
8583769 | November 12, 2013 | Peters et al. |
8788466 | July 22, 2014 | Anglin et al. |
8868987 | October 21, 2014 | Wagner |
8874892 | October 28, 2014 | Chan et al. |
8898114 | November 25, 2014 | Feathergill et al. |
8949208 | February 3, 2015 | Xu et al. |
9122501 | September 1, 2015 | Hsu et al. |
9201751 | December 1, 2015 | Muthirisavenugopal et al. |
9278481 | March 8, 2016 | Hull |
9355036 | May 31, 2016 | Beard et al. |
9521198 | December 13, 2016 | Agarwala et al. |
9710367 | July 18, 2017 | Nagineni |
9749480 | August 29, 2017 | Katano |
9830111 | November 28, 2017 | Patiejunas et al. |
9898224 | February 20, 2018 | Marshak et al. |
10002048 | June 19, 2018 | Chennamsetty et al. |
10031672 | July 24, 2018 | Wang et al. |
10057184 | August 21, 2018 | Prahlad et al. |
10091295 | October 2, 2018 | Savic et al. |
10097620 | October 9, 2018 | Reddy et al. |
10152254 | December 11, 2018 | Kang |
10185624 | January 22, 2019 | Akutsu et al. |
10241695 | March 26, 2019 | Baptist et al. |
10339455 | July 2, 2019 | Parush-Tzur et al. |
10409778 | September 10, 2019 | Zhao et al. |
10452301 | October 22, 2019 | Farhan |
10503413 | December 10, 2019 | Gal et al. |
10503611 | December 10, 2019 | Srivastav et al. |
20010044879 | November 22, 2001 | Moulton et al. |
20030036882 | February 20, 2003 | Harper |
20030058277 | March 27, 2003 | Bowman-Amuah |
20030065873 | April 3, 2003 | Collins et al. |
20030172145 | September 11, 2003 | Nguyen |
20030177205 | September 18, 2003 | Liang et al. |
20040128587 | July 1, 2004 | Kenchammana-Hosekote et al. |
20050076049 | April 7, 2005 | Qubti et al. |
20050086557 | April 21, 2005 | Sato |
20050262385 | November 24, 2005 | McNeill, Jr. |
20070283011 | December 6, 2007 | Rakowski et al. |
20080244204 | October 2, 2008 | Cremelie et al. |
20090077165 | March 19, 2009 | Rhodes et al. |
20090094250 | April 9, 2009 | Dhuse et al. |
20090265360 | October 22, 2009 | Bachwani et al. |
20110099351 | April 28, 2011 | Condict |
20110246597 | October 6, 2011 | Swanson et al. |
20120096309 | April 19, 2012 | Kumar |
20130067459 | March 14, 2013 | Sannidhanam et al. |
20130086006 | April 4, 2013 | Colgrove |
20130151683 | June 13, 2013 | Jain et al. |
20130339818 | December 19, 2013 | Baker et al. |
20140020083 | January 16, 2014 | Fetik |
20140297603 | October 2, 2014 | Kim et al. |
20150012775 | January 8, 2015 | Cudak |
20150058582 | February 26, 2015 | Baldwin et al. |
20150095596 | April 2, 2015 | Yang |
20150161000 | June 11, 2015 | Kim et al. |
20150220400 | August 6, 2015 | Resch et al. |
20150286545 | October 8, 2015 | Brown |
20150355980 | December 10, 2015 | Volvovski et al. |
20160062674 | March 3, 2016 | Benight et al. |
20160085630 | March 24, 2016 | Gardner |
20160246537 | August 25, 2016 | Kim |
20170099187 | April 6, 2017 | Dale et al. |
20170147437 | May 25, 2017 | Borlick |
20170160983 | June 8, 2017 | Fiske et al. |
20170192865 | July 6, 2017 | Pan |
20170192868 | July 6, 2017 | Vijayan et al. |
20170206034 | July 20, 2017 | Fetik |
20170235609 | August 17, 2017 | Wires et al. |
20180018235 | January 18, 2018 | Arslan et al. |
20180060894 | March 1, 2018 | Beveridge et al. |
20180157532 | June 7, 2018 | Kumar et al. |
20180165169 | June 14, 2018 | Camp |
20180189109 | July 5, 2018 | Nagai et al. |
20180217898 | August 2, 2018 | Tormasov |
20180278597 | September 27, 2018 | Helms et al. |
20180307560 | October 25, 2018 | Vishnumolakala et al. |
20180322558 | November 8, 2018 | Padmanabh et al. |
20190104398 | April 4, 2019 | Owen et al. |
20190109772 | April 11, 2019 | Lipstone et al. |
20190197023 | June 27, 2019 | Chennamsetty et al. |
20190332473 | October 31, 2019 | Yang et al. |
20190379588 | December 12, 2019 | Rao |
20200050689 | February 13, 2020 | Tal et al. |
WO-2015114643 | August 2015 | WO |
- “Features—RoboHead”; Project Management Software for Marketing & Creative Teams, Aquent; 2018 (https://www.robohead.net/features) 1 page).
- “Make more time for the work that matters most”; Asana, Inc.; 2017 (https://asana.com/) (7 pages).
- “Online Project Management Tools & Features”; ProWorkflow; Jun. 2015 (https://www.proworkflow.com/features-project-management-tools/) (7 pages).
- Extended European Search Report issued in corresponding European Application No. 20152194.5, dated Jun. 29, 2020 (10 pages).
- Extended European Search Report issued in corresponding European Application No. 20152195.2, dated Jun. 2, 2020 (6 pages).
- Suzhen Wu et al. ;“Improving Reliability of Deduplication-based Storage Systems with Per-File Parity”; 2018 IEEE 37th International Symposium on Reliable Distributed Systems (SRDS); Salvador, Brazil; 2018; pp. 171-180 (DOI:10.1109/SRDS.2018.00028) (10 pages).
Type: Grant
Filed: Mar 6, 2020
Date of Patent: Sep 14, 2021
Assignee: DELL PRODUCTS L.P. (Hopkinton, MA)
Inventors: Dharmesh M. Patel (Round Rock, TX), Ravikanth Chaganti (Bangalore), Rizwan Ali (Cedar Park, TX)
Primary Examiner: Matthew M Kim
Assistant Examiner: Albert Li
Application Number: 16/810,947
International Classification: G06F 11/10 (20060101); G06F 11/30 (20060101); G06F 11/14 (20060101); G06F 11/20 (20060101); G06F 11/16 (20060101);