Abstract: A first archive of a first snapshot of a source storage is caused to be stored to a remote storage. At least a portion of content of the first archive is stored in data chunks stored in a first chunk object of the remote storage and the first archive is associated with a first data policy. A second archive of a second snapshot of the source storage is caused to be stored to the remote storage. At least a portion of content of the second archive is referenced from data chunks stored in the first chunk object and the second archive is associated with a second data policy. Policy compliance of the chunk object storing data chunks referenced by the first archive and the second archive that are different is automatically managed based on the first data policy and the second data policy that are different.

Abstract: A method, apparatus, and system for managing file level restore agents is provided. The operations comprising: determining a computing resource usage level at a data protection appliance; determining whether the computing resource usage level at the data protection appliance is below a first threshold; in response to determining that the computing resource usage level at the data protection appliance is below a first threshold, assigning a new file level restore-associated job to a first tier file level restore agent for performance; and in response to determining that the computing resource usage level at the data protection appliance is not below a first threshold, assigning the new file level restore-associated job to a second tier file level restore agent for performance.

Abstract: A method, apparatus, and system determines an optimal storage configuration of storing backup data. The method may include receiving a request from a client device for determining an optimal storage configuration for storing backup data of a client. The method may include determining a cloud utilization pattern of the backup data based on prior access activities and determining a first suitability score for the first storage class based on the cloud utilization pattern. The method may include, for each of the storage classes of the first storage provider, determining a suitability score for the corresponding storage class if the backup data were stored in the corresponding storage class. The method may include transmitting to the client device a recommendation of a second storage class of the first storage provider having a suitability score higher than the first suitability score based on the suitability score to reduce cloud resources usage.

Abstract: Disclosed are various approaches for implementing a navigation system that facilitates navigation through hierarchically-related views within a user interface using multiple independent entry points. The user interface can include multiple navigation components that each independently provide a user access to available views in the user interface. Upon a user selection for a view through interaction with one of the navigation components, a request for data used to build the view can be sent to a server. If the view is valid, an instance of the view is dynamically generated and displayed in the user interface and each navigation component is updated to be consistent with the selected view. If the view is invalid, updated navigation data is used to rebuild the navigation hierarchy and the navigation components are updated according to the updated navigation hierarchy.

Abstract: Techniques for processing data may include: receiving a plurality of data chunks for a data set; performing data deduplication processing for the plurality of data chunks; determining, in accordance with one or more criteria, whether a frequency distribution of a frequency histogram of digest byte frequencies is sufficiently uniform; and responsive to determining that the frequency distribution of the frequency histogram is not sufficiently uniform, performing processing to update data deduplication settings for the data set. Updating the data deduplication settings may include using a stronger hash algorithm and/or a larger size digest when generating subsequent digests. The data deduplication processing may include: determining, using a current hash algorithm, a plurality of digests for the plurality of data chunks of the data set; and updating the frequency histogram of digest byte frequencies for the data set in accordance the plurality of digests.

Abstract: A Thing Machine is provided having a processor, non-transitory memory, non-transitory computer readable media, and performable machine code P(TM). The P(TM) is comprised of a first set of performable machine code actions, having one or more performable machine code P(TM(i)) action, wherein each performable machine code P(TM(i)) action is configured as an implementation of an algorithmic procedure of a model, wherein a first P(TM(i)) provides an action of self-configuring a first vocabulary of Things in said non-transitory memory of the Thing Machine, said Things representative of Things that said processor can perform as actions, and the set of Things an action can act upon, and wherein at least one P(TM(i)) machine code action is performed to configure a second vocabulary of Things in the non-transitory memory of the Thing Machine representative of a core vocabulary through which an application can be provided.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.

Abstract: Apparatus and methods are disclosed, including identifying and tagging data in a group of volatile memory cells of a host device to be written to and maintained contiguously on non-volatile memory of a storage system, and writing the tagged data to the group of non-volatile memory cells. A host device includes a host processor and the group of volatile memory cells, and a storage system includes the group of non-volatile memory cells.

Abstract: The present disclosure provides a device for processing a programmable logic controller (PLC) program. In one embodiment, the device includes a storage containing PLC programs stored on a block basis, and a program block table defining storage locations of the PLC programs; and a controller configured: when a PLC program is modified during a PLC is operating, to determine a start address of a storage location where the modified PLC program is to be stored with referring to a block number of the modified PLC program, size information of the modified PLC program, and the program block table; and to store the modified PLC program in the storage based on the determined start address. According to the present disclosure, multiple users may simultaneously access a PLC and then may simultaneously modify PLC programs during the PLC is operating.

Abstract: A method includes compressing input data to form compressed data and comparing a size of the compressed data to a maximum allowed size determined from a fixed sector size for a lower tier of the multi-tier storage system and a minimum pad length for a pad that is stored in the same sector as the compressed data when the compressed data is migrated to the lower tier. When the size of the compressed data is greater than the maximum allowed size, the input data is stored instead of the compressed data in an upper tier of the multi-tier storage system.

Abstract: Embodiments provide for a primary storage array having multiple storage tiers. The multiple storage tiers include one or more performance tiers and at least one deduplicated storage tier storing deduplicated data. One embodiment provides for a data storage system including management logic to manage the deduplicated storage tier. The management logic can receive a request to store deduplicated data and respond to the request to store the deduplicated data based on workload statistics associated with the data. In one embodiment the response to the request includes storing at least a portion of the deduplicated data to a first storage device in the second set of storage devices and the workload statistics include input/output (I/O) statistics associated with the unit of deduplicated data. In other embodiments the response includes reallocating drives between storage tiers and/or providing migration hints with respect to data on the deduplicated storage tier.

Abstract: Systems, methods, and apparatuses can present optimized data migration. Embodiments can provide improved data migration processes using a data migration assistant that executes on a source system to facilitate the migration of documents and data to a destination system. The migration assistant can communicate with a migration controller server to authenticate and identify the migration assistant and migration project that the migration assistant is to be used for. The migration controller server can use the identifier to determine commands corresponding to the migration project to send to the migration assistant to collect data and make changes to the source system.

Abstract: In a method for allocating data to multiple disks for storage, a capability value of each of the multiple disks is obtained. The capability value is ratio of a performance indicator value of a disk to a maximum capacity value of the disk. A greatest capability value is identified from multiple capability value. Based on the greatest capability value and a maximum capacity value of each of the multiple disks, allocation shares of the multiple disks are determined. Based on the determined allocation shares, data is allocated for the multiple disks. Through the method, the overall storage performance of a storage system is improved.

Abstract: Systems and methods for backing up data. A backup operation can identify blocks that have changed since a most recent backup operation. The blocks that have changed since the most recent backup operation are compared with the blocks that are being used. Only the blocks that are both used and have changed are backed up. This can reduce the size of the backup be excluding blocks that may have changed but are no longer used.

Abstract: In the dedupe storage network priority to data replication needs to be given over the garbage collection (GC) activity. For this purpose GC running on any site should not impose any locking on its peer sites for replication. For replication activity GC activity must be completely transparent. In the dedupe storage network, the retention policy of a replicated image is controlled by the site where the image was originated. Distributed GC running on the originating site can only inform the replication sites the list of expired replicated images and cleanup of replicated garbage chunks from its remote FS database for corresponding replication sites. But it cannot cleanup garbage chunks from replication sites. Garbage chunks can only be cleaned by the local GC running on that site.

Abstract: Systems and methods are disclosed for editing metadata for data objects on a data storage device, which may be configured to identify data objects that have been modified and may be associated with storage locations of a file system. The data storage device may also be configured determine whether a first period of time has elapsed and may further be configured to determine whether a second period of time has elapsed or whether a threshold number of data objects in the file system have been modified in response to a first determination that the first time period has elapsed. The data storage device may further be configured to edit a set of metadata for a set of data objects in response to a second determination that the second period of time has elapsed or the threshold number of data objects in the file system have been modified.

Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: A system for dynamically utilizing data storage comprises a processor and a memory. The processor is configured to determine whether a data storage criterion is satisfied; and, in the event that the data storage criterion is satisfied: determine a new archiving threshold based on a target data storage usage level; and set the archiving threshold. The memory is coupled to the processor and is configured to provide the processor with instructions.

Abstract: A mechanism is provided in a data processing system for managing filesystem inodes. The mechanism monitors inode consumption in a filesystem. The mechanism periodically determines a number of inodes to add to the filesystem based on the inode consumption and adds the number of inodes to the filesystem.

Abstract: Apparatus, methods, and other embodiments associated with providing front end intelligence for monitoring back end state are described. One example method includes controlling a first entity to request that a second entity perform a multi-state stateful operation (e.g., file create). The method includes receiving notice from the second entity that the stateful operation has attained a first state (e.g., partial file create complete) and, upon receiving the notice, maintaining, at the first entity, information (e.g., sequence number, metadata) associated with the stateful operation. The method also includes, upon receiving notice that the second entity lost state before the stateful operation attained a second state, selectively providing to the second entity a portion of the information maintained at the first entity, and upon receiving notice from the second entity that the stateful operation has attained the second state, terminating maintaining the information associated with the stateful operation.

Abstract: A data storage method for a computing device is disclosed. The method includes: receiving, at a computing device, a file and a profile, wherein the profile includes a level of magnitude; storing, at the computing device in a first database, the file if the level of magnitude is greater than the lowest level; storing, at the computing device in a second database, the file if the level of magnitude is the highest level; and deleting, at the server, the file if the level of magnitude is the lowest.

Abstract: Methods and systems for implementing a network-backed file system are disclosed. Data indicative of a plurality of subscriptions by a plurality of clients to respective subsections of a file system is stored. The subscriptions comprise one or more read subscriptions and one or more write subscriptions. The read subscriptions are fulfilled by sending modified file chunks of subsections to subscribing clients. The write subscriptions are fulfilled by modifying file chunks of subsections based on write requests submitted by subscribing clients.

Abstract: System and method for the opportunistic allocation of copies of data across data storage resources in file systems. Embodiments may opportunistically and transparently trade storage space in a file system for performance. In embodiments, when data is stored to the file system, copies of the data may be opportunistically stored to two or more data storage resources. The data may then be opportunistically accessed from a less active resource if the resource storing a primary copy of the data is busy. The opportunistically stored copies of data remain online in the file system. In hierarchical file systems or tiered storage hierarchies, embodiments may allow higher-level storage tiers to “borrow” performance from lower-level tiers by opportunistically allocating copies of data from a higher-level tier within a lower-level tier. Copies of data stored on a data storage resource may be deleted if storage space is needed on that data storage resource.

Abstract: A system reserves and manages a hidden service partition through components of the hardware platform of a computing device. The hidden partition is not accessible by way of a host operating system on the computing device. A hardware platform controller provisions a portion of nonvolatile storage through configuration settings of the hardware platform controller. When the host system requests settings related to storage in the system, the request is routed through the interfaces of the hardware platform, and the hardware platform controller reports in accordance with the configuration settings, hiding the service partition. The hidden partition is dynamically modifiable through secure remote access to the hardware platform controller, not through the host system such as operating system or BIOS.

Abstract: Methods and systems for searching a backup volume use, in one embodiment, a move marker file to indicate attributes for a new name space and optionally indicate attributes for a prior name space of a directory on a source volume, the directory having been moved between backup snapshot operations in which data on the source volume is backed up to the backup volume. Other embodiments are also described.

Abstract: Various systems, processes, products, and techniques may be used to manage legacy data. In one general implementation, a system, process, and/or product for managing legacy data may include the ability to determine whether a data request has been received and, if a data request has been received, determine whether the data request is associated with legacy data of an external storage management system. If the data request is not associated with legacy data of an external storage management system, the system, process, and/or product may retrieve data from a local storage array, and if the data request is associated with legacy data of an external storage management system, the system, process, and/or product may request legacy data from an external storage management system. The system, process, and/or product may also generate a response to the data request.

Abstract: Mechanisms for managing data segments in a tiered storage system are provided. The mechanisms maintain at least one counter for each data segment in the tiered storage system. Each counter in the at least one counter counts a number of access operations to a corresponding data segment for a predetermined time interval. The mechanisms further perform one or more analytical operations based on one or more values of the at least one counter for each data segment to make residence determinations for each data segment. The mechanisms also adjust a storage location of one or more data segments in tiers of the tiered storage system to thereby move the one or more data segments to appropriate tiers of the tiered storage system based on results of the one or more analytical operations.

Abstract: A computer-implemented method for projecting hierarchical storage management functions may include enumerating objects stored on an endpoint system via an agent on the endpoint system. The computer-implemented method may also include transmitting, via a web service, information relating to the enumeration to a server. The computer-implemented method may further include receiving, via the web service, a hierarchical storage management policy from the server. The computer-implemented method may additionally include enacting the hierarchical storage management policy via the agent. Various other methods and systems are also disclosed.

Abstract: A method, article of manufacture, and apparatus for protecting data. In some embodiments, this includes taking a snapshot of a virtual machine, parsing the virtual machine to determine a header section and a file system, indexing the virtual machine based on the parsing, storing the index in a storage device, and storing the snapshot in a deduplicated data storage device.

Abstract: Renamed directory handling for incremental file migration is described. A directory is stored in a storage device. An inode number associated with the directory in a directory table is identified. A determination is made whether the inode number equals an inode number associated with another directory in the directory table. At least one of a file and a subdirectory associated with the directory in the file system is stored into the directory in the storage device if the inode number equals the inode number associated with another directory in the directory table.

Abstract: A system and method for setting global actions in a data storage system is described. In some examples, the system determines a policy based on information from the system, and implements that policy to the system. In some examples, the system adds or modifies global filters based on information from the system.

Abstract: Some embodiments of the invention provide techniques for performing system reset and/or system restore operations. Some embodiments store a record of any changes performed to a data file stored on a virtual hard disk using differencing virtual disks that are captured over time. To restore the file to the state in which it existed at any particular point in time, embodiments of the invention may allow for rolling back the system to a differencing virtual disk captured at or before that time and setting aside blocks of the differencing virtual disk chain captured after that point. To perform a system reset, the system may be rolled back to the parent virtual hard disk, by setting aside all of the blocks of the differencing virtual disk chain. Some embodiments of the invention provide for separating information on the system into a plurality of entities, and maintaining a separate set of differencing virtual disks for each entity.

Abstract: A large number of objects, such as objects representing beams and columns in an object-oriented enterprise engineering system, may be copied in a model database by partitioning the objects according to certain criteria into a number of ordered small copy groups and copying the objects in each group as an atomic operation. Objects that are to be copied are organized into the ordered groups, and the groups are copied in order, such that all predecessors of a given object are copied into the database before, or in the same small operation as, the given object. If a large copy operation abnormally terminates before all the small copy operations have been completed, the model database is, nevertheless, left in a consistent state, and the copy operation may be resumed from the point of interruption. Furthermore, the number of objects that may be copied is not constrained by the amount of memory available in the system.

Abstract: Disclosed is a method and system for selectively restoring file-level data from a disk image backup. In embodiments, a virtual machine backup may be performed by dividing a virtual machine virtual disk file into a plurality of discrete fixed-sized data blocks sharing a common index file that is stored on a backup medium, such as a hard drive, to form a backup set. The index file is referenced to determine which fixed-sized block contains volume information, such as a partition table, of the backed-up virtual machine file. The individual blocks are processed as a virtual filesystem which is mounted and presented to an access module, which traverses the filesystem and provide access to individual files in the image backup to a client process. The restore files may be delivered to the client in a container file, which may be compressed to increase transfer speed. The container file may include executable instructions for automatically restoring the files to a desired location.

Abstract: Time required for a file server to determine files to be migrated from a first storage region to a second storage region is reduced. A file service system coupled to the storage system for providing a first storage region and a second storage region stores identifiers of files that have been accessed by a client, in accessed file identifier information and determines the files to be migrated from the first storage region to the second storage region based on metadata of the files stored in the first storage region and the accessed file identifier information.

Abstract: A backup and restore system may present recovered backup data as local resources and in a read and write manner so that the recovered backup data may be accessed using an application that created the data. In some embodiments, the recovered data may be read only data, but a differencing virtual hard disk (VHD), Common Internet File System protocol, or differencing driver may be used to present a read/write version of the data. Additionally, the data may be presented using iSCSI or other technologies to present the data as local data, even though the data may be remotely located. The backup and restore system may additionally use VHD differencing technologies to create multiple backups.

Abstract: An aspect of the present invention stores files of a source directory in a target directory. In an embodiment, a unique identifier is generated for each of the files and a new location and a new name are generated for the file. The new location represents the specific sub-directory of the target at which the file is stored. The file is stored at the new location with the new name. Such storing in a new location with a new name can be advantageously used to address various issues in corresponding environments. In one environment, the target directory is stored in an embedded system, with limited resources and the source directory contains several files with substantial overlapping names (which can require substantial resources to search for a specific file). The unique identifiers are generated according to media transfer protocol (MTP), which generates an object identifier for each of the files/directories, etc.

Abstract: Systems and methods for retrieving data from a storage system having a plurality of storage pools are provided. The method comprises processing configurable data retrieval instructions to determine a first storage pool from which target backup data is to be retrieved, in response to a data restore request; and retrieving the target backup data from the first storage pool to satisfy the restore request. The configurable data retrieval instructions are managed by a source external to the storage system with administrative authority to change the configurable data retrieval instructions to optimize data restoration from the storage system.

Abstract: PourOver is the process of replicating databases across clusters, in order to achieve real and/or near real-time database consistency between clusters (e.g., continually provide the ability to construct a consistent database on the alternate cluster that is up to date within a few minutes worth of transactions).

Abstract: A system and method for enhancing data throughput in data warehousing environments by connecting multiple servers having local storages with designated external storage systems, such as, for example, those provided by SANs. The system and method may preserve a full reference copy of the data in a protected environment (e.g., on the external storage system) that is fully available. The system and method may enhance overall I/O potential performance and reliability for efficient and reliable system resource utilization.

Abstract: A mechanism performs a metadata image backup of a hierarchical storage management (HSM) file system. Concurrent with performing the metadata image backup of the HSM file system, the mechanism identifies a set of resident files that are not yet stored off-line in the HSM file system. The mechanism identifies immutable files within the set of resident files to form an optimized set of resident files and passes the optimized set of resident files to a traditional backup system. The traditional backup system backs up file data for the set of resident files. Responsive to detecting an inability to restore a migrated managed state of a file, the mechanism unsets an immutable bit of the file attribute, restores file data for the file using the traditional backup system, and sets the immutable bit attribute of the file.

Abstract: Improving file system reliability in storage mediums after a data corrupting event using file system journaling is described. In one embodiment, a method, which includes scanning beyond an active transactions region within the file system journal to locate additional valid transactions for replay to bring the storage medium into a consistent state; the scanning performed until an invalid transaction is reached.

Abstract: Migrating data from an old storage device to a new storage device includes transferring the data from a first logical device on the old storage device to a second logical device on the new storage device, determining replication relationships for the first logical device, transferring the replication relationships; and transferring logical devices used for the replication relationships from the old storage device to the new storage device. The replication relationships may include a clone of the first logical device. Migrating data from an old storage device to a new storage device may also include suspending writes to the first logical storage device prior to transferring the data for the first logical storage device. The replication relationships may include at least one snapshot.

Abstract: A federated configuration management database includes a plurality of management data repositories, and a backup destination determination unit that determines backup destination management data repositories for backing up data overlapping among the plurality of management data repositories at specific management data repositories and that notifies each of the management data repositories of the data to be backed up.

Abstract: Decreasing disk costs make it possible to take frequent snapshots of past storage system states and retain them for a long duration. Existing snapshot approaches offer no satisfactory solution to long-lived snapshots. Split snapshots are an approach that is promising because it does not disrupt the current state storage system in either the short or the long run. An unsolved problem has been how to maintain an efficient access method for long-lived split snapshots without imposing undesirable overhead on the storage system. Skippy is a new approach that inexpensively indexes long-lived snapshots in parallel with snapshot creation. An embodiment of Skippy uses append-only index data structures to optimize writes while simultaneously providing low-latency snapshot lookup. Performance evaluations of Skippy indicate that this new approach is effective and efficient. It provides close-to-optimal access to long-lived snapshots while incurring a minimal impact on the current-state storage system.

Abstract: A method, system, and computer usable program product for deploying directory instances are provided in the illustrative embodiments. A configuration of an existing directory instance is cloned to the new directory instance. The existing directory instance may execute in a first data processing system and the new directory instance may execute in a second data processing system. A schema of the existing directory instance is cloned to the new directory instance. A determination is made whether the new directory instance is a peer of the existing directory instance. Data from the existing directory instance is cloned to the new directory instance if the new directory instance is a peer of the existing directory instance. The new directory instance is made operational in a directory topology.

Abstract: Systems and methods for performing hierarchical storage operations on electronic data in a computer network are provided. In one embodiment, the present invention may store electronic data from a network device to a network attached storage (NAS) device pursuant to certain storage criteria. The data stored on the NAS may be migrated to a secondary storage and a stub file having a pointer pointing to the secondary storage may be put at the location the data was previously stored on the NAS. The stub file may redirect the network device to the secondary storage if a read request for the data is received from the network device.

Abstract: A method for representing records includes, assigning a unique identifier to a record stored at a record collection site, entering the unique identifier in a hierarchical tree structure, and sending the hierarchical tree structure to a central storage site.

Abstract: A method is disclosed for reliably updating a data group in a read-before-write data replication environment. The method reliably updates the data group by receiving an updated data group sent from a first storage medium to a second storage medium, comparing the updated data group with a previous data group previously existing on the second storage medium and writing the updated data group to the second storage medium. The read-before-write and differencing method disclosed maintain reliability by storing multiple copies of changes made to the second storage medium during and after the write process.