Abstract: A transaction request compliant with a first version of a journal schema of a multi-data-store storage system is received at a journal manager. The journal schema indicates attributes of data objects which may be materialized at various data stores of the system. The journal manager stores an entry in the system's journal if the transaction meets acceptance criteria. Writes indicated in the entry are materialized at the data stores after verifying that the entry is compliant with the journal schema. After verifying that member data stores have approved a proposed change to the journal schema, another entry indicating a different version of the journal schema is added to the journal. Client-side components of the system obtain the current version of the journal schema to prepare the transaction requests.

Abstract: A system and method for a database management system. The method comprises: receiving a database transaction for inserting a set of records in a database table of the database management system or for creating indexes. The database transaction may be executed using the database management system. The database management system may rollback the database transaction by truncating the database table or the created indexes.

Abstract: One example method includes performing replication operations and/or failure recovery operations for replication operations in a computing system using matrix clocks and replication controllers. The replication controller includes or has access to vectors of a matrix clock. The replication controller also includes a multiplexer. The multiplexer is configured as an m×n multiplexer using the matrix clock to determine m and n. The operation is performed using the configured multiplexer. This allows an imbalance of resources to be managed by the replication controller. The matrix clock can be updated, which may result in the multiplexer being reconfigured.

Abstract: One example method includes performing replication operations and/or failure recovery operations for replication operations in a computing system using matrix clocks. Each node or process in a computing system is associated with a matrix clock that reflects streams required and/or available at the source and destination nodes. As the replication operation is performed, the matrix clocks are updated. When a failure is detected or when insufficient streams are available, the nodes may be throttled until sufficient streams are available. The matrix clocks facilitate a recovery line such that replication can resume when sufficient streams are available.

Abstract: A journaled database system may comprise data nodes that maintain a collection of data structured in accordance with a schema. A change to the schema may be applied by a journal module while a journal consumer continues to operate using a prior version of the schema. A buffer may be formed and have stored therein records describing state change instructions according to the prior view of the schema. An index of correspondence between the records in the buffer and in the source journal may be maintained.

Abstract: Systems and methods are provided for efficiently maintaining a transaction sequence witness service to ensure that requests to read data provide consistent results across all storage nodes in a distributed system. Each storage node that stores data in response to a particular update event may first update the transaction sequence witness service with the sequence number for the update event. Thus, any other storage node that services a subsequent request for the data will be able to see, via the transaction sequence witness service, the sequence number that must be available on the storage node to respond to the request.

Abstract: Asynchronous snapshot invalidation techniques are described. According to various such techniques, an enhanced file handle structure may be defined that includes a snapshot generation ID that is to comprise a value that singularly identifies a snapshot performed at a particular point in time. In some embodiments, when a snapshot ID assigned to that snapshot is reused at a subsequent point in time, a different snapshot generation ID may be assigned to that subsequent snapshot. With respect to an in-core cache, the differing snapshot generation IDs may eliminate unacceptable ambiguity regarding respective file information sets corresponding to the initial and subsequent snapshots sharing the same snapshot ID. As a result, obsolete file information sets may be cleared from the in-core cache asynchronously, enabling improved performance. The embodiments are not limited in this context.

Abstract: A scalable multi-cluster resource sharing facility. A global witness process runs on a first computing platform that communicates over one or more networks to any number of nodes situated over two or more clusters. The global witness process listens on the network for occurrences of leadership and/or resource requests from nodes of different clusters. The global witness processes a request by retrieving a resource request and a respective last known state value, comparing the last known state value to a global stored state value, then storing a new state value when the respective last known state value is equal to the stored state value. Any number of contemporaneous requests can be processed by the global witness process, however only one request can be granted. The other requestors each receive a rejection of their resource request when their proffered last known state value is not equal to the stored state value.

Abstract: Some embodiments provide a method for operating a physical server in a network. The method stores multiple copies of a virtual machine (VM) image at a network-accessible storage. The method uses a first copy of the VM image as a virtual disk to execute a VM on a hypervisor of a first physical computing device. The method uses a second copy of the VM image as a virtual disk accessible via a smart network interface controller (NIC) of a second physical computing device to execute an operating system of the second physical computing device.

Abstract: A method and system for co-locating journaling and data storage based on write requests. A write request that includes metadata and data is received from a client. A logical storage unit for storing the metadata and the data is identified. The logical storage unit is divided into a journal partition and a volume partition. The journal partition includes a first log and a second log. Which of the first log and the second log is an active log and which of the first log and the second log is an inactive log are identified. The metadata is recorded in a first location in the active log and the data is recorded in a second location in the active log during a single I/O operation. A reply is sent to the client after the metadata and the data are recorded in the journal partition.

Abstract: A method for managing metadata for a transactional storage system include receiving a query request at a snapshot timestamp. The query request requests return of at least one data block from a plurality of data blocks. Each data block includes a corresponding write epoch timestamp and a corresponding conversion indicator indicating whether the data block is active or has been converted at a respective conversion timestamp. The method also includes setting a read epoch timestamp equal to the earliest one of the write epoch and determining whether any of the respective conversion timestamps occurring at or before the snapshot timestamp occur after the read epoch timestamp. The method also includes determining the at least one data block requested by the query request by scanning each of the data blocks including corresponding write epoch timestamps occurring at or after the read epoch timestamp.

Abstract: Examples of the present disclosure describe implementing bitmap-based data replication when a primary form of data replication between a source device and a target device cannot be used. According to one example, a temporal identifier may be received from the target device. If the source device determines that the primary replication method is unable to be used to replicate data associated with the temporal identifier, a secondary replication method may be initiated. The secondary replication method may utilize a recovery bitmap identifying data blocks that have changed on the source device since a previous event.

Abstract: The present teaching relates to a method, system and programming for recovering a data storage. In response to a failure of the data storage, a last checkpointed version is retrieved from a checkpoint file created previously for the failure of the data storage. A recovery table is updated to indicate the last checkpointed version and an incarnation number indicating a number of previous recovery of the data storage. The data storage is recovered based on the recovery table. A version number of the last checkpointed version is reset to zero upon a successful recovery of the data storage. The incarnation number is incremented.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable enabling online access to encounter data across multiple disparate sources. The method includes providing a synchronizer application configured to access data of a practice group server, receiving, by an aggregator server, a start time and a duration of a healthcare appointment, and determining a completion time of the healthcare appointment. Based on the completion time, the aggregator server transmits a request for encounter data for the healthcare appointment to the synchronizer application and then receives and stores the encounter data in an aggregator database. The aggregator server generates an electronic alert addressed to a computing device accessible by the patient and transmits the alert to the computing device. The aggregator server receives an electronic response to the alert and provides an interactive user interface to the computing device configured to display the encounter data on the computing device.

Abstract: A method for constructing a persistent memory index in a non-uniform memory access architecture includes: maintaining partial persistent views in a persistent memory and maintaining a global volatile view in a DRAM; an underlying persistent memory index processing a request in a foreground thread when cold data is accessed; when hot data is accessed, reading a key-value pair for a piece of hot data in the global volatile view in response to a query operation carried in the request, and in response to an insert/update/delete operation carried in the request, updating a local partial persistent view and the global volatile view; and in response to a hotspot migration, a background thread generating new partial persistent views and a new global volatile view, and recycling the partial persistent views and the global volatile view for old hot data into the underlying persistent memory index.

Abstract: Embodiments of the present invention include generating, at a first time, a first copy of a database including multiple tables, the first copy including a set of first structured text files. During a time period subsequent to the first time, queries that request updates to the tables during the time period are periodically generated and conveyed to a system managing the database. Responsively to the periodic queries, any updates to the tables and their respective update times are received and stored to a journal including one or more second structured text files. At a subsequent time, a request to generate a second copy of the database is received, the second copy including a state of the database at a specified time during the time period. The first copy of the database is updated with the updates whose update times are prior to the specified time, thereby generating the second copy.

Abstract: Object service receives communication of fingerprints stream, corresponding to file segments, from file source, and identifies sequential fingerprints in fingerprints stream as fingerprints group. Object service identifies group identifier for fingerprints group, and communicates fingerprints group to deduplication service associated with group identifier range including group identifier. Deduplication service identifies fingerprints in fingerprints group which are missing from fingerprint storage, and communicates identified fingerprints to object service, which communicates request for file segments, corresponding to identified fingerprints, to file source. Deduplication service receives communication of requested segments from file source, and stores requested segments.

Abstract: Examples of the present disclosure describe implementing bitmap-based data replication when a primary form of data replication between a source device and a target device cannot be used. According to one example, a temporal identifier may be received from the target device. If the source device determines that the primary replication method is unable to be used to replicate data associated with the temporal identifier, a secondary replication method may be initiated. The secondary replication method may utilize a recovery bitmap identifying data blocks that have changed on the source device since a previous event.

Abstract: Example embodiments relate generally to systems and methods for continuous data protection (CDP) and more specifically to an input and output (I/O) filtering framework and log management system to seek a near-zero recovery point objective (RPO).

Abstract: Methods and systems for restoring a tenant database of a multitenant database system with multiple tenant databases is provided. In response to a request to restore a first tenant database, a system verifies that the first tenant database is present in the multitenant system database system with a same identifier when a backup of the first tenant database was taken during a backup operation and when the restore request was received. A restore method is selected based on a storage type used for storing the first tenant database. The selected restore method is executed for restoring the first tenant database utilizing an internal file associated with the first tenant database, while processing requests for other tenant databases.

Abstract: A data recovery method recovers corrupted data after a disaster event in a storage facility operating with deduplication in which copies of data are maintained between volumes through a copy services relationship. The deduplication operates with referencing domains, each having a source grain containing a reference pointing to a location where data is stored and referrers pointing to the source. The data recovery method identifies any source that is pointing to data which is corrupt and then establishes whether a copy services relationship exists between the referencing domain of the source with corrupted data and another referencing domain, in which case the other referencing domain will have a copy of the corrupted data. Provided that the copy is valid, it is written across to replace the corrupted data. The method allows corrupted source data to be recovered without having to perform a full resync or restore.

Abstract: An apparatus comprises a processing device configured to receive, at a given data management entity running on a given processing node, a request to create a given cluster of data management entities for a given client. The processing device is also configured to determine membership requirements for the given cluster, to discover additional data management entities running on additional processing nodes, and to select at least one of the additional data management entities for membership in the given cluster based at least in part on the membership requirements. The processing device is further configured to establish a replication relationship for automating sharing of metadata in the given cluster, the metadata comprising access information and location information for data stores where portions of data items of the given client are stored. The processing device is further configured to perform data management functions for the given client utilizing the metadata.

Abstract: Systems and methods for live reindexing include receiving, at a dispatch server, a request to reindex a database, wherein, at the time the request to reindex is received, the dispatch server is configured to send queries for information in the database to a first index server, the first index server comprising a first index of information in the database. The systems and methods further include, responsive to receiving the request to reindex and while the database is live, generating a second index of the information in the database, storing the second index at a second index server, and configuring the dispatch server to use the second index server as a live index server and to no longer use the first index server as a live index server.

Abstract: A method, system and computer program product for improving the availability of API endpoints in container orchestration platforms. A service request handler module of a microservice application programming interface (API) fault manager (MAFM) invokes a microservice API fault management service in response to detecting an unresponsive microservice managed by a container orchestration platform. An API inspection module of the MAFM identifies an unresponsive API of the unresponsive microservice. A remedial action module of the MAFM determines a remedial action to correct an unresponsive state of the unresponsive API. A container platform interface module of the MAFM communicates the remedial action to the container orchestration platform.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for analyzing change stream data. The method includes identifying, by one or more computer processors, a list of changes in a replication stream. The method further includes determining, by one or more computer processors, if one or more changes in the list of changes matches a criteria for a hint. The method further includes in response to determining that one or more of the changes in the list of changes matches the criteria for a hint, inserting, by one or more computer processors, the hint into the list of changes prior to a first change of the one or more changes that triggers a match in criteria.

Abstract: Examples described herein include systems and methods for backing up and recovering a software-defined data center (“SDDC”). In one example, entities of the SDDC, such as virtual machines, hosts, and clusters, can coexist with corresponding entity stores. The entity stores can store current state information for each SDDC entity. For example, an identifier or name of a virtual machine can be stored in that virtual machine's corresponding entity store. When recovery of a controller is needed, the controller can rebuild state information that has changed after the controller was backed up, by retrieving state information from entity stores of the various SDDC entities.

Abstract: Described is a system that securely recovers critical backup and data protection infrastructure such as a backup application that manages data for a backup and recovery system. Such a capability may be provided by securely storing a copy of metadata associated with a production backup application into an isolated recovery environment. By storing the metadata on the isolated recovery environment, the metadata may be secure from potential security threats such as ransomware that may attack the production infrastructure itself. Accordingly, the secure copy of metadata may then be restored to a sandboxed environment with the isolated recovery environment and used to create a recovery instance of the backup application. The system may also perform various tests on the recovered instance of the backup application in response to particular security threats before being exposed to a production system for recovery purposes.

Abstract: Data processing methods, data processing apparatuses, and storage media are provided. The method is applicable to a data processing system. The data processing system includes a storage device and a programmable device. Data is transmitted between the storage device and the programmable device via a bus. A controller and an accelerator are deployed on the programmable device. The controller is enabled with at least two kinds of data format conversion function. The method includes: the controller obtaining the first data; the controller performing data format conversion on the first data to obtain second target data in a target data format; and the controller storing the second data to the storage device and/or sending the second data to the accelerator.

Abstract: Techniques to back up data associated with a cluster environment are disclosed. In various embodiments, an indication is received to back up data associated with the cluster. A backup configuration data associated with the cluster is used to back up, in a unified backup operation, one or more save sets associated with virtual resources associated with the cluster and one or more saves sets associated with physical nodes associated with the cluster, including by storing each respective save set in a manner that associates the save set with a virtual or physical node comprising the cluster suite.

Abstract: A plurality of scanned backup snapshots are generated. A backup snapshot among a plurality of backup snapshots is selected. At least a portion of the selected backup snapshot is restored in a temporary environment to create a restored instance of at least the portion of the selected backup snapshot. A vulnerability scan of the restored instance of at least the portion of the selected backup snapshot is performed. One or more vulnerabilities of the scanned portion of the selected backup snapshot are tracked. A request associated with identifying a scanned backup snapshot to restore from the plurality of scanned backup snapshots is received. In response to the request, at least a predetermined identification of the one or more vulnerabilities of the selected backup snapshot is provided.

Abstract: A technique includes storing, by a computer, a composite metadata object in a catalog. The composite metadata object represents information about a plurality of database objects of a table structure; the table structure includes a plurality of rows and a plurality of columns; the plurality of database objects is associated with a subset of rows of the plurality of rows; and the plurality of database objects is associated with a plurality of files corresponding to the columns. The technique includes processing, by the computer, a database transaction based on the based on composite metadata object.

Abstract: Examples of the present disclosure describe implementing bitmap-based data replication when a primary form of data replication between a source device and a target device cannot be used. According to one example, a temporal identifier may be received from the target device. If the source device determines that the primary replication method is unable to be used to replicate data associated with the temporal identifier, a secondary replication method may be initiated. The secondary replication method may utilize a recovery bitmap identifying data blocks that have changed on the source device since a previous event.

Abstract: Example embodiments relate generally to systems and methods for continuous data protection (CDP) and more specifically to an input and output (I/O) filtering framework and log management system to seek a near-zero recovery point objective (RPO).

Abstract: A method for managing metadata for a transactional storage system include receiving a query request at a snapshot timestamp. The query request requests return of at least one data block from a plurality of data blocks. Each data block includes a corresponding write epoch timestamp and a corresponding conversion indicator indicating whether the data block is active or has been converted at a respective conversion timestamp. The method also includes setting a read epoch timestamp equal to the earliest one of the write epoch and determining whether any of the respective conversion timestamps occurring at or before the snapshot timestamp occur after the read epoch timestamp. The method also includes determining the at least one data block requested by the query request by scanning each of the data blocks including corresponding write epoch timestamps occurring at or after the read epoch timestamp.

Abstract: Embodiments of the present disclosure provide a method, a device and a computer program for data backup. A method of backing up data comprises: in response to receiving, from an application system, a request for backing up first data, storing the first data into a first backup node; generating first metadata corresponding to the first data, the first metadata comprising first digest information of the first data; storing the first metadata into a block chain system to which the first backup node belongs; and verifying integrity of the first data stored in the first backup node with the first metadata stored in the block chain system. By utilizing a block chain system, the embodiments of the present disclosure ensure that data backed up are protected from being tampered with.

Abstract: Changes made to a database table are accumulated, in durable storage, and snapshots of partitions of the table are obtained. For successive snapshots of a partition, the system accesses a previous snapshot, applies changes from the accumulated changes, and stores the updated snapshot to a durable data store. The accumulated changes and the successive partition snapshots are made available to restore the database to any point in time across a continuum between successive snapshots. Although each partition of the table may have a backup snapshot that was generated at a time different from when other partition snapshots were generated, changes from respective change logs may be selectively log-applied to distinct partitions of a table to generate an on-demand backup of the entire table at common point-in-time across partitions. Point-in-time restores of a table may rely upon a similar process to coalesce partition snapshots that are not aligned in time.

Abstract: A database may store a plurality of database revisions corresponding to a time period, including a current version and one or more backup versions of a data object. After a threshold time has passed, the database may identify a plurality of time intervals within the time period and identify, for each of the plurality of time intervals, at most one database revision corresponding to the time interval. The database may delete, for each of the plurality of time intervals, all of the database revisions except for the identified at most one database revision. The database may periodically repeat this process to reduce the granularity of backup data over time.

Abstract: A method may include iterating through at least a portion of a table stored in a database by identifying a timestamp block associated with a range of rows forming the portion of the table. In response to determining that the timestamp block is a sparse timestamp block storing timestamps associated with only some of the rows in the first range of rows, generating a bitmap. Each of the binary values in the bitmap may correspond one of the rows in the range of rows. Moreover, each of the binary values in the bitmap may indicate whether the timestamp block includes a timestamp associated with a corresponding row from the range of rows. Iterating through the range of rows may include accessing, based on the bitmap, the timestamp block to read or write timestamps of transaction affecting the rows in the range of rows.

Abstract: A technique for replicating file systems to any number of targets includes obtaining a writeable snapshot of a file system on a source, performing cloud tiering of files in the snapshot, replacing the file contents of the snapshot with stubs pointing to the cloud, and generating a namespace backup of the snapshot that includes directories and stubs. The source system ships the namespace backup to the cloud, allowing any authorized target system to restore from the namespace backup to a local file-system replica, such that contents of the cloud-tiered files are made accessible on the target system from the restored namespace via the stubs.

Abstract: Example implementations relate to memory mode categorization. An example boot process modification can include invoking a first service of a client operating system (OS) on top of an enabled host OS, modifying a boot process of the host OS using the first service, and executing applications within the client OS based on the modified boot process of the host OS.

Abstract: Restoring a database image of a secondary database system comprises initializing the database image from a recovery image that includes metadata logs and non-metadata logs, of which at least one metadata log includes an undo log of non-committed data that is applied to the initialized database image. A first pagelist and first indexes are then rebuilt based on first data segments of the database image and the corresponding metadata objects. The database image is then updated by replaying the metadata redo log generated from the undo log, the metadata logs, and the non-metadata logs. Then, a second pagelist and second indexes for the database system are rebuilt based on second data segments of the database image and the corresponding non-metadata objects, while each of the first data segments is statically separated from the second data segments in physical memory space. Related apparatus, systems, techniques and articles are also described.

Abstract: A system according to certain aspects may include a secondary storage controller computer configured to: in response to a first instruction to obtain a first secondary copy of a first data set from a secondary storage device(s), the first instruction associated with a first restore operation: instantiate a first restore thread on a processor of the secondary storage controller computer; using the first restore thread, retrieve the first secondary copy from the secondary storage device(s); and forward the retrieved first secondary copy to a primary storage subsystem for storage; and in response to a second instruction to obtain a second secondary copy of a second data set from the secondary storage device(s), the second instruction associated with a second restore operation: using the first restore thread, retrieve the second secondary copy from the secondary storage device(s); and forward the retrieved second secondary copy to the primary storage subsystem for storage.

Abstract: Techniques for metadata-driven dynamic content serving. Metadata content is stored as a source instance. The metadata content is utilized to provide dynamically-constructed pages. The metadata content is published to runtime pods communicatively coupled to receive the metadata content. The source instance includes a metadata definition repository and is a primary source of dynamic data for serving pages in the runtime pods. The runtime pods are groups of multiple servers that act as a single entity to dynamically generate metadata-driven content in response to requests received from client devices. A request for content is received with a selected one of the runtime pods for a specific site. The specific site is mapped to a user identified by a user identifier. The user identifier is utilized to retrieve site metadata from a site metadata server. Content is provided in response to the request with the selected runtime pod utilizing the metadata content.

Abstract: Described is a system for secure management of recovery data for data protection assets such as backup data and a backup application of a production backup system. The system may provide the ability to synchronize and secure critical recovery data of an isolated recovery environment. Accordingly, the system may reduce the breadth of potential cyber security attack vectors and increase the likelihood of efficiently recovering critical data and/or applications. To provide such capabilities, the system may only activate a data connection between a production system and a recovery system when synchronizing recovery data. In addition, the system may apply a retention lock to maintain a set of immutable copies of the recovery data and may restore the recovery data to a sandboxed environment where it may be tested and verified before being deployed to the production system as part of a recovery process.

Abstract: Recovery points can be used for replicating a virtual machine and reverting the virtual machine to a different state. A filter driver can monitor and capture input/output commands between a virtual machine and a virtual machine disk. The captured input/output commands can be used to create a recovery point. The recovery point can be associated with a bitmap that may be used to identify data blocks that have been modified between two versions of the virtual machine. Using this bitmap, a virtual machine may be reverted or restored to a different state by replacing modified data blocks and without replacing the entire virtual machine disk.

Abstract: A system is configured to allocate storage space on existing devices within the entity's networked system to create cloud storage space. In particular, unallocated space on computing devices, typically user devices, within an entity's network is utilized as a cloud data repository. Cloud data is indexed, divided into chunks, encrypted, and stored on numerous disparate endpoint devices connected to the network. Copies of cloud chunk data may be duplicated across multiple endpoint devices to allow for data redundancy, thereby ensuring cloud data uptime according to the availability needs of the entity. Cloud data may further be allocated to different devices based on regional data restrictions. In this way, the system provides an efficient and secure way to generate an internal cloud data storage repository within an entity's networked system.

Abstract: Methods, apparatus, systems and articles of manufacture to perform backup and recovery of data files using hard links are disclosed. Example methods disclosed herein include determining whether a first operation to be performed on a target file by a thread of a process is a file deletion operation, wherein the target file is referenced by a first hard link. Disclosed example methods also include creating, in response to the first operation being a file deletion operation, a second hard link to the target file, where the creation of the target hard link occurs while the thread of the process performing the deletion is blocked. Disclosed example methods further include returning control to the thread of the process performing the deletion after the second hard link is created to enable the operation to be performed on the target file.

Abstract: A computer-implemented method, computer-implemented apparatus, and non-transitory, computer-readable medium for restoring state data of a blockchain. Account state data in the blockchain is organized into a Merkle state tree and stored in a database. The Merkle state tree includes a current Merkle state tree obtained by organizing a latest account state of each blockchain account and a historical Merkle state tree obtained by organizing a historical account state of each blockchain account. A target block to be restored is determined, where the target block is any historical block preceding a latest block of the blockchain. State restoration logic is iteratively performed until a latest account state corresponding to each blockchain account on a current Merkle state tree of the latest block is restored to a historical account state corresponding to each blockchain account on a historical Merkle state tree of the target block.

Abstract: Embodiments described herein relate to testing the integrity of a storage system's metadata while corresponding structured data remains online. Embodiments also relate to enabling corrupt storage system metadata to be repaired while the metadata remains in use and while its structured data remains online. Corruption detection and repair is described with respect to allocation metadata and reference count metadata. The embodiments are applicable to many types of storage systems, including file systems and databases, for example.

Abstract: An information processing device includes a statistical value holding unit configured to hold a statistical value of data included in each of two or more blocks into which a data set is divided; a query history holding unit configured to hold information about a past query on the data set as a query history; an estimation unit configured to, based on the query history, estimate a block size that minimizes an average cost of deriving an answer to a query by using the statistical value of at least one of the blocks; and a block creation unit configured to, based on the block size estimated by the estimation unit, create two or more blocks by dividing the data set, calculate the statistical value for each of the created blocks and cause the statistical value holding unit to hold the calculated statistical values.