Abstract: In a storage area network, agents provide transactions of data sets containing updates to manageable entities in a SAN. A SAN server avoids intermittent dissemination of events pertaining to a single transaction by storing events in a cache until completion of all subtransactions in the transaction, and then disseminating all events pertaining to the transaction. Transactions are broken down into update sections, each containing one or more subtransactions collectively defining the entire transaction. Update sections corresponding to the transaction are tagged with a section ID. Events indicating completion of processing of the update section are identified by the section ID, and suppressed temporarily by storing them in the event cache pending completion of the transaction. Other events not matching a tagged section ID are allowed to disseminate. Upon completion of the transaction, events for all section IDs corresponding to the transaction are disseminated from the cache.

Abstract: The described implementations relate to synchronizing records between databases. One implementation can cause historical data of a source database to be recorded as entries on a transaction log and can subsequently cause new data of the source database to be recorded on the transaction log in a same manner as the historical data. This implementation can also identify a distinct attribute associated with an individual entry and generate a message that reflects the individual entry.

Abstract: A system for processing a transaction is disclosed. The system comprises a processor and a memory. The processor is configured to check a condition using data in a first database, wherein the data is associated with a transaction, wherein the data in the first database is latched before checking the condition and is unlatched after checking the condition. The processor is further configured to indicate to a second database to check the condition using data in the second database, wherein the data is associated with the transaction. The data in the second database is latched before checking the condition and is unlatched after checking the condition. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: An application workload capture and replay system with a transactionally consistent application workload replay feature is provided. More particularly, the feature includes capture-phase components for capturing and recording a real application workload submitted to a production web application system and includes replay-phase components for replaying the captured application workload against a test web application system in a transactionally consistent manner. The feature provides guarantees about the order of database transactions that are caused when the workload is replayed such that there is a consistency between the replay-phase order of the database transactions and the order of those transactions that occurred when the workload was captured. These consistency guarantees facilitate a faithful reproduction of database changes observed in the production web application system in the test web application system using a captured real application workload.

Abstract: A method is provided for replicating transaction data from a source database to a target database wherein the transaction data is communicated from a change queue associated with the source database to the target database. An initial path is provided between the change queue and the target database for transaction data to flow. The initial path has a maximum transaction load capacity. It is then detected whether the current transaction load is close or equal to the maximum transaction load capacity of the initial path. If so, another path is provided between the change queue and the target database.

Abstract: A method for replicating a program and data storage according to one embodiment comprises sending program replication data from a first program to a second program, the second program having an application program that is a replica of an application program of the first program; sending data storage requests from the first program to a first storage system; and replicating data stored in the first storage system in a second storage system. Additional methods, systems, and computer program products are disclosed.

Abstract: A method of integrating large volumes of updates in a database system. Each individual update includes of a coherent set of data records supplied by a data provider. The database system includes a master file repository and an active image repository. The method includes first obtaining for each individual update a unique modification identifier from a logistic table of the database system. The master file repository is then updated with the coherent set of data records of each uniquely identified individual update. A unique commit identifier is further obtained from the logistic table for each individual update that has been committed by the master file. The active image repository is synchronized by successively loading into the active image each individual update. Completion of the propagation of each individual update into the active image repository is achieved when the loading of each corresponding set of data records is committed by the active image.

Abstract: Methods, systems and apparatus, including computer program products, for propagating data across shards of a database system are described. According to one method, a first request that includes instructions to place a second request in a queue of a second shard is placed in a queue of a first database shard, and the change is committed to the records of the first shard. A queue processor polling the queue fetches and dispatches the first request, placing the second request in a queue of a second database shard. The second request includes instructions for making the change to the second shard. A queue processor of the second database shard polls the queue of the second database, fetches the second request, and dispatches it to make the change on the second database shard.

Abstract: Techniques are described for providing a storage service that stores information about large numbers of transactions in a persistent manner, such as with a high degree of reliability, availability and scalability based at least in part on use of a distributed computing and storage system. In some situations, the transaction information storage service stores various information about transactions that each include at least one monetary payment (e.g., a micro-payment) between financial accounts of two or more of numerous users having accounts with one or more entities. The transaction information storage service may be provided by or otherwise affiliated with a merchant, and customers of the merchant may purchase usage of the storage service for programs executed by or otherwise affiliated with the customers, with the storage service available to remote executing programs via a defined API of the storage service, such as a Web services-based API.

Abstract: A method is provided for replicating transaction data from a source database to a target database wherein the transaction data is communicated from a change queue associated with the source database to the target database. An initial path is provided between the change queue and the target database for transaction data to flow. The initial path has a maximum transaction load capacity. It is then detected whether the current transaction load is close or equal to the maximum transaction load capacity of the initial path. If so, another path is provided between the change queue and the target database.

Abstract: Methods and systems are disclosed that relate to single instance storage of asset parts utilizing a back reference count associated with each asset part. Entries in a journal associated with the asset parts reflect the intention to increment or decrement an asset part's back reference count. The journal may be used to maintain an accurate and reliable reference count for asset parts replicated across multiple disk drives in a distributed storage system, where the replication is structured to reduce the risk of data loss in the event of a disk drive failure. The journal entries may comprise generation numbers used to maintain a back reference count associated with an asset part and to verify a back reference count associated with an asset part.

Abstract: A method and a device transport a postal object to an incompletely specified destination address. The object has at least one information item relating to a destination address to which the object is to be transported. A set of address components is predetermined. An address database is used which in each case contains a computer-available record per destination address for a set of possible destination addresses. Each record contains in each case one entry for each predetermined address component. At least one destination address information item on the object is detected. A screen form is used which contains in each case one input field for each address component. The detected destination address information is compared with the records of the address database. When exactly one record is consistent with all detected destination address information items, a transportation of the object to the destination address of the record is triggered.

Abstract: A method of determining whether a data segment is a duplicate using cooperating deduplicators is disclosed. The data segment is received. A first deduplicator is operated to to determine whether the incoming data segment is a duplicate based on first information available to the first deduplicator regarding stored data segments that are stored in a memory. A second deduplicator is selectively operated to determine whether the incoming data segment is a duplicate based on second information available to the second deduplicator; wherein the selective operation of the second deduplicator depends on the determination made by the first deduplicator.

Abstract: A system providing improved methods for data replication is described. A method for replicating a transaction from a primary database to a replicate database while the replicate database remains available for use comprises: recording information about a transaction being performed at a primary database in a transaction log; synchronously copying the information about the transaction in the transaction log to a mirrored transaction log; generating a reconstructed transaction based on the information about the transaction copied to the mirrored transaction log; and applying the reconstructed transaction at the replicate database while the replicate database remains available for use.

Abstract: A method is provided for replicating transaction data from a source database to a target database wherein the transaction data is communicated from a change queue associated with the source database to the target database. An initial path is provided between the change queue and the target database for transaction data to flow. The initial path has a maximum transaction load capacity. It is then detected whether the current transaction load is close or equal to the maximum transaction load capacity of the initial path. If so, another path is provided between the change queue and the target database. Also, a method is provided of replicating transaction data from a source database to a target database wherein an associated with the target database has a maximum transaction threshold limit. The applier normally posts transaction data to the target database only upon receipt of a commit step or operation associated with respective transaction data.

Abstract: Exemplary embodiments of the present invention relate to methodologies, computer program products and systems for the support of content replication, wherein transaction-level data replication consistency is guaranteed. Further, the exemplary methodologies of the present invention can be implemented while incurring little or no additional I/O overhead.

Abstract: Architecture for replicating a database through log shipping. A replication service replicates data using one or more new internal application program interfaces (APIs), a replay configuration API which abstracts configuration information for each destination (or target) instance. APIs for log file management as part of the replication process are provided for opening communications for log shipping, tracking success of log shipping and closing out log shipping communications. Log shipping APIs are also provided for log truncation control. A ship control object is provided as a base class which includes an API for other components of the replication service to operate on log files when the log files are placed into a log directory. The ship control API is utilized by the replication service and for interaction with storage system log files.

Abstract: A method and apparatus for efficiently logging changes to a file system are disclosed. A file system filter driver intercepts I/O requests directed to a file system, and analyzes the I/O requests to determine whether the requests are write or modify requests directed to a file included in a list of files to be monitored for changes. If the I/O request is directed to such a file, the file system filter driver logs the I/O request to a b-tree structured change log file. Furthermore, in one embodiment, the file system filter driver is configured to log file system changes to multiple log files to coincide with snapshot operations, such that the file system changes to be included in each snapshot are included in separate log files.

Abstract: A method of and system for collecting, storing, and retrieving data in a data base management system. The database management system includes a master database server (4), at least one workgroup server (315), and a plurality of workgroup user clients (310). The workgroup server (315) is interposed between the master database server (4) and said workgroup user clients (310). The method creating a transaction in a local database resident on one of the workgroup user clients (310), entering the transaction into a transaction log resident on the workgroup user client (310), and creating a transaction file corresponding to the transaction in an outbox of said workgroup user client (310). Next, the transaction file is copied to an inbox identified to the workgroup user client (310) and updating the transaction file into a workgroup database (305) resident on the workgroup server (315). The workgroup database (305) includes a transaction log.

Abstract: A system, backup components and computer program product are provided to improve the efficiency of the transmission of consistency groups across multiple storage components by ensuring that volume transactions are evenly allocated among backup components. Each volume is initially assigned to a specified backup component. Once a color period begins, volume transactions are transmitted to the assigned backup component. Each backup component accumulates data transfer information for its assigned volumes throughout at least a portion of the color period and calculates a new volume assignment for the next color period. Before the start of the next color period, a captain storage controller transmits any new assignments to the backup components. During a next color period, the process is repeated using the new assignments.

Abstract: Systems and methods for partitioning information across multiple storage devices in a web server environment. The system comprises a web server database which includes information related creating a web site. The information is divided into partitions within the database. One of the partitions includes user information and another of the partitions includes content for the web site. Portions of the content for the web site is replicated and maintained within the partition including the user information. Further, a portion of the user information is replicated and maintained in the partition where the content for the web site is maintained. The methods include dividing information into partitions, de-normalizing the received data and replicating the data portions into the various web site locations.

Abstract: In one aspect, a method for operating databases with a plurality of first data is provided. A temporary copy of at least some logically related parts of the plurality of first data is generated as second data for an editing procedure. The second data is offered by a user such that he can edit the second data. On conclusion of the editing procedure an individual comparison of the second data with the first data is performed such that when a second datum of the second data matches a first datum of the first data, a reference is created in the database to the first datum corresponding to the second datum. At least some of the second data for which no match with first data is determined is saved in the database.

Abstract: To ensure data consistency for a new data replica created for a computing system, the transmission and receipt of messages altering the data are monitored and replication of the data is permitted when all changes to the data that have been transmitted to the data have been received.

Abstract: A sequence of processing transactions operating on a cached data set, which depend on each other because they operate on the same items of data from the data set. The transactions are saved until the master copy of the data becomes available. The transactions are played back against the master copy until one of the played back transactions fails because another transaction which operated on the same items of data but against a separate cached copy and originating from another application, was successfully played back on the master copy at an earlier time. At this point, all subsequent transactions which depended on the failing transaction are rolled back on the cached copy (with appropriate failure reports generated) and activity resumes from there. “Chained” (optimistic) transactions can therefore be applied to cached data and can be allowed to progress by recording their dependencies so they can be undone when required.

Abstract: An exemplary system enabling credential roaming among a plurality of different computing devices may include an event handler to receive event notifications such as, e.g., a client logon. The event handler may invoke a management service in response to receiving an event notification. The management service may include a synchronizing module to synchronize a user's credentials with a remote directory service, such as, e.g., Active Directory, so that the user's credentials are available from any of a number of different computing devices.

Abstract: The present invention creates a model that maps object classes in an object-oriented environment to a data source. The model maps the relationship between properties of each object class and data of the data source. The present invention can be used with a data source such as a relational database, user interface, file system, or object-oriented database. An application's object classes and data source schema are designed independent of the other since the model can be used to map one to the other. The model is comprised of entities and attributes. An entity maps to an object class and to at least one table of the DBMS. An entity contains attributes either simple or derived. A simple attribute maps to a DBMS column. A derived attribute is a combination of other attributes and does not directly map to a DBMS column. A relationship creates a link between entities of the model. A relationship can be used to flatten an attribute or flatten a relationship.

Abstract: One embodiment is a method that builds a target database with transaction logs from an online source database. The transaction logs include both existing data in the source database and updates that occur to the source database while the target database is being built.

Abstract: A database-rearranging program which is capable of rearranging a database while maintaining high reliability, without stopping services provided by the database. When a request for rearranging the copy source database is input, a record copying module sequentially selects pages in the copy source database, and the records in the selected page are copied to the copy destination database. Further, the record copying module sets a progress of copying of each page of the copy source database to a page status management table. Thereafter, when a request for writing updated records is input, a record writing module refers to the page status management table, and when a page of the copy source database in which the updated records should be written is in an uncopied status, the updated records are written in the page of the copy source database, and when the same is in a copied status, the updated records are written in the copy source database and the copy destination database.

Abstract: Systems, methodologies, media, and other embodiments associated with index replication using crawl modification information are described. One exemplary system embodiment includes an enterprise search system comprising a target search system comprising an index logic that uses modified crawl information related to items associated with sources to maintain an index that supports searching of the items; and, a crawl search system comprising a pipeline processor configured to receive modified crawl information related to the items and to propagate the modified crawl information to the target system.

Abstract: The invention relates to data processing methods and systems including: a method of database replication in which information strings are assigned to serialization groups for processing; a method of memory management in which data is read from a storage space area whilst no data is written to it; a method of replicating a database in which a dynamic table is created to provided processing information for database members; and/or a method of replicating a database wherein tasks are allocated to program components without program components interacting.

Abstract: A computer implemented method, apparatus, and computer program product for obtaining an absolute path name for an open file system object. A file descriptor for the open file system object is identified in response to a process in a first software partition opening a file system object to form the open file system object. The absolute path name for the open file system object is cached in a path name field in a file structure associated with the file descriptor for the open file system object. The absolute path name for the open file system object in the path name field is used during migration of the first software partition to restore the open file system object in a second software partition.

Abstract: It is an object of the present invention to conduct data transfer or data copying between a plurality of storage systems, without affecting the host computer of the storage systems. Two or more auxiliary storage systems 100B, 100C are connected to a primary storage system 100A connected to a host device 180. The auxiliary storage systems 100B, 100C read journals of data update from the primary storage system 100A at respective independent timings, save the journals in prescribed logical volumes JNL 2, JNL 3, produce copying of the data present in the primary storage system 100A based on the journals present in the logical volumes JNL 2, JNL 3 at the independent timings, and save the copies in auxiliary logical volumes COPY 1, COPY 3. The primary storage system 100A holds the journals till both auxiliary storage systems 100B, 100C read the journals and restore. The timing of journal read can be controlled according to the journal quantity, processing load, and the like.

Abstract: A system and methods for parallelized backup and restore process and system are disclosed. In one embodiment, a method includes providing a massively parallelized analytic database, serializing a schedule of a transaction history of the massively parallelized analytic database, and creating a transactionally consistent copy of the massively parallelized analytic database. The method may include restoring one or more of an original system and a configurationally equivalent system to a transaction consistent state as of a time the transactionally consistent copy was created. The transactionally consistent copy may be stored on a separate system than the original system. Accessibility to the transactionally consistent copy may be retained on the separate system even when the original system is inaccessible by the storing of the transactionally consistent copy on the separate system.

Abstract: A cluster recovery process is implemented across a set of distributed archives, where each individual archive is a storage cluster of preferably symmetric nodes. Each node of a cluster typically executes an instance of an application that provides object-based storage of fixed content data and associated metadata. According to the storage method, an association or “link” between a first cluster and a second cluster is first established to facilitate replication. The first cluster is sometimes referred to as a “primary” whereas the “second” cluster is sometimes referred to as a “replica.” Once the link is made, the first cluster's fixed content data and metadata are then replicated from the first cluster to the second cluster, preferably in a continuous manner. Upon a failure of the first cluster, however, a failover operation occurs, and clients of the first cluster are redirected to the second cluster.

Abstract: A combination of synchronous and asynchronous replication of data is used to replicate a local database to a replicated database. The typical tradeoff between synchronous and asynchronous replication is optimized by using hybrid replication, which is to use synchronous replication for inserting new data and asynchronous replication for updating existing data. The combined use of synchronous and asynchronous in this manner provides an efficient replicated database where the replicated database can tolerate some delay in data updates but requires no data loss of new data.

Abstract: Various methods and apparatus are described for archiving of an electronic document between multiple interconnected archive units of a distributed server network in geographically-dispersed locations in order to store identical copies of the electronic document at the same time. An archival portal server in the distributed server network sends a five-step, two-phase commit protocol to a selected set of two or more transaction manager instances resident on remote archive units. The archival system reconciles if an error occurs between a start of a transmission of the electronic document and a permanent archiving of that electronic document, or the electronic document is stored in a permanent data storage location within each of the archive units at an end of the two-phase commit protocol making archiving of an electronic document to multiple locations an atomic operation.

Abstract: A system and method for maintaining multiple inode containers is used to manage file system objects in a single logical volume of a network storage server. The system provides multiple inode containers to store metadata for file system objects in the logical volume. The system may use a first inode container to store private inodes used by the storage server and a second inode container to store public inodes that are useable by clients of the storage server. During a replication process, a source storage server generates a set of replication operations based on inodes in the public inode container and excluding inodes in the private inode container. In a destination server implementing multiple inode containers, the server generates inodes based on the replication operations and stores the inodes in the public inode container. These new inodes are stored in the public inode container with the same inode number or identifier as the corresponding inode on the source storage server.

Abstract: An apparatus, system, and method are disclosed for replication of data management information. A base replication module replicates a base table on an active server to a first and second standby server. The base table comprises configuration parameters for storage devices, logical volumes, physical interconnections, logical interconnections, and logical subsystems of a storage system, and the like. A session replication module replicates first sessions from a session table stored on the active server to the first standby server and second sessions to the second standby server. The session table stores a plurality of sessions. Each session comprises a plurality of copy sets. The active server manages the sessions using the base table and the session table, and the first and second standby servers do not manage the sessions. A transfer module transfers management of the first sessions to the first standby server.

Abstract: A model edit control system controls editing of a data model. A model repository manager allows multiple users to copy a stored model stored in a repository for executing changes on multiple model copies in parallel. An action log manager crates a model action log for the stored model and tracks actions executed on the stored model. The action log manager also creates a current action log for each model copy to record actions executed on the model copy. A model merger manager merges the model copies into the stored model in the repository, by playing the actions in the current action log against the stored model in the repository.

Abstract: A method is provided for replicating transaction data from a source database to a target database wherein the transaction data is communicated from a change queue associated with the source database to the target database. An initial path is provided between the change queue and the target database for transaction data to flow. The initial path has a maximum transaction load capacity. It is then detected whether the current transaction load is close or equal to the maximum transaction load capacity of the initial path. If so, another path is provided between the change queue and the target database. Also, a method is provided of replicating transaction data from a source database to a target database wherein an associated with the target database has a maximum transaction threshold limit. The applier normally posts transaction data to the target database only upon receipt of a commit step or operation associated with respective transaction data.

Abstract: A method is provided for processing base level data of a change queue that is not serialized at the logical level. The base level change queue data is transaction data previously posted to a source database. A logical replication engine is provided to replicate and post the change queue data to a target database in the order that the transaction data is stored in the change queue. Upon detection in the logical replication engine of change queue data that is not serialized at the logical level, the change queue data is reserialized at the logical level before the logical replication engine applies the change queue data to the target database. The change queue data that is not serialized at the logical level may be posted to the target database using asynchronous database access.

Abstract: Performing a synchronization operation is disclosed. A first processing thread is used to copy a first file from a first data storage to a second data storage using a first connection. A second processing thread is used to copy a second file from the first data storage to the second data storage using a second connection. The first file is copied substantially concurrently with the copying of the second file. The synchronization operation results in first copied file and a second copied file in the second data storage. A real-time replication operation is performed. The first copied file and the second copied file are updated in an order determined at least in part by an order in which changes were made to the first file and the second file, respectively, as stored in the first data storage.

Abstract: A system and method for querying file system content. In one embodiment, the system may include a storage device configured to store a plurality of files and a file system configured to manage access to the storage device. The file system may be further configured to perform in-band detection of a file system content access event and to responsively generate a given metadata record. The system may further include a query system configured to query the given metadata record.

Abstract: It is an object of the present invention to conduct data transfer or data copying between a plurality of storage systems, without affecting the host computer of the storage systems. Two or more auxiliary storage systems 100B, 100C are connected to a primary storage system 100A connected to a host device 180. The auxiliary storage systems 100B, 100C read journals of data update from the primary storage system 100A at respective independent timings, save the journals in prescribed logical volumes JNL 2, JNL 3, produce copying of the data present in the primary storage system 100A based on the journals present in the logical volumes JNL 2, JNL 3 at the independent timings, and save the copies in auxiliary logical volumes COPY 1, COPY 3. The primary storage system 100A holds the journals till both auxiliary storage systems 100B, 100C read the journals and restore. The timing of journal read can be controlled according to the journal quantity, processing load, and the like.

Abstract: A system and method for management for data replication in a data storage environment is disclosed. The system includes an architecture, software, and methodology for recovering from failure of software agents, which are operational in the data storage environment, by an ordered allocation of non-failed software agents to compensate for the failed agents.

Abstract: A checkpoint free log mining mechanism comprising a capture process and an apply process that are logically associated with each other may be provided in a database system. In an embodiment, log mining information published by the capture process to the apply process comprises a complete set of log information for each interesting transaction. A system change number of a start transaction record of a transaction may be used to represent a start time of the entire transaction. The capture and apply processes may work cooperatively to establish a safe time point in the form of a particular system change number in logs to begin mining, under various circumstances such as in an initial startup state, in a steady state, in a process restart scenario in the middle of checkpoint free log mining, or in a transition between the checkpoint mode and the checkpoint free mode.

Abstract: During replication of transaction data from a source database to a target database via a change queue associated with the source database, one or more multiple paths are provided between the change queue and the target database. The one or more multiple paths cause at least some of the transaction data to become unserialized. At least some of the unserialized data is reserialized prior to or upon applying the originally unserialized transaction data to the target database. If the current transaction load is close or equal to the maximum transaction load capacity of a path between the change queue and the target database, another path is provided. If the maximum transaction threshold limit of an applier associated with the target database has been reached, open transactions may be prematurely committed.

Abstract: The present invention provides a method and system for the automatic extraction of data from a transaction system to an analytics system, which is capable of handling large volumes of application data.

Abstract: A last transaction for a business object may be identified. Transactions related to that occur prior to the last transaction and subsequent to a last upload event are associated with the last transaction. Data describing the last transaction and the associated transactions may be transmitted over a communications network. Related apparatuses, techniques, systems, computer program products are also described.

Abstract: A method and apparatus for providing web pages to a client with improved speed and efficiency. Web pages are cached in a web server and are referenced from a database in a database server. When a request from a client is sent to the web server for a web page, the web server checks the web page cache table in the database server. The web page cache table contains a field wherein a flag is set to indicate whether any data that affects the web page has been updated. Database triggers are used to change the value of the flag when specified data fields corresponding to a web page have been changed. If the value of the flag indicates that there has not been any change in the data, the web server reads the html file reference in the web page cache table, retrieves the corresponding page from the web page cache in the web server and sends the page back to the client.