Abstract: A method, a system, and a computer program product are provided for data management. The method may comprise identifying a predefined set of data storage parameters and a predefined set of data scrubbing parameters. The method further may comprise determining the predefined set of data scrubbing parameters for first data in a first data storage based on the predefined set of data storage parameters for the first data and performing data scrubbing for the first data using the determined predefined set of data scrubbing parameters. Furthermore, the method may comprise comparing first data in the first data storage and corresponding second data in a second data storage. If a mismatch is determined between the first data and the second data, the method further may comprise synchronizing the first data with the second data as a result of the comparison.

Abstract: A method and system for monitoring and maintaining the consistency of replicated databases in a shared-nothing database cluster architecture is presented. The method involves the continuous monitoring of the database transaction logs that are maintained by the autonomous database managers that manage the individual database replicas in the cluster. In the event that data discrepancies are detected via the comparison of the transaction logs of the individual database replicas, remedial action is carried out according to configured rules in the main cluster controller. These remedial actions may include, but are not limited to, making a database node inactive within the cluster, or automatically repairing any data discrepancies that are detected among the database replicas in the cluster.

Abstract: Embodiments disclosed herein are directed to verifying query results of an untrusted server. A data owner outsources a data stream to the untrusted server, which is configured to respond to a query from a client with the query result, which is returned to the client. The data owner can maintain a vector associated with query results returned by the server and can generate a verification synopsis using the vector and a seed. The verification synopsis includes a polynomial, where coefficients of the polynomial are determined based on the seed. The data owner outputs the verification synopsis and the seed to a client for verification of the query results.

Abstract: A database management system for determining consistency in a hierarchical database sequentially reads pointers of a plurality of segment data, generates an address hash value, which is a hash value of retention addresses of the segment data having been read, generates a pointer hash value, which is a hash value of the pointers included in the segment data having been read, corrects the address hash value, when the contents of the address already read is updated, based a retention address of segment data added or deleted by the update, corrects the pointer hash value based on each pointer changed by the update, and determines that the hierarchical database is consistent if the corrected address hash value coincides with the corrected pointer hash value.

Abstract: A method is provided for resolving collisions in a database replication system. The system includes a source database and a target database, wherein changes made at the source database are replicated to the target database. Each database has one or more tables, and at least one of the tables has multiple constraints. A collision is detected during data replication as a result of one of the tables having multiple constraints. One constraint that contributes to the collision is identified. The constraint that contributes to the collision is relaxed, or the cause of the constraint that contributes to the collision is removed. This process is repeated for all additional constraints that contribute to the collision. In this manner, the collision is resolved.

Abstract: Based on a probability of summary metadata associated with an entity of a media hosting service being inaccurate, a determination is made as to whether to verify the accuracy of the summary metadata. In response to determining to verify the accuracy of the summary metadata, a determination is made as to whether the summary metadata is inaccurate with respect to master metadata associated with the entity of the media hosting service. The summary metadata is updated with a correct value determined from the master metadata, in response to determining the summary metadata is inaccurate.

Abstract: Network security is enhanced in a multi-tenant database network environment using a query plan detection module to continually poll the database system to locate and raise an alert for suspect query plans. Security also can be enhanced using a firewall system sitting between the application servers and the client systems that records user and organization information for each client request received, compares this with information included in a response from an application server, and verifies that the response is being sent to the appropriate user. Security also can be enhanced using a client-side firewall system with logic executing on the client system that verifies whether a response from an application server is being sent to the appropriate user system by comparing user and organization id information stored at the client with similar information in the response.

Abstract: A method for reducing data loss and unavailability by integrating multiple levels of a storage hierarchy is provided. The method includes receiving a read request. In addition, the method includes recognizing a data failure in response to the read request. The method further includes locating an alternate source of the data to be read in response to recognizing the data failure. The alternate source includes data cached at devices in the storage hierarchy, data in a backup system, and cumulative changes to the data since the last backup. Moreover, the method includes responding to the read request with data from the alternate source.

Abstract: In accordance with one embodiment of the disclosed technology, inconsistencies are detected between various records relating to data that has been associated with an identification tag. Data packages associated with the inconsistencies may then be removed. In accordance with another aspect of the disclosed technology, requests relating to data packages associated with inconsistencies in the various stored records are identified and removed. The disclosed technology may be implemented in data warehouses.

Abstract: Application Programming Interfaces (APIs) and UI modifications for existing applications are used to implement efficient methods for taking content offline with full synchronization, incremental synchronization, in the case of document libraries per-folder synchronization, and the like. The APIs provide support for synchronizing changes to attachments, the ability to download a document library's complete contents or only a list of the available documents, and read-only folder hierarchy replication in the case of document libraries. These abilities coupled with corresponding user interface features enable easier browsing and searching of large document library structures, making edits to those documents offline, being able to see an up-to-date version of those documents in a preview mode, and receiving reminders when offline edits remain on the local computer and need to be updated to a server.

Abstract: An apparatus, method, and machine-readable medium are described for managing data within a friend service. In one embodiment, a key/value data store can be used to provide an efficient and highly-scalable friend service database. Various techniques can be used to ensure data consistency between friend service data records. For example, in one embodiment, operations are detected which require updates to a plurality of related friend state records. A key may be generated to represent each of the operations and then used to create an entry in a write-ahead log database. In one embodiment, each entry in the write-ahead log database may specify the operation to be performed on the friend state records. If the plurality of friend state records are successfully updated, then the entry in the write-ahead log database may be deleted. However, if the plurality of friend state records are not successfully completed, then the entry is not deleted from the write-ahead log database.

Abstract: An investment portfolio management facility is disclosed. The facility is a computerized system for providing data integrity for investment portfolios. When an entry is made, the entry is checked for consistency with previously-made entries and with rules for portfolio entries. If an error is detected, the system alerts users to the error and also provides signals to check for further errors. The user may be made aware of the error or errors through a user interface.

Abstract: A method, an apparatus, and a system for subscription and notification are disclosed. The method includes the following steps: a Home Subscriber Server Front-End (HSS FE) receives a first subscription request which is sent by an Application Server (AS) and carries data content that the AS requests to subscribe to; the HSS FE sends a second subscription request to a convergent database, where the first subscription request and the second subscription request have a subscription relation; the HSS FE receives a notification sent by the convergent database according to the second subscription request; and the HSS FE sends data content carried in the notification to the AS according to the subscription relation. The present invention enables the AS to obtain the subscribed data from the HSS FE.

Abstract: An apparatus and method define the concept of a “database breakpoint.” One or more conditions in the database are specified as a database breakpoint in a debug environment for debugging a computer program that accesses the database. When the database conditions exist, one or more debug functions may be performed with respect to the execution of the computer program. Examples of debug functions include halting execution of the computer program and halting operations to the database by other programs. With both the computer program and database halted, the user may then interrogate the database to determine its state, and may interrogate the program to determine its state. Database breakpoints greatly enhance the utility of a debugger by providing additional conditions for debugging a computer program that accesses a database.

Abstract: A system and method verifies and repairs mirrored data sets. Verification modules executing on both the source and destination computers cooperate to identify a common persistent consistency point image (PCPI) of the source and destination data sets. Once the common image has been identified, the source sends a data stream comprising the data from the source data set to the destination to be compared with the destination data set.

Abstract: A database modernization system and method is disclosed. One example embodiment comprises a method for receiving a record stored as a source physical data model, parsing the source physical data model into a source logical data model, wherein the source logical data model includes semantic information from the record, transforming the source logical data model to a destination logical data model independent of a source or a destination record physical implementation, wherein the destination logical data model includes at least a portion of the semantic information from the record, and storing the destination logical data model in a destination physical data model. In this manner, a destination physical data model may be stored in a relational database management system on a per record basis.

Abstract: A source application reads a body of data in data block sized units and calculates a checksum value for each data block before sending the data block, the calculated checksum value and the identifier. Upon receipt, a destination application independently calculates a checksum value for each received data block and compares the two checksums. Non-matching checksums indicate a network-induced error in the data block. Identifiers for the erroneous data blocks are transmitted to the source application after all of the data blocks have been initially transmitted. The source application thereafter resends only those data blocks identified. The destination application repeats the process of comparing checksums and transmitting identifiers to the source application until all of the data blocks of the body of data have been correctly received, and then uses the data blocks to recreate the body of data.

Abstract: A system and method for managing data during consistency points in a storage system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the storage system may permit write operations to continue to a data container undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

Abstract: Records, including data for a virtual catalog, may be received from a number of sources. Each of the received records may be validated and a determination may be made as to whether one or more previously-received records matches the received record. When two or more records are determined to be matching records, a composite object may be created and information from the two or more records may be included in the composite object. If the matching records include conflicting information, then previously-defined rules may be applied to determine which items of the conflicting information have precedence over other items of the conflicting information. A virtual catalog may be created from the received records and the created composite object based on customized attributes.

Abstract: A system and method for analyzing official notices of an electronically filed patent application are based on a file server that connects to patent office websites, a database, and client computers. An official notice of the patent application is downloaded from one of the patent office websites according to a download command from one of the client computers, and is converted to an image format document. Character data of the image format document are identified, and patent information of the electronically filed patent application are extracted from the character data using a regular expression. In addition, the patent information are analyzed to generate analysis results of the official notice according to the comparison.

Abstract: A transaction system that allows for analysis and administration at the global transaction participant level is disclosed.

Abstract: According to an aspect of the present invention, information indicating the list of referrer documents referring to a referred document, is maintained. Thus, when the document identifier (name and directory location of the document) changes, the list of referrer documents that need to be changed can be easily identified. In an embodiment, the administrators of the referrer document are notified (e.g., by an automatic email) of the change of the document identifier of the referred document. According to another aspect of the present invention, a first mapping of the Uniform Resource Locator (URL) of each web accessible page/document to a virtual link is maintained in a web server. The content server maintains a second mapping of the virtual link to the document identifier. Thus, when the document identifier changes, only the second map in the content server needs to be updated for continued access of the content.

Abstract: Generally, the described system and process enables resolution of conflicts in a synchronized folder. Within the described mesh operating environment, each of the devices may be configured to do the same processing so that the file system view of the synchronized folder looks the same on all devices (pending local capabilities). Updates that cannot be immediately realized to the local store due to conflicts may be deferred for later attempts when, for example, additional updates at the system level or local level are made to resolve or eliminate the conflict for the update item. Generally, further changes may be propagated by a user in addressing a particular conflict that the user is notified about (e.g., via a selected winner that the user disagrees with). Alternatively, the conflict may resolve itself when a further update occurs that overrides or renders moot the previous update (e.g., a deleted item having a modified enclosure, where the enclosure had a previous concurrency conflict).

Abstract: An approach is provided for generating a polymorphic note. A polymorphic note manager receives a request, from a device, to generate a polymorphic note, wherein the polymorphic note includes information of a plurality of application types that is developed over a period of time. The polymorphic note manager then generates the polymorphic note in response to the request and stores the polymorphic note for access by the device.

Abstract: A computer-based method and apparatus for identifying hazardous materials based on observable physical properties and signs and systems of exposure to the hazardous materials. The observable physical properties are obtained by preprocessing plain language data to obtain normalized descriptors and modifiers pertaining to a plurality of properties and signs and symptoms. The normalized descriptors and signs and symptoms are presented to a user in lists for comparison with a sample in the field. Once a material is identified, information can be obtained from the apparatus which is necessary for response to an incident involving the hazardous material. The preferred embodiments of the invention are particularly useful to responders to a hazardous waste incident.

Abstract: A method begins by a processing module determining dispersed storage (DS) units that are storing a set of encoded data slices associated with a data segment and sending a revision level check request message to each of the DS units. The method continues with the processing module receiving revision level check response messages from at least some of the DS units within a time period to produce received revision level check response messages and determining concurrency of a revision level of the set of encoded data slices stored by the DS units based on the received revision level check response messages. The method continues with the processing module initiating a revision concurrency process when less than a threshold number of encoded data slices have concurrency of the revision level.

Abstract: A method for execution by a DS processing unit begins with the DS processing unit receiving a plurality of integrity data, which was generated by a plurality of DS storage units regarding one or more data slices of one or more pluralities of data slices. The method continues with the DS processing unit evaluating the received integrity data. The method continues with the DS processing unit requesting naming information of the one or more pluralities of data slices when the evaluation of the received integrity data yields unfavorable results. The method continues with the DS processing unit receiving the naming information from the plurality of DS storage units. The method continues with the DS processing unit evaluating the received naming information to identify at least one data slice of the one or more of pluralities of data slices having a data inconsistency.

Abstract: A system for the storage and retrieval of data files includes a data disk, a configuration disk and a controller. Each data file includes one or more data blocks, each data block comprising a data section. The data disk stores the data blocks and is divided into a plurality of block groups. The configuration disk stores a copy of at least a portion of the corresponding meta-data. The controller controls both the data disk and the configuration disk.

Abstract: Methods and apparatus, including computer program products, implementing a technique for repairing links in a file. The file can contain one or more links, each link referring to a target file. For one or more of the links, a file identifier such as a file name, file number, or file address is stored. Alternatively, for one or more of the links, target information is stored that identifies an old target and a new target for the link. The links can be repaired based on the stored information. The stored information can be used to identify changes to the links that occurred while the file containing the link was inaccessible to the system, for example, due to the file being locked by a user.

Abstract: A method for managing virtual machines, the method causing a management server having a database includes link relation information and connected to a plurality of physical servers, on which virtual machine hosts are installed, the method includes updating the link relation information in such a manner that a link between a virtual machine host and a first virtual machine guest is disconnected, when information regarding the first virtual machine guest cannot be acquired from the virtual machine host and updating the link relation information stored in such a manner that a link between a virtual machine host and a second virtual machine guest, when information regarding the second virtual machine guest is acquired from the virtual machine host.

Abstract: Systems, devices, and methods for modularizing data flows and for providing stand-alone execution of incomplete data flows are provided based on an improved data warehousing framework. Data flow modularization and code reuse is improved through the use of the disclosed flowlets and allows for consistent and efficient standalone testing of incomplete data flows. The disclosed details enable various refinements and modifications according to system design considerations.

Abstract: For a data processing system having memory for storing a database, a method, a system and a computer program product for directing the data processing system to process a record to be inserted into the database is disclosed. The database includes a plurality of base tables.

Abstract: Systems, devices, and methods for modularizing data flows and for providing stand-alone execution of incomplete data flows are provided based on an improved data warehousing framework. Data flow modularization and code reuse is improved through the use of the disclosed flowlets and allows for consistent and efficient standalone testing of incomplete data flows. The disclosed details enable various refinements and modifications according to system design considerations.

Abstract: A file operations engine is provided that manages many user interactions with their files via a computer system. The operation engine may provide a user with the option to keep both files that have a file name conflict. It may further permit the user to rename a file involved with a file name conflict. The operations engine may also automatically rename one of the files of a file name conflict by appending a character to a root of the filename. The character may include the lowest integer available for the root in a destination for the files. The operations engine may provide the option to keep both files as part of a pre-calculation of potential errors for a requested operation. The operations engine may place file name conflicts in an error queue and permit the user to select an option to keep both files after the conflict is encountered.

Abstract: A nonvolatile system memory of a server of a network file computer system temporarily stores data received from client computers prior to the data being stored on a mass storage device. A single copy of the data and pointers which corresponds to the memory addresses of the data are stored in a temporary memory storage portion, preferably partitioned from nonvolatile system memory. The pointers are used to access the data to enable the transfer and writing of the data from the temporary memory storage to the mass storage device. Writing the data occurs during the normal operation of the network file computer system or after an unclean shutdown. The use of the pointers increases the performance of the network file computer system by conserving system memory storage space and reducing system memory traffic.

Abstract: A system and method to efficiently perform database cleanup. Database objects are initially grouped based on selected attributes. The groups may be filtered to identify erroneous data within the group. Structure query language statements are then dynamically generated to directly correct the erroneous data within the database.

Abstract: Parallel reconstruction of file components following a failure of one or more of the storage devices is implemented in the context of a storage system that includes a plurality of storage devices for storing file components and a plurality of metadata managers. A storage device having one or more unrecoverable read errors requiring reconstruction is identified. A metadata manager which will serve as a scheduler, and a plurality of metadata managers which serve as a plurality of workers, are identified. The plurality of workers includes metadata managers other than the scheduler. A scheduler service running on the metadata manager identified as the scheduler is used to construct a list of file components from the storage device affected by the one or more unrecoverable read errors requiring reconstruction. The scheduler service assembles a work list corresponding to each of a plurality of the workers. The work list for each worker includes a subset of file components from the list requiring reconstruction.

Abstract: A file operations engine is provided that manages many user interactions with their files via a computer system. The operation engine may provide a user with the option to keep both files that have a file name conflict. It may further permit the user to rename a file involved with a file name conflict. The operations engine may also automatically rename one of the files of a file name conflict by appending a character to a root of the filename. The character may include the lowest integer available for the root in a destination for the files. The operations engine may provide the option to keep both files as part of a pre-calculation of potential errors for a requested operation. The operations engine may place file name conflicts in an error queue and permit the user to select an option to keep both files after the conflict is encountered.

Abstract: A file operations engine is provided that manages many user interactions with their files via a computer system. The operation engine may provide a user with the option to keep both files that have a file name conflict. It may further permit the user to rename a file involved with a file name conflict. The operations engine may also automatically rename one of the files of a file name conflict by appending a character to a root of the filename. The character may include the lowest integer available for the root in a destination for the files. The operations engine may provide the option to keep both files as part of a pre-calculation of potential errors for a requested operation. The operations engine may place file name conflicts in an error queue and permit the user to select an option to keep both files after the conflict is encountered.

Abstract: A method for controlling a relational database system, with a query statement comprised of keywords being analyzed, with the RTN being formed of independent RTN building blocks. Each RTN building block has an inner, directed decision graph which is defined independently from the inner, directed decision graphs of the other RTN building blocks with at least one decision position along at least one decision path. The inner decision graphs of all RTN building blocks are run by means of the keywords in a selection step and all possible paths of this decision graph are followed until either no match with the respectively selected path is determined by the decision graph and the process is interrupted, or the respectively chosen path is run until the end.

Abstract: A computer-implemented method for receiving information at a first device is provided. The method includes storing a first data object on the first device in a first device-specific representation, receiving a batch of information in response to the first device transitioning from an off-line state to an online state, processing the modification event to recover the modification, updating the first data object based on the recovered modification, and transmitting an acknowledgement indicating that the modification event was successfully processed by the first device. The batch of information includes a modification event. The content of the batch was determined based at least on a state of data stored on the first device. The modification event was generated based on at least a modification made to a second data object stored at a second device in a second device-specific representation.

Abstract: The present principles provide methods and apparatus for preventing data corruption on a storage device by integrating a journaling file system with a cache system. To ensure journal accuracy with respect to data that is most likely to affect file system integrity, a method in accordance with an aspect of the present principles includes bypassing the cache when writing such data to a main platter of a storage device. Furthermore, to ensure overall efficiency in reading and writing data, a method in accordance with an aspect of the present principles includes writing to a cache, in addition to writing to the platter, data that has a relatively less damaging effect on file system integrity. Thus, aspects of the present principles optimally integrate a cache system with a journaling file system to provide both a robust file system integrity and an efficient reading and writing mechanism.

Abstract: A system and method for managing file data during consistency points in a file system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the file system may permit write operations to continue to a file undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

Abstract: The present invention relates to a method for reducing a data repository with a plurality of sequentially ordered records. Each record is characterized by an identifier and comprises one or more data attributes and/or one or more links. The method includes the step of choosing a first record as base record and verifying whether the identifier of the sequentially next record can be generated from the base record by applying an identifier generating function. As a next step it is verified whether the data attributes and/or the links of the sequentially next record are identical with the data attributes and/or the links of the base record or can be generated from the base record by applying an attribute generating function and/or link generating function. If the above conditions are satisfied, a counter in the base record is incremented and the sequentially next record is deleted. The above steps are repeated for the subsequent records until the above conditions and are not satisfied for any record.

Abstract: Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Abstract: In an embodiment, estimated costs are used to pick the best refresh technique (e.g., method) for a given MV. In another embodiment, estimated costs are used to schedule the refresh of a set MVs in an order such that overall time to refresh is reduced, or in an order that uses the smallest overall time, for refreshing when compared to other alternative refresh schedules. In another embodiment, the estimated cost of refreshing each of a set of MVs is used to select the set of CPU resources to allocate to refreshing each MV. Based on the estimated costs, the allocation of CPU resources is chosen to be more optimal (e.g., use less CPU time) than other available sets of CPU resources. In an embodiment, when refreshing an MV, delayed index maintenance is chosen if a significant portion of the base table and/or the partitions of the table has changed.

Abstract: A source document set including one or more source documents to be migrated from a source application package to a destination application package can be analyzed for migration issues. The migration issues can be reported to a user and/or resolved prior to migration by software. Software can also be provided for the extraction of a source macro from a source document, parsing of the source macro, mapping of specific source APIs to destination APIs and the generation of destination macro code.

Abstract: A system and method verifies and restores the consistency of inode to pathname mappings. In a first embodiment, an off-line verification procedure is modified to verify and correct the primary name inode to pathname mapping information within inodes of a file system. In a second embodiment, an on-line file system verification process is modified to verify inode to pathname mapping information upon the loading of each inode within the file system.

Abstract: Methods and apparatus, including computer program products, implementing a technique for repairing links in a file. The file can contain one or more links, each link referring to a target file. For one or more of the links, a file identifier is stored that uniquely identifies the target. Alternatively, for one or more of the links, information is stored that identifies an old target and a new target for the link. The links can be repaired based on the stored information. The stored information can be used to identify changes to the links that occurred while the file containing the link was in a non-modifiable state, for example, in a deactivated state.

Abstract: An apparatus or method for preserving the relationships or dependencies between content of files in a file system or between content of files in separate file systems. The preservation of relationships or dependencies may require the preservation of the files and their content as well. In one embodiment, the method includes receiving a first instruction that, when implemented, performs an operation involving a first file in a file system, wherein the first file is related to the second file. In response to receiving the first instruction, a second instruction is selected that, when implemented, performs a second operation involving the first file or a second file, wherein the second instruction is implemented before or after implementing the first instruction.