Abstract: A method for online authentication includes receiving membership authenticating information specific to members of a particular affiliation from the members and from one or more remote databases. The information is aggregated and stored in an aggregate database. An individual is authenticated, via a widget at least one of integrated into, and accessible by, at least one of a mobile application and a website of a provider of at least one of a particular program and a particular service, as a member of the particular affiliation based on a comparison of authenticating indicia provided online by the individual and the information stored in at least one of the aggregate database and the remote databases. Digital credentials are provided to the individual for access to the at least one of the particular program and the particular service when the individual is authenticated. The credentials include a unique identifier, a login and password.

Abstract: The present disclosure relates to a system and techniques for preventing corruption of snapshot data by limiting the visibility of committed data. To do this, the system may maintain an index that indicates the highest transaction identifier value such that no future commits will have a transaction identifier less than or equal to the indexed transaction identifier value. In embodiments, if a read is performed, only transactions having a transaction identifier less than or equal to the index value can be read. Each time that a transaction is committed, the index value is updated to the transaction identifier for the transaction having the highest transaction identifier without any intermediary transactions.

Abstract: One or more portions of a text corpus indicative of operational risk from a set of risk assessment documents is identified. Contextual features from the one or more portions of the text corpus are determined by applying a natural language processing (NLP) algorithm on the one or more portions. Risk identifiers are extracted based on the determined contextual features. A risk assessment software is generated based on the extracted risk identifiers and an operational risk category.

Abstract: Systems and methods include detection of changes to data of a plurality of source objects of a source system, replicating the data changes to corresponding ones of the plurality of target objects, and, during the detection of changes to data and the replication of the data changes: detecting a first change to a first source object, identifying a first maintenance event based on the detected first change; determining a first one or more solutions to resolve the first maintenance event; presenting the first maintenance event and the first one or more solutions to resolve the first maintenance event to a user; receiving an indication from the user of a first solution of the first one or more solutions to resolve the first maintenance event; and, in response to the indication, executing the first solution.

Abstract: A multi-cluster configuration of a database management system in a virtual computing system includes a server that defines a first policy for a source database on a first cluster of a plurality of clusters. Each of the plurality of clusters is registered with the server and the first policy defines capture of snapshots and/or transactional logs from the source database on the first cluster. The server defines a second policy for the source database to replicate at least some of the snapshots and/or transactional logs from the first cluster to a second cluster of the plurality of clusters, captures a first snapshot and/or a first transactional log from the source database in accordance with the first policy, and replicates the first snapshot and/or the first transactional log to the second cluster in accordance with the second policy.

Abstract: A system and method is described for database split generation in a massively parallel or other distributed database environment including a plurality of databases and a data warehouse layer providing data summarization and querying functionality. A database table accessor of the system obtains, from an associated client application, a query for data in a table of the data warehouse layer, wherein the query includes a user preference. The system obtains table data representative of properties of the table, and determines a splits generator in accordance with one or more of the user preference or the properties of the table. The system generates, by the selected splits generator, table splits dividing the user query into a plurality of query splits, and outputs the plurality of query splits to an associated plurality of mappers for execution by the associated plurality of mappers of each of the plurality of query splits against the table.

Abstract: A replicated block storage service provides durable and high performance network-attached storage replicated in two or more zones of a single region, and remains available despite a single zone failure. A probe file is generated to determine a health state of a replicated disk. When a disk is degraded, a lease is created indicating which replica is trusted and providing visibility to backend jobs to facilitate reconciliation of data between the first replica and the second replica. Moreover, degraded file markers are generated for use by the backend jobs in quickly identifying the data to be copied.

Abstract: A processor may receive a request with a column-oriented operation. The query request may be related to updates to a row-oriented database based on the query request previously performed on the row-oriented database. The processor may determine the amount of time for replicating the updates to a column-oriented database. If the amount of time for replicating the updates to the column-oriented database is greater than a threshold amount of time, then the processor may create a transient file with the updates. The transient file may be in a format that is recognized by the column-oriented database. The processor may query the column-oriented database based on the query request and the transient file.

Abstract: A method for execution by a storage network, the method begins by determining a failure rate of storage nodes of an active storage pool, establishing a number of standby storage nodes based on the determined failure rate, identifying resource identifiers for the failed storage node, selecting an available standby storage node, facilitating populating the selected available standby storage node with data slices associated with the failed storage node, utilizing the selected available standby storage node, facilitating population of a replacement storage node with the data slices from the selected available standby storage node and facilitating processing of further receive data access requests for data associated with the associated resource identifiers by utilizing the replacement storage node.

Abstract: Data can be placed by an edge node in a computing environment using multiple criteria in a placement policy. For example, a processing device of an edge node can receive a write request for storing a data object. The processing device can select first and second criteria from a placement policy based on a tag for the data object. The processing device can determine a set of remote components that fulfill the first criterion. The processing device can then identify, from the set, a destination component that fulfills the second criterion. The processing device can transmit the data object to the destination component.

Abstract: A request for performing a data storing operation directed to a database table that comprises a plurality of table columns is received. Columnar compression metadata is accessed to identify one or more table columns in the database table, each of the one or more table columns being designated to store compressed columnar values. The columnar compression metadata is used to apply one or more columnar compression methods to generate, from one or more uncompressed columnar values received with the request for the data storing operation, one or more compressed columnar values to be persisted in the one or more table columns in the database table. A database statement is executed to persist the one or more compressed columnar values in the one or more table columns in the database table.

Abstract: System extracts metadata of system committed change numbers, system change numbers for backup copies. System generates start system change numbers as minimums of committed change numbers, end system change numbers as maximums of change numbers, and identifiers for archive log backup copies. System identifies dependent system change numbers from start number to end number for backup copy. System identifies backed-up system change numbers, from start numbers to end numbers for log backup copies, which are in dependent numbers. If backed-up numbers don't include all dependent numbers, system outputs missing system change numbers. System generates start numbers, end numbers, and identifiers for new archive log backup copies, identifies new backed-up system change numbers, from start numbers to end numbers for new log backup copies, which are in missing numbers. If new backed-up numbers include all missing numbers, system stores backup copy with each log backup copy corresponding to any backed-up numbers.

Abstract: A shard polling system fairly distributes stored items from producers to consumer processes and includes polling threads that poll for items from respective portions of a storage source, place the items in respective queues, and increment a global permit counter, restricted to a configurable maximum, that tracks the quantity of messages across the respective queues. The polling threads are restricted by respective shard permit counters that limit a quantity of items that may be moved from a storage source to a respective queue. A global thread applies a fairness technique, such as round robin, to fairly obtain the items from each of the respective queues and places the items in a global queue, decrementing the global permit counter, and incrementing the respective shard permit counter for each item obtained. Consumer threads poll the global queue (e.g., a blocking output queue) for the messages.

Abstract: Methods and systems for shard splitting are provided. Exemplary methods include: marking a source index as read only, the source index comprising a source shard, the source shard comprising a source reference; creating a target index, the target index comprising target shards, each target shard of the target shards comprising a target reference of target references; copying the source reference, the copying producing the target references; hashing identifiers in the source reference, each identifier being associated with a document of a plurality of documents of the source shard, the hashing assigning each document of the plurality of documents to a target shard of the target shards, the plurality of documents being stored in a file associated with the source reference; hard linking the file into the target references; marking the target index as read-write; and deleting the source index.

Abstract: System extracts backup metadata, comprising committed change numbers, system change numbers, types, database version identifiers, and details, for backup copies. System generates data structure, comprising start system change numbers, based on minimums of system committed change numbers, end system change numbers and maximums of system change numbers, database version identifiers, and identifiers including details, for the backup copies. System identifies at least first type of backup copies as candidate dependent backup copies and at least second type of backup copies as candidate preceding backup copies.

Abstract: Systems and methods include detection of changes to data of a plurality of source objects of a source system, replicating the data changes to corresponding ones of the plurality of target objects, and, during the detection of changes to data and the replication of the data changes: detecting a first change to a first source object, identifying a first maintenance event based on the detected first change; determining a first one or more solutions to resolve the first maintenance event; presenting the first maintenance event and the first one or more solutions to resolve the first maintenance event to a user; receiving an indication from the user of a first solution of the first one or more solutions to resolve the first maintenance event; and, in response to the indication, executing the first solution.

Abstract: Systems and methods provide an entity identifier (EID) for use in distributed systems, where the entity identifier includes inherent privacy features and where an estimate of the distinct count of the entity identifiers in a distributed system can be determined. A unique identifier (e.g., a GUID) for an entity is received. A hash value can be generated for the unique identifier using a hash function that is not guaranteed to generate unique values. An EID is created using a portion of the bits of the hash value and stored in a database. An estimated distinct count of entities based on a count of EIDs in the database can be determined based on the count of EIDs in the database and the size of the EID space.

Abstract: Systems, methods, and apparatuses enable a machine learning model to determine a risk probability of a URL. A query configurator receives a URL in a query and normalizes the URL. The normalized URL is segmented into a plurality of segments. The plurality of segments is serially provided to the machine learning model trained to provide an indication of risk associated with the URL. The indication of risk associated with the URL can be a probability value based on one or more risk probabilities determined for segment-segment transitions of the URL. A security service compares the probability value of the URL to a threshold value and performs a security action based on a result of comparing the probability value to the threshold value.

Abstract: A hardware accelerator device detects duplicate data blocks independently from a processor of an information handling system. The device includes an interface, a hash function, a bloom filter, and a data comparator. The interface receives data blocks. The hash function determines a hash of a received data block. The bloom filter includes comparison hashes, each associated with a comparison data block. The bloom filter determines if the hash matches a comparison hash. The data comparator directs the interface to receive the comparison block when the hash matches the comparison hash, and compares the received data block with the comparison data block to determine if they are identical. The device stores a pointer when the data comparison block determines that the blocks are identical.

Abstract: A method is used in managing deduplication of data in storage systems. A candidate data object is identified for deduplicating a data object by evaluating digests stored in a current digest segment to determine whether another digest matching a digest associated with the data block is stored in the current digest segment. The current digest segment includes a set of digests associated with a set of data blocks previously received for deduplication. Based on the evaluation, a deduplicating technique is applied to the data object. The current digest segment is stored in an index table. A previous digest segment associated with a digest stored in the index table matches the digest associated with the data block is replaced by the current digest segment.

Abstract: Implementations of the disclosure describe an expandable index including pages that may store object records. The index may be divided into buckets, and each bucket may be associated with pages from a pool of unused pages shared by the buckets. A table or log may be used to maintain a dynamic list of all unused pages. It may also maintain a list of pages currently associated with buckets. As pages are linked to or unlinked from a bucket, the table or log may be updated to reflect that a page has been linked to a bucket or that a page that is no longer linked to a bucket belongs to the pool of unused pages. As additional storage media are added to the system, additional pages may be added to the pool of the expandable index for use by existing buckets without changing the number of buckets of the index.

Abstract: A method, computer program product, and computer system for destaging, by a computing device, a metadata page that includes delta tuples from a hash-based index structure. A delta tuple indicator may be added to the hash-based index structure based upon, at least in part, destaging the metadata page. The metadata page that includes the delta tuples may be destaged from the hash-based index structure to a metadata store. Any of the delta tuples added prior to the delta tuple indicator may be ignored when destaging the metadata page.

Abstract: Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data processing. One of the methods includes: generating a first Bloom filter based on data of a local device, and sending the first Bloom filter to a remote device; receiving a second Bloom filter from the remote device, wherein the second Bloom filter is generated by the remote device based on data of the remote device; filtering data of the local device by using the first Bloom filter and the second Bloom filter to produce successfully filtered data and unsuccessfully filtered data; and performing a join operation on unsuccessfully filtered data.

Abstract: A technique for transferring data over a network leverages a standard NAS (Network Attached Storage) protocol to augment its inherent file-copying ability with fingerprint matching, enabling the NAS protocol to limit its data copying over the network to unique data segments while avoiding copying of redundant data segments.

Abstract: A method and apparatus of a device that grows and/or shrinks a table that is shared between a writer and a plurality of readers is described. In an exemplary embodiment, a device receives an entry to be added to the shared table. In response to receiving the entry, the device remaps shared table to add a new storage segment to the shared table. The device further adds the entry to the shared table, where the entry is stored in the new storage segment. In addition, the device updates a shared table characteristic to indicate that the shared table has changed. The device further shrinks the shared table by remapping the table to remove a segment of the table.

Abstract: A method, computer program product, and computing system for associating each data container of a first set of data containers in volatile memory with a metadata page. One or more metadata changes associated with a metadata page may be written to the data container associated with the metadata page. The one or more metadata changes stored in the first set of data containers may be written to a storage array.

Abstract: At a stream management service, one or more storage servers of a first repository are assigned to store at least a first partition of a first data stream. At the first repository, records of different partitions are interleaved. In response to determining that a set of data records of the first partition meet a first criterion, a rearranged version of the set in which the data records of the first partition are contiguous is stored at a second repository. Using the rearranged version, one or more records of the first partition are provided to an application which meets a lag criterion.

Abstract: A distributed software system and a method for routing transactions for execution are disclosed. The distributed software system has a database sub-system partitioned into shards and a transaction routing sub-system for ordering transactions. The transaction routing sub-system has a plurality of coordinator ports and a plurality of mediator ports. The coordinator ports receive transactions to be executed by the shards and generate local per-shard orders for the received transactions. The local per-shard orders are received by the plurality of mediator ports which are pre-assigned with respective shards. The mediator ports generate centralized per-shard orders of execution based on the received per-shard orders. A given centralized per-shard order of execution is an order of execution of transactions received by a given mediator port and that are destined to be executed by a given shard that is pre-assigned to the given mediator port.

Abstract: Technologies for scaling user interface (UI) backend clusters for database-bound applications include a plurality of UI application servers and a master database server. Each UI application server is configured to retrieve a session sequence ID associated with a present user session that corresponds to a sequence ID of a master database of the master database server during a most recent write operation of data to a master database during the present user session. Additionally, the UI application servers are configured to read, from a local read-only database, a local database sequence ID that corresponds to a sequence ID of the master database during a most recent asynchronous replication of the master database to the local read-only database. The UI application servers are further configured to read the data from the local read-only database in response to a determination that the session sequence ID is empty, or is less than or equal to the local database sequence ID.

Abstract: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Data is written in segments and Metadata is stored in the segments indicating LBAs of data stored therein and offsets within the segments at which data for LBAs are stored. For write requests, index entries are stored in an index buffer for a segment and written to the segment when the buffer is full. A redo entry is created in a redo buffer for multiple storage volumes and slices for each write request. Write requests are acknowledged when the redo buffer is written to redo segments on a storage device. On restart, index buffers are reconstructed from the redo segments if needed.

Abstract: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A block map records segments where current data for an LBA of a slice of a storage volume is stored. Block maps may be written to a storage device in order to free memory. Block maps may be read back into memory when needed. Writing and reading of block maps may be performed upon fragments of block maps. On restarting of the storage node, block maps may be restored from block maps stored in the storage device.

Abstract: A method includes reading, by a capture engine, log data. The log data can be synchronously hardware-replicated. The log data reflects that an update was started for a database of a first site. The capture engine corresponds to a capture engine of a second site. The method also includes identifying, by the capture engine, the update. The method also includes determining whether the update was committed to the database of the first site. The method also includes replicating, by an apply engine, the update to a database of the second site based on the determining. The update is replicated to the database of the second site based on a determination that the update was committed to the database of the first site. The committed update corresponds to a consistent replication of updates across two or more databases that reside in multiple database management systems.

Abstract: Described herein are systems and methods for transparent resiliency to multiple data centers. The systems and methods depicted determine if an application programming interface (API) call is idempotent. If the API call is idempotent, the calls can be concurrently forwarded to multiple datacenters. If the API call is not idempotent, the calls can be sent to each of a multiple datacenters in turn until a response is received or timeout occurs. Automatically providing multi-region calls in synchrony provides a faster response time during data center or regional failures. Automatically providing multi-region calls in synchrony at the appliance server side, moves the logic out of the client and into a transparent and centrally managed service. This can allow business logic to focus on the core logic and not on logic to retry requests or manage the multi-regional aspect of a dependent service.

Abstract: A storage system includes a storage device, and a processor configured to store first data in the storage device, receive second data, specify a first data size in accordance with a second data size of the second data, divide the first data into a plurality of first blocks each having the specified first data size, divide the second data into a plurality of second blocks each having the specified first data size, and store, in the storage device, data included in a block of the plurality of second blocks, while data included in another block of the plurality of second blocks is not stored in the storage device, wherein the data of the block is not identical to data of any of the plurality of first blocks, and the data of the another block is identical to data of any of the plurality of first blocks.

Abstract: Database replication is provided by accessing, by a subscribing application, a message queue through a messaging queue manager. The subscribing application obtains a message provided by a publishing application from the message queue. The message includes message data that corresponds to shard data in a shard database. The subscribing application then provides the message data included in the message to a de-shard database as de-shard data. The de-shard data is an instance of the shard data and is modifiable by the subscribing application without changing the shard data.

Abstract: A computer-implemented method for replicating the effect of batch jobs using a replication agent is provided. The computer-implemented method comprises a replicating agent maintaining a maximum level in rows and a minimum level in rows for one or more columns in a database table of one or more database systems. The replicating agent further analyzes a mapping defined in the replicating agent to identify source columns of a source database system of the one or more database systems which are mapped to a target column of a target database system of the one or more database systems. According to at least one embodiment, the replicating agent further identifies a logical clause of the source database system and the target database system to define a range refresh. The replication agent further initiates the defined range refresh.

Abstract: A method for replicating the effect of batch jobs using a replication agent is provided. The method comprises a replicating agent maintaining a maximum level in rows and a minimum level in rows for one or more columns in a database table of one or more database systems. The replicating agent further analyzes a mapping defined in the replicating agent to identify source columns of a source database system of the one or more database systems which are mapped to a target column of a target database system of the one or more database systems. According to at least one embodiment, the replicating agent further identifies a logical clause of the source database system and the target database system to define a range refresh. The replication agent further initiates the defined range refresh.

Abstract: A non-volatile memory (NVM) apparatus and an iteration sorting method thereof are provided. The NVM apparatus performs the iteration sorting method to select one target block from a plurality of blocks of a NVM, and to perform a management operation on the target block. The iteration sorting method includes: selecting a plurality of candidate blocks among the blocks of the NVM to join into a sorting set, sorting all of the candidate blocks in the sorting set according to metadata, picking one candidate block with maximum (or minimum) metadata from the sorting set to serve as the target block, and keeping M candidate blocks in the sorting set and discarding the rest of the candidate blocks from the sorting set.

Abstract: For calculation of digest segmentations for input data using similar data in a data deduplication system using a processor device in a computing environment, a stream of input data is partitioned into input data chunks. Similar repository intervals are calculated for each input data chunk. Anchor positions are determined between an input data chunk and the similar repository intervals, based on data matches between a previous input data chunk and previous similar repository intervals. Digest segmentations of the similar repository intervals are projected onto the input data chunk, starting at the anchor positions.

Abstract: Data may be stored in a distributed server cluster which includes a plurality of nodes. A specific implementation of a method for storing data in the distributed server cluster comprises: receiving a data write request, the data write request comprising to-be-written data; broadcasting the data write request to each of groups in the server cluster, the each of the groups comprising at least one node, and at least one of the groups comprising at least two nodes; and sending a notification message of successful writing to a client sending the data write request, in response to receiving a confirmation message fed back from more than a half of the groups.

Abstract: A system, computer program product, and computer-executable method for managing replication from a production site to a target site, wherein both the production site and the target site are content addressable data storage systems, the system, computer program product, and computer-executable method including processing a replication session from the production site to the target site, collecting information related to the replication session from the production site to the target site, analyzing the information to determine whether to change the processing of the replication session, upon a positive determination, modifying the processing of the replication session.

Abstract: A method of generating user groups using sharding parameters is disclosed. The method identifies a set of sharding parameters including a number of user groups and a set of attributes associated with the number of user groups. The method defines one or more shards based on the number of user groups. The method further creates a user group based on the set of sharding parameters. The method further in response to determining that a number of users in the created user group exceeds a maximum number of users, modifies the set of sharding parameters. The method generates one or more modified user groups based on the modified set of sharding parameters, wherein each of the one or more user groups includes more than a single user.

Abstract: Techniques are provided for diverting, to cloned metadata, live access to original metadata of an application container that is being concurrently upgraded. In an embodiment, a database server stores, within an application container of an application, original metadata that defines objects for use by pluggable databases of the application. The database server receives a maintenance request to adjust the original metadata. The database server creates, in response to receiving the maintenance request, a reference container that contains cloned metadata that is a copy of the original metadata. The database server receives, during or after creating the reference container, a read request to read one of the objects. The database server concurrently performs both of: executing the maintenance request upon the original metadata, and executing the read request upon the cloned metadata of the reference container.

Abstract: Disclosed is a storage efficient multi-key sharding approach for use in a data management system. The data residing on the data management system can be partitioned based on multiple columns independently of one another, thus providing faster querying on each of the columns used for partitioning. The more partitioning attributes other than the key attribute of data records in the data records being partitioned can be used. Specifically, a multi-key partitioning approach consistent with embodiments described herein can be applied to perform load balancing between multiple parallel servers or server processes.

Abstract: In addition to caching I/O operations at a host, at least some data management can migrate to the host. With host side caching, data sharing or deduplication can be implemented with the cached writes before those writes are supplied to front end storage elements. When a host cache flush to distributed storage trigger is detected, the host deduplicates the cached writes. The host aggregates data based on the deduplication into a “change set file” (i.e., a file that includes the aggregation of unique data from the cached writes). The host supplies the change set file to the distributed storage system. The host then sends commands to the distributed storage system. Each of the commands identifies a part of the change set file to be used for a target of the cached writes.

Abstract: The disclosure is directed to optimizing data writes in a distributed computing system, e.g., a social networking application. The system employs a write-ahead log (WAL) and a hybrid logical clock (HLC) together to improve data writes across data centers in the distributed computing system, including improving: (1) consensus monitoring and enforcement in replicating data across the data centers; (2) scalability of data writes, e.g., so that different data centers can together handle a sudden increase in data writes during specified periods and consistently maintain the time ordering of the data writes; (3) replication configuration of data centers so that client devices can specify a replication hierarchy per transaction, e.g., at the time of writing the data, rather than specifying at data center configuration time; and (4) a draining process to write transactions committed to the WAL to data storage systems in the data centers.

Abstract: A data storage management approach is disclosed that performs backup operations flexibly, based on a dynamic scheme of monitoring block changes occurring in production data. The illustrative system monitors block changes based on certain block-change thresholds and triggers block-level backups of the changed blocks when a threshold is passed. Block changes may be monitored in reference to particular files based on a reverse lookup mechanism. The illustrative system also collects and stores historical information on block changes, which may be used for reporting and predictive analysis.

Abstract: In accordance with an embodiment, described herein is a system and method for exporting and importing a PaaS service and associated configurations across service domains are provided. The system comprises a cloud platform component provided as an installable software suite within a cloud environment; a test to production (T2P) framework module that includes a PaaS plugin and a plurality of T2P plugins and tools; and a user interface where the plugins and tools can be invoked to export domain configurations from a source service domain, and import the domain configurations to a target domain. In accordance with an embodiment, the system further includes a service provider interface (SPI) that can be implemented by an SME of each service housed in a service domain, to provide service backup and restore functionalities for exporting and importing that service.

Abstract: In accordance with an embodiment, described herein is a system and method for providing a clone a service in a cloud platform environment. The system can clone a source service within a service domain or across service domains, by creating a new service from a same service definition package used for creating the source service, extracting post-provision changes from the source service, e.g., user applications and post-provision configurations, and applying the changes to the new service.

Abstract: Systems, computer-readable mediums, and methods are configured to receive a first request to write a first data block to a solid state storage device. A second request is received to associate the first data block with a first snapshot identifier (ID). The first data block is stored on the solid state storage device in response to the first request. The first data block is stored on a first segment of a plurality of physical segments of memory on the solid state storage device. A first data pointer that corresponds to the first snapshot ID is generated in response to the second request. The first data pointer points to the first data block stored on the solid state storage device.