Abstract: A document collaboration system allows applications to collaborate on a binary data file even if the binary data file is not in a collaborative document format. In response to a request from an application to access a binary data file, the document collaboration system gives the application access to a document model corresponding to the binary data file and the application. If the document model does not already exist, it may be created by generating an empty document model, transmitting the binary data file to the application, and allowing the application to fill in the empty document model based on the binary data file. The document model may be provided to and modified by its related application through an application programming interface (API), and changes to the document model may be applied to the binary data file.

Abstract: A method comprising, in a cloud computing system: receiving a new job at the cloud computing system; sampling VMs (Virtual Machines) of the cloud computing system for the load currently handled by each of the VMs; if the load currently handled by the VMs is within operational bounds, sending the new job to one of the VMs which currently handles the highest load compared to other ones of the VMs; and if the load currently handled by the VMs is beyond operational bounds, sending the new job to one of the VMs which currently handles the lowest load compared to other ones of the VMs.

Abstract: A file system structure logically stored on a set of logical data volumes of a block storage system is discovered. A request to retrieve a first data block that is logically stored on the set of logical data volumes is received, and responsive to receiving the request, a data file within the file system structure is identified, in which a first portion of the data file is logically stored in the first data block. A second data block logically stored on the set of logical data volumes and physically stored on a set of data storage devices is located, in which a second portion of the data file is logically stored in the second data block. The second data block is read from the set of data storage devices, and stored in a cache memory of the block storage system.

Abstract: An indication of one or more changes to file system data since a last backup snapshot is received. One or more changes to the file system data between a requested backup version and the last backup snapshot are determined. One or more data blocks corresponding to the one or more changes to the file system data since the last backup snapshot and one or more data blocks corresponding to the one or more changes to file system data between the requested backup version and the last backup snapshot are provided to a primary system.

Abstract: According to one embodiment, a storage system comprises a controller being configured to: receive a write data of an application from a computer; manage the write data as a plurality of data sets, each of the plurality of data sets having a size of a deduplication unit; and apply a deduplication function to each of the plurality of data sets. When a data set of the plurality of data sets has an application metadata portion containing application metadata, the controller separates the application metadata from the data set and fills the application metadata portion by a padding data and applies the deduplication function to the data set including the padding data.

Abstract: Embodiments of the present disclosure may be utilized to analyze a content item comprising text to identity: a quote, a named entity that is the source of the quote (e.g., a person or organization such as a company), and identification information for the named entity (such as a title of the person giving the quote). Quotes may also be ranked to determine, for example, the best quotes to include in an article.

Abstract: Provided herein is a filing system creating and deleting apparatus, comprising an available inode pool in which inodes are to be stored, a file system object creating processor creating an inode allocating request signal in response to a file system creating request from an external resource, an inode allocating processor allocating at least one of inodes stored in the available inode pool in response to the inode allocating request signal, a file system object deleting processor creating an inode deallocating request signal in response to a file system deleting request from the external resource, an inode deleting thread collecting inodes requested to be deleted corresponding to the file system deleting request, and an inode deallocating processor deallocating the inode requested to be deleted from the inode deleting thread, wherein a number of inodes stored in the available inode pool is limited to no more than a maximum inode number in the available inode pool, and the inode requested to be deleted creates

Abstract: The subject disclosure is directed towards partially recalling file ranges of deduplicated files based on tracking dirty (write modified) ranges (user writes) in a way that eliminates or minimizes reading and writing already-optimized adjacent data. The granularity of the ranges does not depend on any file-system granularity for tracking ranges. In one aspect, lazy flushing of tracking data that preserves data-integrity and crash-consistency is provided. In one aspect, also described is supporting granular partial recall on an open file while a data deduplication system is optimizing that file.

Abstract: A method begins with a processing module receiving an access request and determining security requirements corresponding to the access request. The method continues with the processing module determining a subset of a plurality of dispersed storage units based on the security requirements. The method continues with the processing module determining, based on the security requirements, a connection security level for communicating with the subset of the plurality of dispersed storage units regarding the access request. The method continues with the processing module communicating the access request to the subset of the plurality of dispersed storage units in accordance with the connection security level for processing by the subset of the plurality of dispersed storage units.

Abstract: The concepts relate to data handling, and more specifically to data handling scenarios where data is revised on one computer and stored on another computer. One example can obtain a set of blobs relating to revisions of a file. The example can determine a target size of datastore blobs. In an instance where a total size of the set of blobs is less than the target size, this example can aggregate the set of blobs into an individual datastore blob. Otherwise, the example can identify new or edited individual blobs of the set and aggregate the new or edited individual blobs into first datastore blobs. The example can also aggregate other individual blobs of the set into second datastore blobs.

Abstract: Data deduplication is performed by separating data into a plurality of data chunks that correspond to first through Nth positions and include symbols, calculating discrimination indices of the positions using frequencies of the symbols in the different positions; arranging the order of the positions based on values of the discrimination indices; and generating fingerprints of the data through combination of data chunks that correspond to a number of the positions, based on the arranged order of the positions.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: A method for data storage includes storing data in a set of memory blocks of a non-volatile memory. Each memory block, which holds a respective portion of the data, is classified as valid or invalid depending on whether the memory block holds a most updated version of the portion, and as anchor or non-anchor depending on whether the portion belongs to a coherent snapshot of the data. Upon recovering from a power interruption, the coherent snapshot of the data is reconstructed from the memory blocks, based on classification of the memory blocks as valid or invalid and as anchor or non-anchor.

Abstract: A method of operating a DSS to provision space on a volume on which a filesystem and its snapshots rest, having thin and thick files. It includes (a) receiving a reservation-altering storage command, (b) calculating a volume liability by summing together (1) a total number of allocated data blocks stored on the volume and (2) a maximum value, over each of the filesystem and the snapshots, of a difference between (A) a count of guaranteed thick blocks for that filesystem or snapshot based on a count received from the filesystem including a sum, over each thick file on the filesystem, of data blocks guaranteed to that file and (B) a number of uncompressed mapped data blocks marked as thick by the filesystem for that filesystem or snapshot, and (c) providing a number of blocks of backing store for the volume equal to at least the calculated volume liability.

Abstract: Implementations are provided herein for data deduplication, and more particularly, to post-process data deduplication on a large scale out storage system. Multiple techniques and implementations are disclosed that offer greater efficiency, higher performance, and more stability when performing post-process data deduplication at large scale. Disclosed implementations are based on a process for data deduplication involving four main phases: enumeration, commonality, sharing, and update. Multi-level hashing can be used to identify candidates for deduplication during the enumeration phase, providing a more efficient use of compute resources. In addition, datasets can be phase rotated through the post-process deduplication steps providing a more controllable deduplication environment as well as a more efficient use of resources.

Abstract: Embodiments of the present disclosure may be used to gather, rank, categorize, and perform other processing of various types of content. In some embodiments, content items such as text, images, video, and other content are received from a variety of different sources and are processed to generate an article containing selected content items. While there may be hundreds or thousands of separate articles and stories regarding a particular topic, embodiments of the present disclosure help provide users with a single concise article that contains high-quality content items selected from among a potentially vast number of disparate sources.

Abstract: In one embodiment, a method includes creating a first number of streams for a file system manager of a deduplicating storage system to access concurrently a type of data blocks, where each stream is for one file system and is identified by a stream identifier. The method further includes mapping stream identifiers to each of the type of data blocks passing through the first number of streams. The method further includes accessing the type of data blocks in storage units of the deduplicating storage system through a second number of streams, where the second number of streams are dedicated to the type of data blocks in the deduplicating storage system, where the second number is smaller than the first number, where the data blocks are tracked according to the mapped stream identifiers, and where the data blocks are stored in the storage units after a deduplication process to remove duplication.

Abstract: Ingest data for virtual volumes (V) is split into segments (B1, B2, B3, B4) of a size that can be buffered in main memory. Data deduplication processing then occurs directly on the segments (B1, B2, B3, B4) in main memory, without the need for disk I/O.

Abstract: In a memory system in an embodiment, in a case of normal operation, a control unit returns a write completion response upon completion of reception of write data from a host, and writes the write data into nonvolatile memory in a multiple values. In a case of unordinary power-off, changeover to operation using a backup battery is conducted and the control unit writes dirty data that is not completed in writing into the nonvolatile memory, into the nonvolatile memory with two values. When next boot, the control unit reads the dirty data from the nonvolatile memory into the volatile memory, and thereafter writes the dirty data into the nonvolatile memory in a multiple values.

Abstract: A method of filesystem-based communication of computer applications is provided. The method implemented using a filesystem communications interface (FCI) computer device coupled to a first computer and a second computer on which computer applications are installed. The method includes mounting file systems on the first computer and second computer by installing communications interface drivers, receiving a data transfer command that includes a data unit from the first computer, identifying that the data transfer command corresponds to a filesystem-based data transfer protocol, generating another data transfer command by converting the first data transfer command into a first network-based data transfer protocol, receiving the data unit from the first computer, and transmitting, using the second communications interface driver, the data unit to the second computer application by the using a third data transfer command.

Abstract: An in-database sharded queue for a shared-disk database is provided. First messages from at least one first enqueuer are stored on a first queue shard. The first enqueuer/s execute in a first instance of a shared-disk database system. The first shard includes a first plurality of subshards. Second messages from at least one second enqueuer are stored on a second queue shard. The second enqueuer/s execute in a second instance of the shared-disk database system. The second shard includes a second plurality of subshards. In volatile memory of the respective instances, a first message cache and a second message cache is maintained for the first shard and the second shard. The respective message caches include at least one buffer associated with the plurality of subshards.

Abstract: A store computer system is provided. The computer system includes a sharded store transactional database such as an order or inventory database. The database may include a first database shard which stores a first group of order or inventory data and a second database shard which stores a second group of order or inventory data which is different than the first group of data. The computer system may include a store order management computer system that implements a shard determination module which is programmed to receive a query regarding an order or item associated with the store, identify a database shard associated with the order or item from data in the query, and communicate data regarding the order or item to or from the identified database shard.

Abstract: A method for improving data storage and retrieval performance, comprising, for a tiered storage environment having levels corresponding to storage performance, using reference count information of at least one data segment maintained in the storage environment to determine which of the levels in the environment the data segment will be assigned; obtaining input/output performance data for input/output operations on the data segment and transforming the data into short term moving averages and long term moving averages; comparing the averages in a performance level to the long term moving averages of the data segment in the performance level and examining the reference count information over time to determine whether a performance trend of the data segment is increasing or decreasing; and promoting the data segment to a higher performance level in the tiered storage environment, if the data segment has an increasing performance trend.

Abstract: Embodiments of the present disclosure may be utilized to analyze a content item comprising text to identify: a quote, a named entity that is the source of the quote (e.g., a person or organization such as a company), and identification information for the named entity (such as a title of the person giving the quote). Quotes may also be ranked to determine, for example, the best quotes to include in an article.

Abstract: A computing system includes a storage device in which file data is stored through data blocks and metadata is stored through a node block, and a file system configured to manage the file data and the metadata stored in the storage device. The node block includes data pointers respectively pointing to the data blocks, and one or more extents each indicative of data block groups which include data blocks having continuous physical addresses among the data blocks.

Abstract: A method is used in managing data deduplication in storage systems based on I/O activities. I/O activity of first and second data objects is evaluated. The first and second data objects are selected for applying a deduplicating technique. Based on the evaluation, the deduplicating technique is applied to the first and second data objects.

Abstract: A non-transitory computer-readable recording medium has stored therein a program for causing a computer to execute a process. The process includes identifying a data block from among a plurality of data blocks in a first storage for relocation to a second storage, determining an access mode of the identified data block, the access mode including sequential access or random access, and relocating the identified data block to the second storage based on the determined access mode.

Abstract: According to one embodiment, a method for determining a transfer order of files includes acquiring the access count of a replicated file at a production site and at a remote site, these access counts being reflected in the order for asynchronous replication. In addition, the access count of backed up files at the remote site is also acquired and reflected in the order for asynchronous replication according to another embodiment. This allows for improved determination of the transmission order of files (according to priority based on usage) from a production site to a remote site in an asynchronous replication function.

Abstract: A request for a document is received at a user device while in an offline state. The request includes a public uniform resource locator (URL) corresponding to a location remote from the user device. The user device launches an application that intercepts the request and selects one of multiple local caches based on the public URL. Each of the multiple local caches is associated with a private URL. The user device accesses the selected local caches via a corresponding private URL to retrieve an editor. The user device modifies a URL history to replace the private URL with a public URL and provides the document, via the editor, to the user device.

Abstract: Various systems and methods selectively generate a point-in-time copy of less than all of the data items within a collectively-managed set of data items. One method involves detecting a modification to a first unit of data within a collectively managed set, subsequent to generation of a selective snapshot of the collectively managed set, and then accesses a first tag associated with the first unit of data. Based on a value of the first tag, the method determines whether the first unit of data is included in a subset of the collectively managed set. The subset includes at least two of the units of data and fewer than all of the units of data in the collectively managed set. Based upon whether the unit of data is included in the subset, the method selects whether to preserve an original value of the first unit of data.

Abstract: Aspects of the disclosure relate to routing queries to a network repository and load balancing in a network. For a network repository having a plurality of content storage sites, relative replication latency of data among a pair of content storages sites can be monitored. Data indicative of such replication latency can be distributed among the content storage sites and can be provided, for example, to a network node in a system layer. A traffic and control manager can determine routing pathways for queries based at least in part on the relative replication latency data and performance conditions of network nodes.

Abstract: Database updates to primary and secondary database structures are performed by writing information from a client to both primary and secondary database services and building the database structure at the primary database service while delaying application of data written to the secondary database service until notification at the secondary database service from the primary database service. The primary database service provides notification upon completing the database structure so that the secondary database service can proceed with building the database structure from the information provided by the client or with information provided by the primary database service.

Abstract: A SAN management agent computes a validator corresponding to a full data set from which a relative (change) data set is derived. The validator, such as a checksum, identifies the full data set to which the relative data set is applicable. The validator is transmitted with the relative data set, and the store process compares the validator with the previous validator stored with the full data set in a database. If the validators correspond (i.e. match), then the received relative data set is applicable to the full data set stored at the database, and the relative data set may be processed as a change set. The remote agent therefore computes the validator on a full data set at a remote node, and transmits the validator with the change set. Subsequent relative data sets may then compare the current validator of the DB data set with the validator from which the relative set was derived.

Abstract: Techniques for providing data preview before recalling large data files are disclosed. In one aspect, a data file is made accessible while being offline by converting the data file from a native format to a preview format, storing the data file in the preview format in a primary storage that is locally available and moving, after the conversion to the preview format, the data file in the native format to a secondary storage. When a viewing request is received for the data file, the data file in the preview format is displayed to fulfill the viewing request.

Abstract: Embodiments of the present invention provide a method, system and computer program product for consistent replication of transactional updates. In an embodiment of the invention, a method for consistent replication of data in a transaction processing system is provided. The method includes recording entries in a replication log of different data updates and corresponding transactions and additionally recording entries in the replication log indicating whether or not the transactions have been backed out. The method also includes replicating only those data updates referenced in the log which do not correspond to transactions indicated in the log to have been backed out. For instance the additionally recorded entries in the replication log indicate when a transaction has been backed out. Alternatively, the additionally recorded entries in the replication log indicate when a transaction has been committed.

Abstract: A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.

Abstract: A user uploads content such as files containing audio, video, graphical, data, points of interest, and other information from a user device such as a personal computer to a central server over the internet. Upon determining that a trigger event has occurred, communication and processing circuitry in a vehicle device automatically download the previously uploaded content over the interne and a short-range wireless network and store the content for use by a device such as an audio/visual/navigation unit. Examples of a trigger event include proximity to a short-range wireless communication network coupled to the internet, presence within a geofence, turning off a vehicle's engine, or detecting an SMS wake-up message while the vehicle device's main processor, transceivers not used for SMS, and auxiliary circuitry are in sleep mode. The short-range wireless network can be a vehicle owner's home network, or a commercial wi-fi hot spot, or subscription wireless service.

Abstract: A write request is received at a database server from a client application for writing data to persistent data storage. In response to receiving the write request, the database server selects a set of multiple replication servers. The data is sent from the database server to the selected set of multiple replication servers for writing to the persistent data storage. Confirmation is received at the database server from replication servers in the selected set of multiple replication servers. In response to receiving confirmation from the replication servers in the selected set of multiple replication servers, the database server sends to the client application information indicating success of the write request.

Abstract: Techniques for replicating data to a database include determining a change to at least one database value in a database table on a source repository; updating, based on the change to the at least one database value, a snapshot value associated with the change to the at least one database value stored in a log table; incrementing a commit value associated with the snapshot value stored in a commit table, the commit table comprising a plurality of commit values associated with a plurality of corresponding snapshot values; initiating replication of the database from the source repository to a target repository; determining a maximum commit value of the plurality of commit values; and replicating the database value in the database table from the source repository to the target repository based on the commit value associated with the snapshot value being less than or equal to the maximum commit value.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: A repartitioning optimizer identifies alternative repartitioning strategies and selects optimal ones, accounting for network transfer utilization and partition sizes in addition to traditional metrics. If prior partitioning was hash-based, the repartitioning optimizer can determine whether a hash-based repartitioning can result in not every computing device providing data to every other computing device. If prior partitioning was range-based, the repartitioning optimizer can determine whether a range-based repartitioning can generate similarly sized output partitions while aligning input and output partition boundaries, increasing the number of computing devices that do not provide data to every other computing device. Individual computing devices, as they are performing a repartitioning, assign a repartitioning index to each individual data element, which represents the computing device to which such a data element is destined.

Abstract: A system and method for effectively supporting data transfer procedures includes a source device that registers with an account server to participate in a data backup service. The source device then encrypts and stores user data onto a datacenter. The source device later may request a data transfer task from a vendor. The datacenter responsively transfers the encrypted user data to a vendor server, and an escrow server generates and sends a temporary key to the vendor server. A destination device may then utilize the temporary key to decrypt and securely store the user data onto the destination device.

Abstract: In one aspect, a method to replicate a key-value pair includes intercepting a command to update a key-value pair in a key-value pair database, the key-value database comprising metadata of a virtual volume, sending an updated key-value pair to a data protection appliance, receiving an acknowledgement that the data protection appliance received the updated key-value pair and updating the key-value pair in the key-value database after the acknowledgement is received.

Abstract: Methods and apparatus for receiving uploaded data from a sender at a receiver. A data deduplication technique is described that may reduce the bandwidth used in uploading data from the sender to the receiver. In the technique, the receiver, rather than the sender, maintains a fingerprint dictionary for previously uploaded data. When a sender has additional data to be uploaded, the sender extracts fingerprints for units of the data and sends the fingerprints to the receiver. The receiver checks its fingerprint dictionary to determine the data units to be uploaded and notifies the sender of the identified units, which then sends the identified units of data to the receiver. The technique may, for example, be applied in virtualized data store systems to reduce bandwidth usage in uploading data.

Abstract: A system for managing email attachments receives and opens, on a mobile device, a document attached to an email; converts the opened document to a lower-resolution document; stores the lower-resolution document; and synchronizes the stored lower-resolution document with the mobile device. The system can also generate and store a corresponding thumbnail of the lower-resolution document. The original of the opened document may be stored in a remote storage device, optionally encrypting the low-resolution document before storing. The original of the opened document may also be broken into a plurality of pieces which are stored in different remote storage devices.

Abstract: Embodiments include methods, apparatus, and systems for snapshots in distributed storage systems. One method of software execution includes using a version tree to determine what data blocks are shared between various storage nodes in the version tree in order to create a clone or a snapshot of a storage volume in a distributed storage system that uses quorum-based replication.

Abstract: A computer implemented method, apparatus, and computer program product for managing aircraft software parts. In one advantageous embodiment, a method includes retrieving a set of aircraft software parts and metadata about the set of aircraft software parts from a library in an aircraft network data processing system to form a set of retrieved aircraft software parts. The set of aircraft software parts is stored in a file system. Metadata is stored in a database. The set of aircraft software parts and the metadata is sent to an onboard electronic distribution system, a communications link between an on ground component interface, and the onboard electronic distribution system. Status information about activity on the communications link is maintained.

Abstract: A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.

Abstract: A nonstop service system using voting and a method for updating and providing information in the nonstop service system. The nonstop service system includes a plurality of groups of nodes for storing and managing information on the basis of identifiers for distinguishing clients, each group including a plurality of nodes each of which is capable of storing and managing information independently. The nonstop service system further includes a control dispatcher server, which is located between the group of nodes and the clients and manages state information and connection information of the nodes belonging to the plurality of groups of nodes.

Abstract: An on-disk structure of a file system has the capability to maintain snapshot and file system metadata on a storage system. The on-disk structure arranges file system data sequentially on disk within regions, wherein each region comprises a predetermined amount of disk space represented by blocks. The snapshot and file system metadata is maintained within level 1 (L1) indirect blocks of the on-disk structure. Each L1 indirect block describes (i.e., represents) a corresponding region of the on-disk structure of the file system; in the case of an active region, e.g., an L1 indirect block represents an active file data portion of a large file. The L1 indirect block that references an active region also performs file block number-to-disk block number mapping for the region.