Abstract: Computer-implemented methods for rebalancing a namespace range of a dispersed storage network (DSN) are provided. Aspects include calculating a namespace imbalance factor for each of a plurality of storage devices in the DSN, identifying a source storage device of the DSN having a largest namespace imbalance factor, determining a minimum namespace address, a maximum namespace address, an ideal minimum namespace address, and an ideal maximum namespace address for the source storage device, and based at least in part on a determination that the minimum namespace address is less than the ideal minimum namespace address and/or the maximum namespace address is greater than the ideal maximum namespace address, reassigning the left-deviated or right-deviated portion of the namespace range from the source storage device to its left or right neighbor storage device of the DSN.

Abstract: Systems, computer program products, and methods are described herein for dynamic visual graph structure providing multi-stream data integrity and analysis. The present disclosure is configured to provide a reactive system aimed to trace the root cause of incidents and uncover potential gaps in security of an enterprise system. Maintaining accurate and meaningful information related to incidents is the key for success of data security protocols. The integrity of message data is kept intact for improved forensic investigation, as each database may keep varying information related to a single global session.

Abstract: Provided are a metadata processing method and apparatus, and a computer-readable storage medium. In the metadata processing method, a management server obtains metadata to be processed, wherein the metadata to be processed includes a directory structure and file attributes; and according to a load condition of at least one first node for storing the file attributes and based on a rule that file attributes of a same directory are stored in a same first node, the management server stores, in the at least one first node, the file attributes of directories in the directory structure in a distributed manner.

Abstract: Systems and methods for protection of data across multiple users and devices are disclosed. According to one embodiment, in privacy server comprising at least one computer processor, a method for protection of data across multiple users and devices may include: (1) receiving, from a first user device, a data sharing permission for the first user device and a data sharing permission for a second user device, the first user device and the second user device associated with the same user; (2) provisioning the data sharing permission for the second user device; and (3) communicating the provisioned data sharing permission to the second user device, wherein the second user device shares data with the first user device according to the provisioned data sharing permission.

Abstract: A method classifies missing labels. The method computes, using a neural network model trained on training data, rank-based statistics of a feature of a time series segment to attempt to select two candidate labels from the training data that the segment most likely belongs to. The method classifies the segment using k-NN-based classification applied to the training data, responsive to the two candidate labels being present in the training data. The method classifies the segment by hypothesis testing, responsive to only one candidate label being present in the training data. The method classifies the segment into a class with higher values of the rank-based statistics from among a plurality of classes with different values of the rank-based statistics, responsive to no candidate labels being present in the training data. The method corrects a prediction by an applicable one of the classifying steps by majority voting with time windows.

Abstract: Techniques described herein relate to a method for distributed data management. The method may include obtaining, by a data management agent of a data host and from an application executing on the data host, a request to access data; obtaining, by the data management agent, an information set associated with the data; making a determination, by the data management agent, that at least a portion of the data is not ready to be used by the application; and performing, by the data management agent and based on the determination, a data preparation action set.

Abstract: Methods and systems for managing access and control of data are disclosed. To manage access and control, a data management system may use a variety of data minimization and resiliency techniques to differentiate different repositories for storage of data. The different repositories may store data in different manners, and provide different levels of performance regarding stored data. By differentiating the repositories, criteria may be established and enforced to prioritize different types of data for storage.

Abstract: One or more computer processors receive an address resolution protocol (ARP) request in response to a client connecting to a compute resource. The one or more computer processors identify an internet protocol (IP) address associated with the ARP request and the compute resource. The one or more computer processors determine the compute resource is unavailable or not ready for a connection with the client utilizing the identified IP address. The one or more computer processors, responsive to determining that the compute resource is unavailable, provision and deploy a new compute resource, wherein the new compute resource is available to the client under the identified IP address.

Abstract: A database system configured to manage and execute stored procedures within a secure sandbox process. The system receives a response to a database query and, through an Application Programming Interface (API) executing within the sandbox process, converts the response into a remote procedure call. The sandbox process is modified to restrict communication with external networks while enabling communication with a designated execution node, which facilitates interaction between the stored procedure and database system components. The stored procedure, executing within the confines of the sandbox process, directs the API to communicate with the execution node. The execution node, in turn, submits the database query to the database system component.

Abstract: An apparatus comprises a processing device configured to receive, at a given node of a distributed file system cluster, a request to store one or more portions of data and to monitor performance parameters for each storage device associated with the given node, the performance parameters comprising a first performance parameter characterizing data access frequency and at least a second performance parameter characterizing available space. The processing device is also configured to determine an overall performance metric for each of the storage devices associated with the given node based at least in part on the monitored performance parameters and to select, based at least in part on the overall performance metrics, at least one of the storage devices associated with the given node on which to store the one or more portions of data.

Abstract: Example implementations relate to deduplication operations in a storage system. An example includes generating a housekeeping work map to delete a backup item stored in a deduplication storage system; selecting a first work entry of the housekeeping work map, where the first work entry identifies a first container index and a first manifest; in response to a selection of the first work entry, loading the first container index into the memory, the first container index comprising a back-reference data structure; identifying, in the back-reference data structure, a back-reference entry indexed to the first manifest; determining, using the back-reference entry indexed to the first manifest, a first set of data units included in the first manifest and that are indexed in the first container index; and decrementing, in the first container index, a set of reference counts for the determined first set of data units.

Abstract: Various systems facilitate encrypted file storage. A client device may generate an encrypted version of a file. The client device may obtain at least one reference to at least one storage location for the encrypted version of the file. The client device may cause the encrypted version of the file to be store at the at least one storage location using the at least one reference to the at least one storage location.

Abstract: A device configured to identify a first link between a value of a first data element in a first plurality of data elements and values of a first set of data elements in a second plurality of data elements and to remove the first link between the first data element and the first set of data elements. The device is further configured to input the data elements into a machine learning model that is configured to output a second link between the first data element and a second set of data elements. The device is further configured to create an entry in a relationship table that identifies the first data element and the second set of data elements. The device is further configured to generate a data stream with the first data element and the second set of data elements and to output the data stream.

Abstract: Systems and methods include presentation of a subset of a result set of items received from a remote system, reception of a command to perform an operation on all items of the result set while presenting the subset, and determination, in response to the command, of whether a total number of items in the result set exceeds a threshold value. If it is determined that the total number of items in the result set exceeds the threshold value, a first request is transmitted to the remote system to perform the operation on all items of the result set, where the first request includes filter values associated with the result set. If it is determined that the total number of items in the result set does not exceed the threshold value, a second request is transmitted to the remote system to perform the operation on all items of the result set, where the second request includes an identifier of each item of the result set.

Abstract: A file sharing method, apparatus, and system, mainly relate to the technical field of computers. The method comprises: obtaining a file to be shared; concurrently transmitting the shared file compression packages onto a blockchain; forming data blocks according to a compressed package file name and a hash value thereof, wherein the data blocks comprise the hash value of a previous data block; using the hash value of a last data block as a root hash value; and according to the root hash value, obtaining a file comprised in the last data block, and then obtaining each file compression package in a blockchain indicated by the root hash value.

Abstract: In one embodiment, distributed data storage systems and methods are described for integrating a change tracking manager with scalable databases. According to one embodiment, a computer implemented method comprises managing storage of objects and continuously tracking changes of the objects in a distributed object storage database, creating a record for an object having an object name, the object being stored in a bucket of the distributed object storage database, linking the bucket to a peer bucket based on a directive, generating a peer marker field for the record to store one peer marker of multiple different peer markers depending on a relationship between the bucket and the peer bucket; and automatically adding a work item for the object to the secondary index of a chapter database based on the record being created in the bucket and the peer marker for the peer bucket.

Abstract: In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

Abstract: Adaptive matching is provided, which is used to automatically match a block size of a transactional file system with an IO size of a client. An example method includes creating a share domain, wherein the share domain is created on a first file system, and a block size of the first file system is a first block size. The method further includes determining a block size of the share domain as a second block size, wherein the second block size is not equal to the first block size. If a block size of a second file system is the second block size, the share domain is migrated from the first file system to the second file system. By implementing the present application, it is possible to simplify user operations, improve operational convenience, and help to reduce storage space fragments and indirect blocks, thereby further improving the performance of a storage system.

Abstract: A dataset is represented by an ordered set of B+ trees, with each snapshot of the dataset being represented by an immutable B+ tree. An ordering of B+ trees in the ordered set logically arranges those B+ trees into layers, with the mutable B+ tree representing the current state of the dataset being logically arranged as an upper layer, and with immutable B+ trees representing snapshots of the dataset being progressively arranged as lower layers with the oldest snapshot being a lowest layer. A snapshot deletion includes merging the snapshot's corresponding B+ tree with its highest neighbor B+ tree.

Abstract: Embodiments of this application relate to the terminal field and disclose a data transmission method and a terminal, to increase a speed of data transmission between terminals and ensure stability during the data transmission. The method includes: A terminal establishes a wireless connection to another terminal; in addition, the terminal establishes a USB connection to the another terminal; then, the terminal can display a first interface, where the first interface includes at least one piece of candidate data; a user may select to-be-transmitted data from the at least one piece of candidate data after performing a first input on the first interface; and the terminal may send the to-be-transmitted data to the another terminal through the wireless connection and the USB connection after the user performs a second input on the first interface.

Abstract: A system can migrate a source file system on a first computer to a destination file system on a second computer. The system can receive, at the second computer, a request from a client computer that indicates a file system operation, and that references a source file handle of the source file system that comprises an identifier of the source file system. The system can identify the destination file system based on the identifier of the source file system. The system can determine, from a source inode number of the source file handle, a source inode number segment. The system can determine, from the source inode number segment, a destination inode number segment. The system can determine, from the destination inode number segment, a destination inode number. The system can perform the file system operation on the destination file system using the destination inode number and the identifier of the destination file system.

Abstract: Systems and methods for reducing usage of computational resources when accessing siloed data across disparate locations via a unified metadata graph are disclosed. The system receives a user-specified query indicating a request to access a set of data objects. The system then performs natural language processing on the user-specified query to determine a set of phrases corresponding to the user-specified query. The system then accesses a metadata graph to determine a node corresponding to the set of phrases. Using a location identifier corresponding to the determined node, the system determines a data silo storing at least one data object of the set of data objects. The system then generates for display, on a graphical user interface, a visual representation of the at least one data object.

Abstract: A peer-to-peer global namespace for storage system metadata federations. Specifically, the disclosure proposes a peer-to-peer metadata federation architecture, where enterprise storage metadata for any and all geo-distributed sites may be distributed across and continually reconciled by a network of nodes, instead of being maintained by a single node assigned to a given geographic location. In implementing a federation, the network of nodes may lack exploitable, centralized points of vulnerability, and therefore, may exhibit no central point of failure. Furthermore, the disclosure proposes the implementation of a global metadata namespace for the enterprise storage metadata spanning the entirety of an enterprise (across all geographic locations). The global metadata namespace may be maintained on each node through event-based synchronization.

Abstract: A system and a method for locating application-specific data that has been previously deleted and located in an address of the data storage device marked as being available for storing new data. The method includes accessing unidentified data from at least one data storage device; examining the unidentified data to detect at least one application-specific data pattern associated with at least one application; for each detected application-specific data pattern, executing an application-specific validation process to determine whether the unidentified data includes valid data associated with a corresponding application; and if it is determined that the unidentified data includes valid data associated with the corresponding application, then recovering the valid data.

Abstract: The field of high-speed data acquisition and network data processing, and particularly relates to an Ethernet data stream recording method, an Ethernet data stream recording system, and an Ethernet data stream recording device for a high-speed data acquisition system. It is intended to solve problems such as a low utilization rate of CPU, poor system compatibility, difficulty in packaging and deployment and low reliability of system transmission of the traditional high-speed data acquisition system.

Abstract: Examples described herein include virtualized environments including a virtualized file server. Examples of file server virtual machine managers are described which may setup and/or configure file server virtual machines in accordance with a target number of connections and/or a target throughput for the virtualized file server. The virtualized file server may be scaled up, scaled down, scaled in, and/or scaled out.

Abstract: Aspects and features include using a virtual disk image to improve computational performance when applying a software patch. Compressed extents within a stored disk image are detected. The compressed extents are virtually reordered to form compressed forks within a virtual disk image and the compressed forks are selected for decompression based on code to be patched. A decompressed fork with the patch is virtually written to the same or another virtual disk image as an updated fork, and the virtual disk image is used to write to storage, either to overwrite the same stored disk image or to produce an updated, compressed disk image. In some examples, the virtual disk image is validated prior to writing to the compressed image by comparing an output hash from the compressed disk image with a known hash to validate the virtual disk image.

Abstract: A device configured to identify a first link between a value of a first data element in a first plurality of data elements and values of a first set of data elements in a second plurality of data elements and to remove the first link between the first data element and the first set of data elements. The device is further configured to input the data elements into a machine learning model that is configured to output a second link between the first data element and a second set of data elements. The device is further configured to create an entry in a relationship table that identifies the first data element and the second set of data elements. The device is further configured to generate a data stream with the first data element and the second set of data elements and to output the data stream.

Abstract: Examples described herein include virtualized environments including a virtualized file server. Examples of file server virtual machine managers are described which may setup and/or configure file server virtual machines in accordance with a target number of connections and/or a target throughput for the virtualized file server. The virtualized file server may be scaled up, scaled down, scaled in, and/or scaled out.

Abstract: Cryptographic information is compiled by: (a) determining a first portion of the cryptographic information based on an input and a randomness; (b) checking a rejection criterion based on the first portion; (b1) re-starting step (a) with a different randomness if the rejection criterion is fulfilled; (b2) if not all portions of the cryptographic information have been generated, determining a subsequent portion of the cryptographic information based on the input and the randomness and continuing with step (b) or, otherwise, continuing with step (c); (c) determining the first portion of the cryptographic information based on the input and the randomness; (d) conveying the respective portion of the cryptographic information; and (e) if not all portions of the cryptographic information have been generated, determining a subsequent portion of the cryptographic information based on the input and the randomness and continuing with step (d).

Abstract: Accelerating artificial intelligence workflows, including: receiving, from a computing process of an artificial intelligence workflow, a request for information stored on a data repository; issuing, from a user space of an operating system environment, parallel requests to the data repository using a network protocol that operates serially at the kernel level of the operating system environment; receiving, from the data repository, one or more responses to the parallel requests; and providing, to the computing process of the artificial intelligence workflow and based on the one or more responses to the parallel requests, a response to the request for information.

Abstract: Systems and methods for sharing information between distributed computer systems connected to one or more data networks. In particular, a replication system implements methodologies for sharing database information between computer systems where the databases use different classification schemes for information access control.

Abstract: A system can determine that a first destination inode number of a first destination inode of destination inodes is equal to a first source inode number of a file. The system can create a second file on the source file system, wherein the second file is stored in a second source inode of the source inodes having a second source inode number of the source inode numbers. The system can determine that the destination file system lacks an inode that has the second source inode number. The system can move a third file stored in the first destination inode to a second destination inode of the destination inodes that has a second destination inode number of the destination inode numbers. The system can migrate the first file to the destination file system, wherein the destination file system stores the first file with the first destination inode number.

Abstract: Method and apparatus creating test environments for blockchain systems involving recreating system configurations and state database without block data to set up staging environments for blockchain systems. The example implementations involve a function getting the latest config block and database files of state database from a production environment. Further, there is a function extracting system configurations from the received latest config block, replacing root certifying authority certificate(s) in the system configurations with the certificate(s) for the staging environment, creating a system configuration transaction and sending the system configuration transaction to the staging environment to reflect the system configurations. Further, there is a function extracting a list of states from database files, creating a transaction request message having the list of states and sending a transaction to the staging environment to load the states into the state database.

Abstract: A system (100) for managing data is provided. The system includes at least one processor (102) configured to convert a source dataset (302) to a root dataset (312) by obtaining a plurality of intermediary datasets (306, 308). The root dataset (312) is shorter compared to the source dataset (302) and the intermediary datasets (306, 308) are intermediate to the source dataset (302) and the root dataset (312). At least one of the intermediary datasets comprises reference bits, wherein the reference bits enable identification of one or more functions that should be used to obtain a dataset from which the intermediary dataset comprising the reference bits was obtained in the process of converting the source dataset (302) and the root dataset (312).

Abstract: A computer-implemented method for managing updates to a knowledge graph with incoming change requests is disclosed. The method comprises creating a snapshot of the knowledge graph, building sub-regions of the knowledge graph, classifying each incoming change to the knowledge graph so that it relates to one of the sub-regions, creating at least one change layer, wherein each of the at least one change layer relates to one of the sub-regions, wherein each of the at least one change layer comprises a portion of the incoming change requests, closing the at least one change layer upon reaching a predetermined threshold value for the at least one change layer, and integrating the at least one change layer into the knowledge graph, thereby generating a updated knowledge graph.

Abstract: A distributed cloud-based storage system, where the distributed cloud-based storage system includes: receiving, by one or more storage controller applications of the cloud-based storage system, one or more storage operations; storing, among one or more cloud computing instances of the cloud-based storage system, the one or more storage operations; and distributing, among one or more cloud computing instances within respective one or more cloud computing environments within distinct geographic regions, one or more of the one or more storage operations.

Abstract: An illustrative method includes a storage management system ingesting a data item into a unified storage system via a storage system interface among a plurality of storage system interfaces associated with the unified storage system, determining, in response to the ingesting of the data item into the unified storage system, an operation based on the data item, and providing a notification of the operation to an orchestration system configured to manage an execution of the operation by a computing system associated with the unified storage system. In certain embodiments, the unified storage system may be implemented as compute-aware storage system such as a container orchestrator-aware storage system.

Abstract: A multivariate time series model such as a Vector Auto Regression (VAR) model is built using fabric utilization, disk utilization, and CPU utilization time series data. The VAR model leverages interdependencies between multiple time-dependent variables to predict the start and length of an aperiodic backup time window, and to cause backup operations to occur during the aperiodic backup time window to thereby exploit the aperiodic backup time window for use in connection with backup operations. By automatically starting backup operations during predicted aperiodic backup time windows where the CPU, disk, and fabric utilization values are predicted to be low, it is possible to implement backup operations during time windows where the backup operations are less likely to interfere with primary application workloads, or system application workloads that need to be implemented to maintain optimal operation of the storage system.

Abstract: A method for supporting files names in a storage system, the method may include receiving a first protocol request for creating an object having a requested object name that comprises one or more sets of slashes, wherein the first protocol does not use the one or more sets as hierarchy separators; wherein each set of slashes comprises one or more consecutive slashes; determining a role of each set of the one or more sets in a second protocol that uses slashes as hierarchy separators; and generating a path name for accessing the object, when using either one of the first and second protocols, wherein the generating is based on the requested object name and the role of each set.

Abstract: Disclosed are system and methods for processing and storing data files, using a data edge file format. The data edge file format separates information about what symbols are in a data file and information about the corresponding location of those symbols in the data file. Examples convert a source file comprising symbols into a data edge index having a manifest portion, a symbol portion, and a locality portion. The symbol portion contains a sorted unique set of symbols from the source file, and the locality portion contains a plurality of location values referencing the symbol portion. Examples include normalizing structured data from the source file by modifying the locality manifest portion of the data edge file to include a description of at least one nonexistent column empty locality value at a respective position within the locality file representing an omission of data at an associated position in the source file.

Abstract: A method for the reverse deletion of a plurality of snapshots in a chain of snapshots is provided. The method includes in reverse order, starting from a latest snapshot in time to an earliest snapshot in time of the plurality of snapshots: identifying at least one of a first set of one or more data blocks of a snapshot that are shared with an earlier snapshot in time in the chain of snapshots or a second set of one or more data blocks of the snapshot that are owned by the snapshot and processing the second set of one or more data blocks and skipping processing the first set of one or more data blocks, wherein processing the second set of one or more data blocks comprises performing one or more actions to maintain and/or delete data blocks of the second set of one or more data blocks.

Abstract: Techniques are disclosed relating to managing distributed storage of data for various entities according to classifications for these entities. A database node of a distributed storage system may receive, from a first entity of a plurality of entities, a request to store a set of data. The database node may further obtain metadata associated with the first entity, wherein the metadata specifies one of a plurality of classifications for the entities. The database node may provide the set of data to one or more of a plurality of caches for storage. The caches may be located in two or more availability zones and are configured to store the set of data based on the classification for the first entity identified in the metadata associated with the first entity. The database node may also store the set of data in a shared object storage coupled to the database node.

Abstract: A system (100) for managing data is provided. The system includes at least one processor (102) configured to convert a source dataset (302) to a root dataset (312) by obtaining a plurality of intermediary datasets (306, 308). The root dataset (312) is shorter compared to the source dataset (302) and the intermediary datasets (306, 308) are intermediate to the source dataset (302) and the root dataset (312). At least one of the intermediary datasets comprises reference bits, wherein the reference bits enable identification of one or more functions that should be used to obtain a dataset from which the intermediary dataset comprising the reference bits was obtained in the process of converting the source dataset (302) and the root dataset (312).

Abstract: Embodiments of the present invention provide methods, systems, apparatuses, and computer program products for predicting network asset requests for a future network time interval.

Abstract: A system, apparatus, and method for processing queries wherein the query includes a request to access or delete data and accessing metadata associated with the set of data, the metadata defining data characteristics of the set of data and identifying at least sets of data that need or not need to be accessed or deleted based on the metadata without accessing the actual data in the set of data; also methods to optimize processing of some operations based on the collected metadata on data.

Abstract: Apparatus and methods for managing provision of digital content within a content distribution network. In one embodiment, a content application manager (CAM) process is used to provide one or more centralized redirection functions for network client processes such as just-in-time (JIT) packagers and cloud digital video recorders (cDVRs). A local database accessible to the CAM is updated with “live” or other content storage locations within the network, and accordingly each of the client processes can be made simpler and not require any intrinsic knowledge of content asset storage locations. This simplifies the client process configuration, enhances performance in terms of reduced end-user request servicing latency, and allows for single-point automated database updates versus having to provide updates to each client process as under the prior art.

Abstract: The retrieval of files can be facilitated in real-time in the general context of backup and recovery. A processor can receive a request to retrieve a particular volume of data from an object storage database. A virtual volume can be presented to an emulator for creation of a virtual machine representing the particular volume of data. A request can be received to access a particular file from the particular volume of data at a specific point in time. In response to the request to access the file, the data stream can be paused, the particular file can be fetched, and the particular file can be transmitted to the emulator.

Abstract: The present disclosure provides systems, methods, and non-transitory computer-readable storage media for determining container to leaf switch connectivity information in a data center in a presence of blade switches and servers. In one aspect of the present disclosure, a method of determining container to leaf switch connectivity information of a data center utilizing at least one blade switch and at least one blade server, includes receiving, at a network controller, link connectivity information that includes south-bound neighboring information between the at least one blade switch of the data center and the at least one blade server of the data center; determining, at the network controller, the container to leaf switch connectivity information of the data center, based on the link connectivity information; and generating a visual representation of a topology of the data center based on the container to leaf switch connectivity information.

Abstract: Techniques for accessing a file involve determining whether a client requests a permission for a target file, the permission allowing the client to cache data associated with the target file. The techniques further involve in response to determining that the client requests the permission, obtaining pattern information related to an access pattern in which the client accesses the target file. The techniques further involve determining availability of the permission to the client by applying the pattern information to a decision model, the decision model being trained based on training pattern information and training permission information. The techniques further involve providing, to the client, an indication on the availability. Accordingly, access conflicts can be reduced, so that the performance of the client and server can be improved.