Abstract: Data associated with a source system is ingested. After the data is ingested, a post-processing metadata conversion process is performed including by selecting an entry of a chunk metadata data structure and determining that a data chunk associated with the selected entry is not referenced by at least a threshold number of objects. In response to determining that the data chunk associated with the selected entry is not referenced by at least the threshold number of objects, metadata of a tree data structure node corresponding to a chunk identifier associated with the data chunk is updated to store a reference to a chunk file storing the data chunk and the selected entry is removed from the chunk metadata data structure.

Abstract: A repository of replicated chunk files is analyzed to identify chunk files that meet at least a portion of combination criteria. Selected chunk files are associated together under a data protection grouping container. Erasure coding is applied to the data protection grouping container including by utilizing the selected chunk files as different data stripes of the erasure coding and generating one or more parity stripes based on the different data stripes.

Abstract: Events from one or more primary systems associated with one or more tenants are received. The received events are stored in a message queue. At least a portion of the events in the message queue are ingested for organization and storage in a data store. One or more progress identifiers associated with ingesting of the events in the message queue are tracked. An event query is received from an external system. The event query is rewritten into a first component query for the data store and a second component query for the message queue based at least in part on a progress identifier. A result of the first component query and a result of the second component query are combined to determine a result of the event query.

Abstract: In general, techniques for efficient data retrieval from a backup system are described. An example computing system includes one or more storage devices and processing circuitry having access to the one or more storage devices and configured to: process an input to generate a filter, wherein the input indicates a context for one or more queries; apply the filter to backup data to obtain filtered data from the backup data; generate an index of embeddings from the filtered data; process, based on the index of embeddings, a query to generate a response for the query; and output the response.

Abstract: Data associated with an object to be stored is received from a source system for a destination logical storage container selected among a plurality of destination logical storage containers. A total number of the destination logical storage containers allowed to be concurrently supported by the source system is limited. The selected destination logical storage container is shared by a plurality of objects of the source system. Based at least in part on an identifier associated with the object, a child logical storage container corresponding to the object is identified. The child logical storage container is different from the selected destination logical storage container. The data associated with the object received for the selected destination logical storage container is automatically stored in the identified child logical storage container.

Abstract: A plurality of objects sharing one or more common attributes are identified. A storage distribution mode for the identified objects sharing the one or more common attributes is determined based at least in part on one or more optimization criteria. The storage distribution mode is caused to be implemented by one or more of a plurality of storage clusters.

Abstract: Described are techniques for oblique image rectification. The techniques include receiving an original image depicting an oblique view of a circular object and pre-processing the original image into an edge image. The techniques further include generating, by a machine learning model based on the edge image, a heatmap including an ellipse formed by the oblique view of the circular object. The techniques further include computing ellipse parameters describing the ellipse of the heatmap. The techniques further include performing, using the ellipse parameters, an affine transformation on the original image to generate a rectified image, where the rectified image converts the ellipse to a circle.

Abstract: A plurality of data stripes and one or more parity stripes are generated using a plurality of data chunks stored in a write-ahead log based on an erasure coding configuration. The plurality of data stripes and the one or more parity stripes are stored on corresponding different storage devices. The plurality of data stripes and the one or more parity stripes are associated together under a data protection grouping container.

Abstract: Application-level data in a storage system are evaluated. For example, a backup analysis tool retrieves a backup object stored in the storage system. The backup analysis tool reconstructs an application object from the backup object. The backup analysis tool accesses the first application object according to a native application format associated with the first object.

Abstract: A plurality of logical storage segments of storage drives of a plurality of storage nodes are identified. At least one of the storage nodes includes at least a first logical storage segment and a second logical storage segment included in the plurality of logical storage segments. A distributed and replicated data store using a portion of the plurality of logical storage segments that excludes at least the second logical storage segment is provided. An available storage capacity metric associated with the plurality of logical storage segments is determined to meet a first threshold. In response to the determination that the available storage capacity metric meets the first threshold, at least the second logical storage segment is dynamically deployed for use in providing the distributed and replicated data store in a manner that increases a storage capacity of the data store while maintaining a fault tolerance policy of the distributed and replicated data store.

Abstract: An amount of data change associated with a version of a content file with respect to one or more previous versions of the content file is determined. The amount of change associated with the version of the content file is determined using a tree data structure associated with the content file that is stored on a storage cluster. One or more statistics associated with backup snapshot are provided to a server. The server is configured to determine that the amount of data change associated with the version of the content file is anomalous based in part on the one or more statistics associated with the backup snapshot. A notification that data associated with the backup snapshot is potentially infected by malicious software is received from the server. The version of the content file is indicated as being potentially infected by malicious software.

Abstract: Described are techniques for oblique image rectification. The techniques include receiving an original image depicting an oblique view of a circular object and pre-processing the original image into an edge image. The techniques further include generating, by a machine learning model based on the edge image, a heatmap including an ellipse formed by the oblique view of the circular object. The techniques further include computing ellipse parameters describing the ellipse of the heatmap. The techniques further include performing, using the ellipse parameters, an affine transformation on the original image to generate a rectified image, where the rectified image converts the ellipse to a circle.

Abstract: A mechanism for storing indexing updates is described. A secondary storage system is configured to store a backup of a primary storage system. A plurality of different indexing updates of a data repository associated with the secondary storage system is received. An indexing update may include metadata and/or updates to the metadata for a data item in the backup. The indexing updates may be obtained from an indexing process. At least a portion of the indexing updates are sent for storage in an intermediate store. An indication to commit the indexing updates is received. In response, the indexing updates including the at least the portion of the indexing updates are requested to be committed. The indexing updates batched together into a batch to be committed to a metadata store.

Abstract: An identification of a new primary snapshot created for a primary storage system is received. A change tracking time window that is at least a portion of a period between a first capture time associated with a previous primary snapshot and a second capture time associated with the new primary snapshot is determined. Entries of a storage log of the primary storage system occurring within the change tracking time window are analyzed to coalesce changes identified in the entries of the storage log occurring within the change tracking time window into a change tracking result set. The change tracking result set is used to identify at least a portion of data changes between the previous primary snapshot and the new primary snapshot to capture in a new backup snapshot stored at a secondary storage system.

Abstract: Automated analog gauge reading is provided. The method comprises a computer system receiving input of an image and detecting at least one analog gauge in the image. The computer system corrects the orientation of the analog gauge in the image and detects scene text and tick labels on the analog gauge. The computer system determines a position of a pointer on the analog gauge relative to the scene text and outputs a gauge reading value based on an arithmetic progression of tick labels and angle of the pointer with respect to minimum and maximum values on the analog gauge.

Abstract: A size associated with a first content file is determined to be less than a threshold size. In response to determining that the size associated with the first content file is less than the threshold size, a combined metadata structure is updated at least in part by combining metadata of the first content file with metadata of a second content file in the combined metadata structure. A snapshot tree is updated to reference a first portion of the combined metadata structure corresponding to the first content file and to reference a second portion of the combined metadata structure corresponding to the second content file.

Abstract: An identification of a new primary snapshot created for a primary storage system is received. A threshold time window based on a capture time associated with the new primary snapshot is determined. Entries of a storage log of the primary storage system occurring within the threshold time window are analyzed to identify any objects of the primary storage system that have changed during the threshold time window. Changed objects to capture in a new secondary backup stored at a secondary storage system and corresponding to the new primary snapshot are identified. Metadata of the new primary snapshot and metadata of a previous secondary backup are compared to determine for each of the objects of the primary storage system identified as having changed during the threshold time window whether a change to an object since the previous secondary backup is captured in the new primary snapshot.

Abstract: A backup of one or more objects is determined to be performed. Based on one or more conditions, a corresponding deduplication option among a plurality of deduplication options to utilize when backing up the one or more objects is selected. The one or more conditions at least include a condition based on a detected data change pattern. The plurality of deduplication options include a deduplication option associated with utilizing at least in part a plurality of variable-length data chunks for one or more mismatched ranges and/or one or more missing ranges associated with one of the one or more objects associated with the source system. A request to perform the backup of the one or more objects according to the corresponding selected deduplication option is provided to the source system. Backup data associated with the one or more objects is received and stored.

Abstract: Data associated with a source system is ingested. After the data is ingested, a post-processing metadata conversion process is performed including by selecting an entry of a chunk metadata data structure and determining that a data chunk associated with the selected entry is not referenced by at least a threshold number of objects. In response to determining that the data chunk associated with the selected entry is not referenced by at least the threshold number of objects, metadata of a tree data structure node corresponding to a chunk identifier associated with the data chunk is updated to store a reference to a chunk file storing the data chunk and the selected entry is removed from the chunk metadata data structure.

Abstract: Range information associated with one or more objects is received from a storage system. One or more missing ranges and/or one or more mismatched ranges associated with the one or more objects is determined based on the received range information. A plurality of data chunk identifiers associated with a plurality of variable-length data chunks included in the one or more determined ranges associated with the one or more objects is provided to the storage system. A response that is used to identify among the plurality of variable-length data chunks, one or more variable-length data chunks not already stored in a storage associated with the storage system is received from the storage system. Content of the identified one or more variable-length data chunks is provided to the storage system.