Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.
Abstract: Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.
Abstract: A facility for managing filesystem object storage quotas (i.e., size limits) in a storage environment is disclosed. The facility enables users to establish, modify, and remove quotas on directories and files within a filesystem. Each quota acts as a soft limit on the size of the associated filesystem object, including any child objects of the filesystem object. The facility improves the speed at which the system can test for and identify violations of quotas established for individual filesystem objects by using aggregation and reconciliation techniques rather than constantly traversing a filesystem in its entirety to test for violations of quotas.
Type:
Grant
Filed:
March 30, 2017
Date of Patent:
October 9, 2018
Assignee:
Qumulo, Inc.
Inventors:
Philip Taron, Patrick Jakubowski, Remi Bernotavicius
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as checksums, MIN and/or MAX aggregates, sameness bits, bloom filter aggregates, queryable user tags, moving average aggregates, b-tree aggregates, etc.
Type:
Grant
Filed:
September 18, 2015
Date of Patent:
September 28, 2021
Assignee:
Qumulo, Inc.
Inventors:
Aaron Passey, Kevin O'Neill, Peter Godman, Iain Peet
Abstract: A facility for managing filesystem object storage quotas (i.e., size limits) in a storage environment is disclosed. The facility enables users to establish, modify, and remove quotas on directories and files within a filesystem. Each quota acts as a soft limit on the size of the associated filesystem object, including any child objects of the filesystem object. The facility improves the speed at which the system can test for and identify violations of quotas established for individual filesystem objects by using aggregation and reconciliation techniques rather than constantly traversing a filesystem in its entirety to test for violations of quotas.
Type:
Grant
Filed:
October 4, 2018
Date of Patent:
February 22, 2022
Assignee:
Qumulo, Inc.
Inventors:
Philip Taron, Patrick Jakubowski, Remi Bernotavicius
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in inode date of each directory within a tree. The stored metric values represent summed or otherwise aggregated data from the descendant directories and files. The metric values are utilized to provide representative compositional samplings of data in the filesystem tree structure. Based on the metric value in its inode data, each directory is weighted in a manner that determines the likelihood that each file sample is taken from that directory.
Type:
Grant
Filed:
January 13, 2015
Date of Patent:
October 4, 2022
Assignee:
Qumulo, Inc.
Inventors:
Peter Godman, Kevin O'Neill, Thomas Urban, Graeme Williams
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as total space consumed by a directory and all the descendant directories and files, total number of files within a directory, total data blocks used by a directory and its descendant directories and data files, etc.
Type:
Grant
Filed:
January 12, 2015
Date of Patent:
October 29, 2019
Assignee:
Qumulo, Inc.
Inventors:
Peter Godman, Kevin O'Neill, Thomas Urban, Justin Husted, Graeme Williams
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as checksums, MIN and/or MAX aggregates, sameness bits, bloom filter aggregates, queryable user tags, moving average aggregates, b-tree aggregates, etc. The facility provides mechanisms for efficiently sampling activities performed by the filesystem and visually representing these activities.
Type:
Grant
Filed:
October 5, 2018
Date of Patent:
December 29, 2020
Assignee:
Qumulo, Inc.
Inventors:
Zachary B. Okun, Eric J. Wright, Daniel J. Pirone, Thomas R. Unger, Alixandra M. Han
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as checksums, MIN and/or MAX aggregates, sameness bits, bloom filter aggregates, queryable user tags, moving average aggregates, b-tree aggregates, etc. The facility provides mechanisms for efficiently sampling activities performed by the filesystem and visually representing these activities.
Type:
Grant
Filed:
December 4, 2017
Date of Patent:
October 9, 2018
Assignee:
Qumulo, Inc.
Inventors:
Zachary B. Okun, Eric J. Wright, Daniel J. Pirone, Thomas R. Unger, Alixandra M. Han
Abstract: Embodiments are direct to managing file systems. Alternate data streams (ADSs) of an object in a file system may be modified based on a request from a client and metrics associated with the ADSs may be modified. Values for reconcile fields associated with the object may be set based on the modified metrics such that the object with the modified ADSs may be included in a reconcile list. And, such that an acknowledgment of the request may be provided to the client. A reconcile engine may determine a parent object of the object. The object may be reconciled and the parent object may be placed on the reconcile list for reconciling in turn.
Type:
Grant
Filed:
May 26, 2020
Date of Patent:
March 2, 2021
Assignee:
Qumulo, Inc.
Inventors:
Thomas R. Unger, Michael Patrick Kirby, Thomas Scott Urban, Michael R. Fitz, Boyu Zhang
Abstract: A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as checksums, MIN and/or MAX aggregates, sameness bits, bloom filter aggregates, queryable user tags, moving average aggregates, b-tree aggregates, etc. The facility provides mechanisms for efficiently sampling activities performed by the filesystem and visually representing these activities.
Type:
Grant
Filed:
September 18, 2015
Date of Patent:
December 5, 2017
Assignee:
Qumulo, Inc.
Inventors:
Zachary B. Okun, Eric J. Wright, Daniel J. Pirone, Thomas R. Unger, Alixandra M. Han
Abstract: Embodiments are directed to managing data in a file system. A file system engine provides a file system that includes file system objects that may be accessible by two or more clients that use different native permission schemes. And, a permissions engine may determine map rules based on characteristics of a request to access request a file. The permission engine may execute the one or more map rules to provide platform permission values based on the native permission values. The permission engine may compare requested platform permission values to the platform permission values associated with the file. The permission engine may provide the access rights to the file based on an affirmative result of the comparison.
Type:
Grant
Filed:
December 27, 2018
Date of Patent:
October 29, 2019
Assignee:
Qumulo, Inc.
Inventors:
Michael Patrick Kirby, Iain Michael Christopher Peet, Garrett Mathew Nickel, Neal Thomas Fachan
Abstract: Embodiments are directed to file systems. A replication engine may establish a secure communication channel between a source file system and a target file system. The replication engine may: instantiate a replication job associated with rules; determine changes in the source file system; determine characteristics of the replication job that may be based on the changes; compare the to the characteristics and a black-out schedule; execute the replication job to communicate the changes in the source file system to the target file system based on a result of the comparison. Upon completion of the replication job, the replication engine may automatically instantiating a next replication job to copy subsequent changes in the source file system to the target file system.
Type:
Grant
Filed:
April 30, 2018
Date of Patent:
September 11, 2018
Assignee:
Qumulo, Inc.
Inventors:
Ezra Elias Kilty Cooper, Neal Thomas Fachan, Junjie Liang, Kevin Ross O'Neill
Abstract: A facility comprising systems and method for automatically triggering the collection of comprehensive monitoring information in a distributed processing system. The facility compares the overall performance of distributed processing system to one or more performance metrics and, in response to determining that one or more performance metrics is not satisfied, triggers one or more of the nodes within the distributed processing system to increase one or more of its monitoring rate or its distribution rate. The facility collects and analyzes the collected information to provide resources that can be used to assess and diagnose failures within the distributed processing system. In this manner, the facility reacts to performance anomalies by triggering nodes within in the system to provide comprehensive performance information over a trigger period for diagnostic purposes.
Type:
Grant
Filed:
April 19, 2018
Date of Patent:
June 11, 2019
Assignee:
Qumulo, Inc.
Inventors:
Thomas Rothschilds, Remi Bernotavicius, Edward Brow, William Ehlhardt
Abstract: A file system with distributed resource caching that includes cache volumes and agents that may be associated with clients of the file system may be provided. A cache allocation for each agent may be determined based on a capacity of the cache volumes and a number of the agents such that each cache allocation is associated with tokens that each represents a reserved portion of free space in the cache volumes. Storage jobs may be provided to the agents. Data associated with the storage jobs may be stored in the cache volumes. The cache allocation for each agent may be reduced based on the data stored for each agent.
Type:
Grant
Filed:
January 28, 2022
Date of Patent:
June 6, 2023
Assignee:
Qumulo, Inc.
Inventors:
Conner Saltiel Hansen, Patrick Jakubowski, David Patrick Rogers, III, Thomas Gregory Rothschilds, Porter Michael Smith, Hanqing Zhang
Abstract: A facility comprising systems and method for automatically triggering the collection of comprehensive monitoring information in a distributed processing system. The facility compares the overall performance of distributed processing system to one or more performance metrics and, in response to determining that one or more performance metrics is not satisfied, triggers one or more of the nodes within the distributed processing system to increase one or more of its monitoring rate or its distribution rate. The facility collects and analyzes the collected information to provide resources that can be used to assess and diagnose failures within the distributed processing system. In this manner, the facility reacts to performance anomalies by triggering nodes within in the system to provide comprehensive performance information over a trigger period for diagnostic purposes.
Type:
Grant
Filed:
June 6, 2019
Date of Patent:
June 9, 2020
Assignee:
Qumulo, Inc.
Inventors:
Thomas Rothschilds, Remi Bernotavicius, Edward Brow, William Ehlhardt
Abstract: A facility for snapshot space accounting for a storage system, such as a filesystem is disclosed. The facility enables users to quickly and easily determine the amount of storage space that would be released or recovered if a snapshot were to be purged. The facility may work in conjunction with, or as part of, a snapshot service. The facility maintains an expiration data structure and a count data structure and uses these data structures in implementing the disclosed snapshot space accounting techniques. The expiration data structure represents the life cycle of each snapshot element maintained by the facility while the count data structure represents, for pairs of snapshots, the size of the information stored in the snapshot data that expired and that spans the corresponding pair of snapshots.
Type:
Grant
Filed:
March 29, 2017
Date of Patent:
September 5, 2017
Assignee:
Qumulo, Inc.
Inventors:
Edward Brow, Nicholas Wilson, Robert Gebert, Thomas Scott Urban, Michael Adam Krasnow, Steven Haber
Abstract: Embodiments are directed to file systems. A replication engine may establish a secure communication channel between a source file system and a target file system. The replication engine may: instantiate a replication job associated with rules; determine changes in the source file system; determine characteristics of the replication job that may be based on the changes; compare the to the characteristics and a black-out schedule; execute the replication job to communicate the changes in the source file system to the target file system based on a result of the comparison. Upon completion of the replication job, the replication engine may automatically instantiating a next replication job to copy subsequent changes in the source file system to the target file system.
Type:
Grant
Filed:
September 7, 2018
Date of Patent:
March 31, 2020
Assignee:
Qumulo, Inc.
Inventors:
Ezra Elias Kilty Cooper, Neal Thomas Fachan, Junjie Liang, Kevin Ross O'Neill, Michael Anthony Chmiel
Abstract: Managing data in a file system with a verification engine that may obtain a user identifier associated with a user, an object identifier, and a target object. The verification engine may determine target identities associated with two or more file system protocols based on the user identifier. The verification engine may determine permission entries associated with the two or more file system protocols associated and the target object. The verification engine may employ the target identities and the permission entries to directly verify access rights to the target object for the user. Accordingly, the verification engine may provide a report that includes the target identities, the permission entries, or the access rights.
Type:
Grant
Filed:
December 27, 2018
Date of Patent:
April 7, 2020
Assignee:
Qumulo, Inc.
Inventors:
Michael Patrick Kirby, Iain Michael Christopher Peet, Anthony Moon-Git Low, Aleksei Martynov, Jason Allan Sturgeon
Abstract: A facility for snapshot space accounting for a storage system, such as a filesystem is disclosed. The facility enables users to quickly and easily determine the amount of storage space that would be released or recovered if a snapshot were to be purged. The facility may work in conjunction with, or as part of, a snapshot service. The facility maintains an expiration data structure and a count data structure and uses these data structures in implementing the disclosed snapshot space accounting techniques. The expiration data structure represents the life cycle of each snapshot element maintained by the facility while the count data structure represents, for pairs of snapshots, the size of the information stored in the snapshot data that expired and that spans the corresponding pair of snapshots.
Type:
Grant
Filed:
September 1, 2017
Date of Patent:
February 4, 2020
Assignee:
Qumulo, Inc.
Inventors:
Edward Brow, Nicholas Wilson, Robert Gebert, Thomas Scott Urban, Michael Adam Krasnow, Steven Haber