Abstract: In one embodiment, a system for managing a virtualization environment includes a set of host machines, each of which includes a hypervisor, virtual machines, and a virtual machine controller, and a data migration system configured to identify one or more existing storage items stored at one or more existing File Server Virtual Machines (FSVMs) of an existing virtualized file server (VFS). For each of the existing storage items, the data migration system is configured to identify a new FSVMs of a new VFS based on the existing FSVM, send a representation of the storage item from the existing FSVM to the new FSVM, such that representations of storage items are sent between different pairs of FSVMs in parallel, and store a new storage item at the new FSVM, such that the new storage item is based on the representation of the existing storage item received by the new FSVM.

Abstract: In one embodiment, a system for managing communication connections in a virtualization environment includes a plurality of host machines implementing a virtualization environment, wherein each of the host machines includes a hypervisor, at least one user virtual machine (user VM), and a distributed file server that includes file server virtual machines (FSVMs) and associated local storage devices. Each FSVM and associated local storage device are local to a corresponding one of the host machines, and the FSVMs conduct I/O transactions with their associated local storage devices based on I/O requests received from the user VMs. Each of the user VMs on each host machine sends each of its representative I/O requests to an FSVM that is selected by one or more of the FSVMs for each I/O request based on a lookup table that maps a storage item referenced by the I/O request to the selected one of the FSVMs.

Abstract: A data dictionary generation system utilizes a background service that is programmed to automatically populate and update a data dictionary for listings offering shared data. A data dictionary includes metadata describing the shared data overall as well as the individual objects included in the listing, such as the individual tables, schemas, views, and functions. To generate the data dictionary, the data dictionary generation system analyzes the shared data to identify objects, identifies a set of data fields associated with each identified object and populates the set of data fields associated with each identified object based on the shared data offered by the listing. To ensure that a data dictionary for each listing remains up to date, the data dictionary generation system periodically scans the listings to identify any changes to share access granted to the listings.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. A data dictionary includes metadata describing the shared data, including the individual objects, such as the individual tables, schemas, views, and functions. The shared data and each individual data object may be described in the data dictionary by a set of data fields that corresponds to the shared dataset or the object type of the individual object. The data dictionary can be presented to data consumers along with the description of the listing to provide data consumers with a comprehensive description of the shared data provided by a listing, including a high-level summary of the shared data and description of each individual object included in the shared data. The data dictionary allows data consumers to understand the contents of the shared data and how to use the shared data.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

Abstract: In one embodiment, a system for managing a virtualization environment includes a set of host machines, each of which includes a hypervisor, virtual machines, and a virtual machine controller, and a virtualized file server backup system configured to identify backup data, wherein the backup data comprises data stored on the virtual disks and VFS configuration information, and the first data is identified in accordance with a backup policy, send the backup data to one or more remote sites for storage, and, in response to detection of changes in the backup data, send the changes to the remote sites in accordance with a replication policy. The backup data may be identified based on a protection domain associated with the backup policy. The data stored on the VFS may include one or more storage objects. The storage objects may include shares, groups of shares, files, or directories.

Abstract: A process of fulfilling a database deployment request for a data platform. A compute service manager of the data platform scans one or more accounts of a consumer region of the data platform for a pending listing fulfillment request, where the pending listing request includes a request for deployment of a consumer database and an associated share of grant metadata of the consumer database within the consumer region. When the compute service manager determines that an account of the one or more accounts has a pending listing fulfillment request, the compute service manager determines a listing for the pending listing fulfillment request based on listing data of the account. The compute service manager determines that no other fulfillment task is scheduled for the pending listing fulfillment request and schedules a background fulfillment task to perform the fulfillment process for the pending listing fulfillment request.

Abstract: In one embodiment, a system for managing communication connections in a virtualization environment includes a plurality of host machines implementing a virtualization environment, wherein each of the host machines includes a hypervisor, at least one user virtual machine (user VM), and a distributed file server that includes file server virtual machines (FSVMs) and associated local storage devices. Each FSVM and associated local storage device are local to a corresponding one of the host machines, and the FSVMs conduct I/O transactions with their associated local storage devices based on I/O requests received from the user VMs. Each of the user VMs on each host machine sends each of its representative I/O requests to an FSVM that is selected by one or more of the FSVMs for each I/O request based on a lookup table that maps a storage item referenced by the I/O request to I/O the selected one of the FSVMs.

Abstract: A process of fulfilling a database deployment request for a data platform. A compute service manager of the data platform scans one or more accounts of a consumer region of the data platform for a pending listing fulfillment request, where the pending listing request includes a request for deployment of a consumer database and an associated share of grant metadata of the consumer database within the consumer region. When the compute service manager determines that an account of the one or more accounts has a pending listing fulfillment request, the compute service manager determines a listing for the pending listing fulfillment request based on listing data of the account. The compute service manager determines that no other fulfillment task is scheduled for the pending listing fulfillment request and schedules a background fulfillment task to perform the fulfillment process for the pending listing fulfillment request.

Abstract: A process of fulfilling a database deployment request for a data platform. A compute service manager of the data platform scans one or more accounts of a consumer region of the data platform for a pending listing fulfillment request, where the pending listing request includes a request for deployment of a consumer database and an associated share of grant metadata of the consumer database within the consumer region. When the compute service manager determines that an account of the one or more accounts has a pending listing fulfillment request, the compute service manager determines a listing for the pending listing fulfillment request based on listing data of the account. The compute service manager determines that no other fulfillment task is scheduled for the pending listing fulfillment request and schedules a background fulfillment task to perform the fulfillment process for the pending listing fulfillment request.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

Abstract: In one embodiment, a system for managing a virtualization environment includes host machines implementing a virtualization environment, a plurality of clusters of the host machines, a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), and a VFS cluster manager (CM) configured to distribute storage items among the clusters and receive cluster storage statistics for one or more shares of the VFS. The CM is further configured to, in response to a request from a first FSVM to identify a storage location for a storage item, identify a cluster at which the storage item is to be located based on the cluster storage statistics, identify a second FSVM at which the storage item is to be located based on compute usage statistics of one or more FSVMs in the identified cluster, and send an address of the second FSVM to the first FSVM.

Abstract: In one embodiment, a system for managing a virtualization environment comprises a plurality of host machines, one or more virtual disks comprising a plurality of storage devices, a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), wherein each of the FSVMs is running on one of the host machines and conducts I/O transactions with the one or more virtual disks, and a virtualized file server self-healing system configured to identify one or more corrupt units of stored data at one or more levels of a storage hierarchy associated with the storage devices, wherein the levels comprise one or more of file level, filesystem level, and storage level, and when data corruption is detected, cause each FSVM on which at least a portion of the unit of stored data is located to recover the unit of stored data.

Abstract: Examples of systems described herein include a virtualized file server including a first file server virtual machine and a second file server virtual machine configured to manage a distributed file share of storage items. The second file server virtual machine is configured to manage a particular storage item of the distributed file share of storage items. The first file server virtual machine is configured to, in response to receipt of a referral request for a file share path for the particular storage item from a client, look up a file share path for the particular storage item in a map of at least a portion of the distributed file share of storage items, and provide a referral with the file share path that identifies the second file server virtual machine.

Abstract: Examples of systems described herein include a file server virtual machine of a virtualized file server configured to manage storage of a plurality of storage items. The file server virtual machine including a file system configured to receive an access request directed to a storage item of the plurality of storage items and associated with a user. The file system is further configured to retrieve an access control list having permissions information associated with the storage item, and to cache a permissions profile for the user including all permissions pertaining to the user for the storage item. The file system is further configured to determine whether the access request is permissible based on the cached permissions profile.

Abstract: A data dictionary generation system utilizes a background service that is programmed to automatically populate and update a data dictionary for listings offering shared data. A data dictionary includes metadata describing the shared data overall as well as the individual objects included in the listing, such as the individual tables, schemas, views, and functions. To generate the data dictionary, the data dictionary generation system analyzes the shared data to identify objects, identifies a set of data fields associated with each identified object and populates the set of data fields associated with each identified object based on the shared data offered by the listing. To ensure that a data dictionary for each listing remains up to date, the data dictionary generation system periodically scans the listings to identify any changes to share access granted to the listings.

Abstract: In one embodiment, a system for managing a virtualization environment comprises a plurality of host machines, one or more virtual disks comprising a plurality of storage devices, a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), wherein each of the FSVMs is running on one of the host machines and conducts I/O transactions with the one or more virtual disks, and a virtualized file server self-healing system configured to identify one or more corrupt units of stored data at one or more levels of a storage hierarchy associated with the storage devices, wherein the levels comprise one or more of file level, filesystem level, and storage level, and when data corruption is detected, cause each FSVM on which at least a portion of the unit of stored data is located to recover the unit of stored data.

Abstract: A process of fulfilling a database deployment request for a data platform. A compute service manager of the data platform scans one or more accounts of a consumer region of the data platform for a pending listing fulfillment request, where the pending listing request includes a request for deployment of a consumer database and an associated share of grant metadata of the consumer database within the consumer region. When the compute service manager determines that an account of the one or more accounts has a pending listing fulfillment request, the compute service manager determines a listing for the pending listing fulfillment request based on listing data of the account. The compute service manager determines that no other fulfillment task is scheduled for the pending listing fulfillment request and schedules a background fulfillment task to perform the fulfillment process for the pending listing fulfillment request.

Abstract: A file server manager disclosed herein accesses information regarding a selected share of a source distributed file server for replication, where the selected share stores at least a portion of a namespace of storage items and is hosted by a first file server virtual machine of the source distributed file server. The file server manager accesses a mapping between virtual machines of the source distributed file server and virtual machines of the destination distributed file server and replicates the selected share to a second file server virtual machine of the destination distributed file server based on the mapping. The file server manager directs a request to read a first storage item to the destination distributed file server and directs a request to write to a second storage item to the source distributed file server while the destination distributed file server services the request to read the first storage item.

Abstract: In one embodiment, a system for managing a virtualization environment includes a set of host machines, each of which includes a hypervisor, virtual machines, and a virtual machine controller, and a data migration system configured to identify one or more existing storage items stored at one or more existing File Server Virtual Machines (FSVMs) of an existing virtualized file server (VFS). For each of the existing storage items, the data migration system is configured to identify a new FSVMs of a new VFS based on the existing FSVM, send a representation of the storage item from the existing FSVM to the new FSVM, such that representations of storage items are sent between different pairs of FSVMs in parallel, and store a new storage item at the new FSVM, such that the new storage item is based on the representation of the existing storage item received by the new FSVM.