Abstract: Systems and methods for high availability computing systems. Systems and methods include disaster recovery of two-node computing clusters. A method embodiment commences upon identifying a computing cluster having two nodes, the two nodes corresponding to a first node and a second node that each send and receive heartbeat indications periodically while performing storage I/O operations. One or both of the two nodes detect a heartbeat failure between the two nodes, and in response to detecting the heartbeat failure, one or both of the nodes temporarily cease storage I/O operations. A witness node is accessed in an on-demand basis as a result of detecting the heartbeat failure. The witness performs a leadership election operation to provide a leadership lock to only one requestor. The leader then resumes storage I/O operations and performs one or more disaster remediation operations. After remediation, the computing cluster is restored to a configuration having two nodes.

Abstract: Systems and methods for managing communication between applications (e.g., apps) in a host computing environment. Apps are published to a globally-accessible site. Upon download of an app to a host computing environment, apps can register themselves with a communication gateway as being discoverable and permissive to inter-app communications. Message queues are created to facilitate asynchronous communications between apps. After registration, any of the apps can send and receive messages using the communication gateway. The messages can be directed to any other app that is registered with the communication gateway. Using the message queues, the communication gateway facilitates asynchronous app interactions such that any app can communicate with any other discoverable and permissive app. Aspects of operation, discoverability and other attributes can be codified in an application manifest that is processed by the communication gateway.

Abstract: A method commences upon receiving a first one of a series of commands corresponding to various operations (e.g., in a test or development setting) that are intended to access protected health information (PHI). Initially, the protected health information is stored in a protected source datastore that is logically represented by source metadata. Rather than copy PHI, the method clones the metadata, leaving the PHI uncopied and unmodified. Execution of a read-only operation over the protected health information is performed by referencing the cloned metadata to access the protected health information. In the event of an occurrence of an operation (e.g., a testing operation), a temporary, ephemeral datastore is formed. Responsive to received write operations pertaining to the protected health information, the data pertaining to the intended write operations are pre-staged into the ephemeral datastore. After use, ephemeral data is overwritten, purged from caches, and then the ephemeral datastore is deleted.

Abstract: A system for implementing closed-loop and dynamic capacity management in a web-scale data center is provided. In some embodiments, the method includes a local feedback loop for local system data collection and reconfiguration process that is responsive to inputs received from a global sizing system, where the global sizing system maintains local system data collected at a plurality of systems and transferred to the global sizing system using a data collection feedback mechanism. In some embodiments, the global sizing system manages a set of data representing best practices, and performs data analysis on the local system data collected at a plurality of systems in order to determine if there are any updates to the dataset for the best practices data. Finally, the global sizing system transmits the best practices data to the local system for analysis of the local systems based on the best practices data.

Abstract: Techniques for rebalancing computing workloads between a set of on-premises resources of an on-premises system and a set of cloud computing resources of a cloud computing system. A method embodiment commences upon detecting a rebalancing trigger event. Responsive to the trigger event, a set of resource data corresponding to observations pertaining to the on-premises resources and resource data corresponding to observations pertaining the cloud resources is accessed. Based at least in part on the resource data, a plurality of candidate workload rebalancing plans are generated. The candidate workload rebalancing plans are evaluated against the likelihood of achieving quantitative objectives resulting from the rebalancing. A portion of the scheduling commands to carry out the workload rebalancing plans are sent to the cloud computing system, and another portion of the scheduling commands are sent to the on-premises computing system. The receiving computing systems carry out the scheduling commands.

Abstract: Facilitating deduplication of data in a computing system without managing access to reference count variables. A method embodiment commences upon detecting first data unit and calculating a first checksum value. At a later time, a second data unit is received and the system calculates a second checksum value. If the second checksum value is the same as the first checksum value, then the first data unit and the second data unit are the same data and need not be duplicated. In such cases, an entry in the metadata points to the location of the first data unit that is already stored. Additional metadata entries are made in the metadata to associate a Boolean usage state flag and a Boolean deletion state flag with the second checksum value. Periodically scans of the metadata are performed. When both Boolean flags are in a particular state, the deduplicated data is deleted.

Abstract: Techniques for managing virtualized entities in computing systems. In a method embodiment, processing commences upon receiving I/O activity trace data associated with virtualized entities running in a computing system. Specific I/O activity attributes are extracted from the I/O activity trace data, and the I/O activity attributes are used to form a workload classification model. The workload classification model serves to assign one or more workload classifications to a respective one or more observed workloads running on the computing system. Based on the determined workload classification or classifications, recommended resource allocation operations are formed for further consideration. Considered resource allocation operations include migrations of virtualized entities from a source computing resource to a target computing resource.

Abstract: Systems and methods for managing access to storage devices in a distributed data storage environment. Embodiments operate to manage communications between a client computing device and storage target devices in a distributed storage system. The distributed storage system comprises one or more computing nodes and at least one storage target device. A client computing device interfaced with the distributed storage system uses an IP address to access a leader virtualized controller. Upon receipt of a storage access protocol message by the leader virtualized controller elected from a set of virtualized controllers, a redirect message comprising a second IP address that identifies a second virtualized controller is sent to the client computing device. The client computing device connects to the second virtualized controller. Messages are sent between the client computing device and the second virtualized controller, which in turn accesses the storage target device to carry out storage I/O protocol messaging.

Abstract: Systems for managing firmware updates in a computing system. A computing system comprises multiple computing nodes. A plurality of computing nodes include firmware-upgradable components from multiple vendors. When upgrading the firmware of components of the computing system, a firmware management agent is invoked to interact with firmware management plug-ins through an abstraction layer. The abstraction layer translates vendor-agnostic firmware operations into vendor-specific firmware operations. The firmware management agent determines the then-current status of the firmware-upgradable components and issues a series of vendor-agnostic firmware commands to initiate firmware upgrades over the computing nodes of the computing system. The firmware management agent generates and manages a firmware update schedule to sequence or parallelize firmware updates across multiple nodes of the computing system.

Abstract: An architecture for implementing a system planner for determining a number of nodes required in a networked virtualization system based on increasing node density is provided. The system planner receives various inputs describing a current networked virtualization system, an analysis period during which the workload of the current networked virtualization system is expected to increase, and a projected increase in node density during the analysis period. Based on the inputs, the system planner generates a new configuration of the networked virtualization system that includes a number of new nodes that are added to the current networked virtualization system to provide the resources necessary to support the increase in workload during the specified analysis period.

Abstract: Computing cluster system management. Embodiments implement fine-grained rule-based approaches to error recovery. A service dispatches tasks to components of the computing cluster. At the time of task dispatching, entries are made into a write-ahead log. The write-ahead log entries serve for recording task and component attributes. A monitor detects a failure event raised by one or more of the components of the computing cluster. Responses to the failure event include determining a set of conditions that are present in the computing cluster at the time of the detection, and then using the failure event and the determined conditions in combination with a set of fine-grained failure processing rules to determine one or more recovery actions to take. Recovery actions include redistributing the failed task to a different node or to different service. Certain conditions and rules initiate actions that rollback the state of a component to a previous success points.

Abstract: Systems and methods for rebalancing storage-oriented workloads across resources of a distributed data storage facility. Communications are initiated between a client computing device and a plurality of storage target devices of a distributed storage system. The distributed storage system comprises one or more computing nodes that can host virtualized controllers. A client computing device interfaced with the distributed storage system uses an IP address to access an initial virtualized controller. Upon detecting a change event, conditions of the computing environment are analyzed to determine rebalancing options. Analysis of the rebalancing options results in identification of an alternative virtualized controller. A redirect message that identifies the alternative virtualized controller is sent to the computing device. The client computing device connects to the alternative virtualized controller.

Abstract: Systems for virtual memory computing systems. A set of hardware or software operational elements of a computing system performs virtualized memory paging. The operational elements serve to identify a random access memory device and at least one random access persistent memory device (RAPM) in a computing system. The random access persistent memory device is configured as a swap device that is apportioned as having at least some address space for swap. At least some of the swap address space is assigned to one or more virtualized entities in the computing system. When a page swap event is detected by the computing system, one or more of the operational elements execute one or more paging operations based on characteristics of the page swap event. The paging operations perform swap-in or swap-out of at least one page between the random access memory device and the random access persistent memory device.

Abstract: Systems for distributed computing systems. A topological configuration of computing nodes is selected to manage availability of metadata data in a computing system. A method embodiment commences by accessing a plurality of node topology attributes and using those attributes to map between nodes and availability domains. Resource usage measurements such as computing node load are collected. A plurality of candidate replication configurations are generated, and each candidate replication configuration is scored with respect to several quantitative objectives. Additionally, the candidate replication configurations are given respective resource usage balance scores. One or more candidate replication configurations are selected based on resource usage balance scores and/or a separation skew value. Determination of a selected configuration is dominated by resource usage when there is a tie between best-scoring configurations or when none of the configurations meet a scoring threshold.

Abstract: An architecture for implementing a mechanism for automatically displaying metrics specific to a type of workload being processed by a computer system is provided. The mechanism predicts a classification of the workload based on attributes that characterize the workload using a set of workload profiles and/or a set of classification rules that correlate different combinations of attributes of workloads with different classifications of workloads. Based on the predicted classification of the workload, one or more templates including one or more metrics specific to the classification of workload are identified. The template(s) including the metric(s) specific to the classification of the workload may be identified using set of rules that associate the metric(s) with the predicted classification of the workload. A user interface including the metric(s) is generated based on the template(s). The user interface may then be displayed to a user of the computer system.

Abstract: A method for time series analysis of time-oriented usage data pertaining to computing resources of a computing system. A method embodiment commences upon collecting time series datasets, individual ones of the time series datasets comprising time-oriented usage data of a respective individual computing resource. A plurality of prediction models are trained using portions of time-oriented data. The trained models are evaluated to determine quantitative measures pertaining to predictive accuracy. One of the trained models is selected and then applied over another time series dataset of the individual resource to generate a plurality of individual resource usage predictions. The individual resource usage predictions are used to calculate seasonally-adjusted resource usage demand amounts over a future time period. The resource usage demand amounts are compared to availability of the resource to form a runway that refers to a future time period when the resource is predicted to be demanded to its capacity.

Abstract: A method for providing external access into a secured networked virtualization environment, includes performing a leadership election amongst nodes of the secured networked virtualization environment to elect a leader node, assigning a cluster virtual IP address to the leader node and generating a reverse tunnel, using a processor, by the leader node to allow for an external entity to communicate with the secured networked virtualization environment.

Abstract: Particular embodiments provide a system for managing a virtualization environment, the system comprising a plurality of host machines, one or more virtual disks comprising a plurality of storage devices, and a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), wherein each of the FSVMs is running on one of the host machines and conducts I/O transactions with the one or more virtual disks, and the host machines form a cluster. A health monitoring service may monitor the FSVMs to determine whether any of the FSVMs has failed or is having problems. If the health monitoring service detects a failure of one of the FSVMs on a first one of the host machines, for each resource lock associated with the first FSVM, the health monitoring service may reassign the resource lock to another one of the FSVMs.

Abstract: In one embodiment, a system for managing a virtualization environment includes a set of host machines, each of which includes a hypervisor, virtual machines, and a virtual machine controller, one or more virtual disks comprising a plurality of storage devices, the one or more virtual disks being accessible by the virtual machine controllers, where the virtual machine controllers conduct I/O transactions with the virtual disks, and an upgrade system configured to receive a request to upgrade a virtualized file server on a plurality of host machines from a first program code version to a second program code version, for each host machine, generate a snapshot of a code image associated with the second program code version, and provide the snapshot to the host machine via the virtual disk. The virtual machine controller located on the host machine stores the snapshot on a first storage device associated with the host machine.

Abstract: A system and method for migrating a virtual machine and storage may include receiving a request to migrate a virtual machine from a host machine. The system and method include establishing a storage space on a shared storage space and creating an access table and a location table. The access table includes access values indicative of data being accessed. The location table includes location values indicative of a location of the data in the first storage space or a shared storage space. A transfer of data between the first storage space and the shared storage space is done using the access table and the location table. The data is accessible in both the first storage space the shared storage space based on the one or more location values of the location table and access to the data is based on the one or more access values of the access table.