Abstract: A method may include receiving a request for a big data query including characteristics and user parameters and accessing a set of rules based at least in part on respective properties of one or more big data query engines, the set of rules correlating the one or more characteristics, the one or more user parameters and the respective properties. The method may include determining a candidate list including a subset of the big data query engines, determined based on the set of rules. Using a machine learning model, the method may include generating respective probability scores for each big data query engine. The method may include selecting and executing the big data query using a particular big data query engine. The method may include identifying a trigger indicating a performance issue with the particular big data query engine and switching the execution to a second big data query engine.

Abstract: A method may include receiving a request for a big data query including characteristics and user parameters and accessing a set of rules based at least in part on respective properties of one or more big data query engines, the set of rules correlating the one or more characteristics, the one or more user parameters and the respective properties. The method may include determining a candidate list including a subset of the big data query engines, determined based on the set of rules. Using a machine learning model, the method may include generating respective probability scores for each big data query engine. The method may include selecting and executing the big data query using a particular big data query engine. The method may include identifying a trigger indicating a performance issue with the particular big data query engine and switching the execution to a second big data query engine.

Abstract: Disclosed embodiments provide cluster creation using self-joining cluster nodes. Each node within a cluster is assigned a role of master or non-master. A master node initializes and creates a cluster. A non-master, or “role” node initializes and joins a cluster established by the master. If the master has not yet initialized, the non-master node waits until the master node is operational and then joins the cluster. A description file associated with each node describes the services and configuration required for the node's assigned role. In this way, non-master nodes can be spawned independently of the master node, providing a greater level of scalability, improved fault tolerance, and increased availability.

Abstract: Disclosed embodiments provide cluster creation using self-joining cluster nodes. Each node within a cluster is assigned a role of master or non-master. A master node initializes and creates a cluster. A non-master, or “role” node initializes and joins a cluster established by the master. If the master has not yet initialized, the non-master node waits until the master node is operational and then joins the cluster. A description file associated with each node describes the services and configuration required for the node's assigned role. In this way, non-master nodes can be spawned independently of the master node, providing a greater level of scalability, improved fault tolerance, and increased availability.

Abstract: A mechanism is provided in a data processing system for creating clusters on demand. The mechanism installs a cluster on the data processing system. The cluster comprises a master node and a managed node having distributed software installed thereon. The mechanism stores state of cluster on external volumes and removes references specific to the cluster from the state of the cluster stored on the external volumes. The mechanism takes a snapshot of the state of the cluster stored on the external volumes to form a set of volume templates and takes a snapshot of images of the master node and the managed node to form a set of node type images.

Abstract: A mechanism is provided in a data processing system for creating clusters on demand. The mechanism installs a cluster on the data processing system. The cluster comprises a master node and a managed node having distributed software installed thereon. The mechanism stores state of cluster on external volumes and removes references specific to the cluster from the state of the cluster stored on the external volumes. The mechanism takes a snapshot of the state of the cluster stored on the external volumes to form a set of volume templates and takes a snapshot of images of the master node and the managed node to form a set of node type images.

Abstract: A mechanism is provided for automatically recovering one or more distributed cluster nodes on a host system. Responsive to the host system recovering, restoring, or restarting, a set of distributed clustered systems affected due to the reboot of the host system and details associated with each of the set of distributed clustered systems are identified. Using the details, a set of nodes that operate on the host system are identified. For each node: respective components and configurations are prepared; a heartbeat is initiated thereby causing the node to register with a cluster network and identify peers within their associated distributed clustered system; a determination is made as to whether the node has reestablished connection with its associated distributed clustered system; and, responsive to the node reestablishing connection with its associated distributed clustered system, service components are started on the node.

Abstract: A mechanism is provided for automatically recovering one or more distributed cluster nodes on a host system. Responsive to the host system recovering, restoring, or restarting, a set of distributed clustered systems affected due to the reboot of the host system and details associated with each of the set of distributed clustered systems are identified. Using the details, a set of nodes that operate on the host system are identified. For each node: respective components and configurations are prepared; a heartbeat is initiated thereby causing the node to register with a cluster network and identify peers within their associated distributed clustered system; a determination is made as to whether the node has reestablished connection with its associated distributed clustered system; and, responsive to the node reestablishing connection with its associated distributed clustered system, service components are started on the node.

Abstract: A mechanism is provided for allocating a hyper-threaded processor to nodes of multi-tenant distributed software systems. Responsive to receiving a request to provision a node of the multi-tenant distributed software system on the host data processing system, a cluster to which the node belongs is identified. Responsive to the node being a second type of node, responsive to determining that another second type of node in the same cluster has been provisioned on the host data processing system, and responsive to the number of unallocated VPs on different physical processors from that of the other second type of node being greater than or equal to the requested number of VPs for the second type of node, the requested number of VPs for the second type of node is allocated each to a different physical processor from that of the other second type of node.

Abstract: A mechanism is provided for allocating a hyper-threaded processor to nodes of multi-tenant distributed software systems. Responsive to receiving a request to provision a node of the multi-tenant distributed software system on the host data processing system, a cluster to which the node belongs is identified. Responsive to the node being a second type of node, responsive to determining that another second type of node in the same cluster has been provisioned on the host data processing system, and responsive to the number of unallocated VPs on different physical processors from that of the other second type of node being greater than or equal to the requested number of VPs for the second type of node, the requested number of VPs for the second type of node is allocated each to a different physical processor from that of the other second type of node.

Abstract: A mechanism is provided in a data processing system for creating clusters on demand. The mechanism installs a cluster on the data processing system. The cluster comprises a master node and a managed node having distributed software installed thereon. The mechanism stores state of cluster on external volumes and removes references specific to the cluster from the state of the cluster stored on the external volumes. The mechanism takes a snapshot of the state of the cluster stored on the external volumes to form a set of volume templates and takes a snapshot of images of the master node and the managed node to form a set of node type images.

Abstract: A mechanism is provided in a data processing system for creating clusters on demand. The mechanism installs a cluster on the data processing system. The cluster comprises a master node and a managed node having distributed software installed thereon. The mechanism stores state of cluster on external volumes and removes references specific to the cluster from the state of the cluster stored on the external volumes. The mechanism takes a snapshot of the state of the cluster stored on the external volumes to form a set of volume templates and takes a snapshot of images of the master node and the managed node to form a set of node type images.

Abstract: A mechanism is provided tor automatically recovering one or more distributed cluster nodes on a host system. Responsive to the host system recovering, restoring, or restarting, a set of distributed clustered systems affected due to the reboot of the host system and details associated with each of the set of distributed clustered systems are identified. Using the details, a set of nodes that operate on the host system are identified. For each node: respective components and configurations are prepared; a heartbeat is initiated thereby causing the node to register with a cluster network and identify peers within their associated distributed clustered system; a determination is made as to whether the node has reestablished connection with its associated distributed clustered system; and, responsive to the node reestablishing connection with its associated distributed clustered system, service components are started on the node.

Abstract: A mechanism is provided for automatically recovering one or more distributed cluster nodes on a host system. Responsive to the host system recovering, restoring, or restarting, a set of distributed clustered systems affected due to the reboot of the host system and details associated with each of the set of distributed clustered systems are identified. Using the details, a set of nodes that operate on the host system are identified. For each node: respective components and configurations are prepared; a heartbeat is initiated thereby causing the node to register with a cluster network and identify peers within their associated distributed clustered system; a determination is made as to whether the node has reestablished connection with its associated distributed clustered system; and, responsive to the node reestablishing connection with its associated distributed clustered system, service components are started on the node.

Abstract: A mechanism is provided for allocating a hyper-threaded processor to nodes of multi-tenant distributed software systems. Responsive to receiving a request to provision a node of the multi-tenant distributed software system on the host data processing system, a cluster to which the node belongs is identified. Responsive to the node being a second type of node, responsive to determining that another second type of node in the same cluster has been provisioned on the host data processing system, and responsive to the number of unallocated VPs on different physical processors from that of the other second type of node being greater than or equal to the requested number of VPs for the second type of node, the requested number of VPs for the second type of node is allocated each to a different physical processor from that of the other second type of node.

Abstract: A mechanism is provided for allocating a hyper-threaded processor to nodes of multi-tenant distributed software systems. Responsive to receiving a request to provision a node of the multi-tenant distributed software system on the host data processing system, a cluster to which the node belongs is identified. Responsive to the node being a second type of node, responsive to determining that another second type of node in the same cluster has been provisioned on the host data processing system, and responsive to the number of unallocated VPs on different physical processors from that of the other second type of node being greater than or equal to the requested number of VPs for the second type of node, the requested number of VPs for the second type of node is allocated each to a different physical processor from that of the other second type of node.