Книга: Mastering VMware® Infrastructure3

Configuring HA

Configuring HA

Before we detail how to set up and configure the HA feature, let's review the requirements of HA. To implement HA, all of the following requirements should be met:

? All hosts in an HA cluster must have access to the same shared storage locations used by all virtual machines on the cluster. This includes any fibre channel, iSCSI, and NFS datas tores used by virtual machines.

? All hosts in an HA cluster should have an identical virtual networking configuration. If a new switch is added to one host, the same new switch should be added to all hosts in the cluster.

? All hosts in an HA cluster must resolve the other hosts using DNS names. 

First and foremost, to configure HA a cluster must be created. Once the cluster is created, you can enable and configure HA. Figure 10.28 shows a cluster enabled for HA. 

Figure 10.28 A cluster of ESX Server hosts can be configured with HA and DRS. The features are not mutually exclusive and can work together to provide availability and performance optimization.

Configuring an HA cluster revolves around three different settings:

? Host failures allowed

? Admission control

? Virtual machine options

The configuration option for the number of host failures to allow, shown in Figure 10.29, is a critical setting. It directly influences the number of virtual machines that can run in the cluster before the cluster is in jeopardy of being unable to support an unexpected host failure.

Figure 10.29 The number of host failures allowed dictates the amount of spare capacity that must be retained for use in recovering virtual machines after failure.

To explain the workings of HA and the differences in the configuration settings, let's look at implementation scenarios. For example, consider five ESX Server hosts named Silo101 through Silo105. All five hosts belong to an HA cluster configured to support single-host failure. Each node in the cluster is equally configured with 12GB of RAM. If each node runs eight virtual machines with 1GB of memory allocated to each virtual machine, then 8GB of unused memory across four hosts is needed to support a single-host failure. The 12GB of memory on each host minus 8GB for virtual machines leaves 4GB of memory per host. Figure 10.30 shows our five-node cluster in normal operating mode. 

Figure 10.30 A five-node cluster configured to allow single-host failure.

Let's assume that service console and virtual machine overhead consume 1GB of memory, leaving 3GB of memory per host. If Silo101 fails, the remaining four hosts will each have 3GB of memory to contribute to running the virtual machines orphaned by the failure. The 8GB of virtual machines will then be powered on across the remaining four hosts that collectively have 12GB of memory to spare. In this case, the configuration supported the failover. Figure 10.31 shows our five-node cluster down to four after the failure of Silo101. Assume in this same scenario that Silo101 and Silo102 both experience failure. That leaves 16GB of virtual machines to cover across only three hosts with 3GB of memory to spare? In this case, the cluster is deficient and not all of the orphaned virtual machines will be restarted.

Figure 10.31 A five-node cluster configured to allow single-host failure is deficient in resources to support a second failed node.

The admission control setting goes hand in hand with the Number of host failures allowed setting. There are two possible settings for admission control:

? Do not power on virtual machines if they violate availability constraints (known as strict admission control).

? Allow virtual machines to be powered on even if they violate availability constraints (known as guaranteed admission control).

In the previous example, virtual machines would not power on when Silo102 experienced failure because by default an HA cluster is configured to use strict admission control. Figure 10.32 shows an HA cluster configured to use the default setting of strict admission control. 

Figure 10.32 Strict admission control for an HA cluster prioritizes resource balance and fairness over resource availability.

With strict admission control, the cluster will reach a point at which it will no longer start virtual machines. Figure 10.33 shows a cluster configured for two-node failover. A virtual machine with more than 3GB of memory reserved is powering on, and the resulting error is posted stating that insufficient resources are available to satisfy the configured HA level.

Figure 10.33 Strict admission control imposes a limit at which no more virtual machines can be powered on because the HA level would be jeopardized.

If the admission control setting of the cluster is changed from strict admission control to guaranteed admission control, then virtual machines will power on even in the event that the HA failover level is jeopardized. Figure 10.34 shows a cluster reconfigured to use guaranteed admission control.

Figure 10.34 Guaranteed admission control reflects the idea that when failure occurs, availability is more important than resource fairness and balance.

With that same cluster now configured with guaranteed admission control, the virtual machine with more than 3GB of memory can now successfully power on. In Figure 10.35, the virtual machine has successfully powered on despite the large memory use and lack of available unused resources to achieve the proper HA failover.

Figure 10.35 Guaranteed admission control allows resource consumption beyond the levels required to maintain spare resources for use in the event of a server failure.

Not all virtual machines are equal. There are some that are more important or more critical and that require higher priority when ensuring availability. When an ESX Server host experiences failure and the remaining cluster nodes are tasked with bringing virtual machines back on line, they have a finite amount of resources to fill before there are no more resources to allocate to virtual machines that need to be powered on. Rather than leave the important virtual machines to chance, an HA cluster allows for the prioritization of virtual machines. The restart priority options for virtual machines in an HA cluster include Low, Medium, High, and Disabled. For those virtual machines that should be brought up first, the restart priority should be set to High. For those virtual machines that should be brought up if resources are available, the restart priority can be set to Medium and/or Low. For those virtual machines that will not be missed for a period of time and should not be brought on line during the period of reduced resource availability, the restart priority should be set to Disabled. Figure 10.36 shows an example of virtual machines with various restart priorities configured to reflect their importance. The diagram details a configuration where virtual machines like domain controllers, database servers, and cluster nodes are assigned higher restart priority.

Figure 10.36 Restart priorities help minimize the downtime for more important virtual machines.

The restart priority is only put into place for the virtual machines running on the ESX Server host that experienced an unexpected failure. Virtual machines running on hosts that have not failed are not affected by the restart priority. It is possible then that virtual machines configured with a restart priority of High may not be powered on by the HA feature due to limited resources, which are in part due to lower-priority virtual machines that continue to run. For example, as shown in Figure 10.37, Silo101 hosts five virtual machines with a priority of High and five other virtual machines with priority values of Medium and Low. Meanwhile, Silo102 and Silo103 each hold ten virtual machines, but of the 20 virtual machines between them, only four are considered of high priority. When Silo101 fails, Silo102 and Silo103 will begin powering the virtual machines with a high priority. However, assume there were only enough resources to power on four of the five virtual machines with high priority. That leaves a high-priority virtual machine powered off while all other virtual machines of medium and low priority continue to run on Silo102 and Silo103.

Figure 10.37 High-priority virtual machines from a failed ESX Server host may not be powered on because of a lack of resources — resources consumed by virtual machines with a lower priority that are running on the other hosts in an HA cluster.

At this point in the VI3 product suite, you can still manually remedy this imbalance. Any disaster-recovery plan in a virtual environment built on VI3 should include a contingency plan that identifies virtual machines to be powered off to make resources available for those virtual machines with higher priority as a result of the network services they provide. If the budget allows, construct the HA cluster to ensure that there are ample resources to cover the needs of the critical virtual machines, even in times of reduced computing capacity.

Previously, we introduced the Automated Availability Manager (AAM) and its role in conducting the heartbeat that occurs among all the nodes in the HA cluster. The heartbeat among the nodes in the cluster identifies the presence of each node to the other nodes in the cluster. When a heartbeat is no longer presented from a node in the HA cluster, the other cluster nodes spring into action to power on all the virtual machines that the missing node was running.

But what if the node with the missing heartbeat was not really missing? What if the heartbeat was missing but the node was still running? And what if the node with the missing heartbeat is still locking the virtual machine files on SAN LUN, thereby preventing the other nodes from powering on the virtual machines?

Let's look at two particular examples of a situation VMware refers to as a split-brained HA cluster. Let's assume there are three nodes in an HA cluster: Silo101, Silo102, and Silo103. Each node is configured with a single virtual switch for VMotion, and a second virtual switch consisting of a Service Console port and a virtual machines port group, as shown in Figure 10.38.

Figure 10.38 ESX Server hosts in an HA cluster using a single virtual switch for Service Console and virtual machine communication.

To continue with the example, suppose that an administrator mistakenly unplugs the Silo101 Service Console network cable. When each of the nodes identifies a missing heartbeat from another node, the discovery process begins. After 15 seconds of missing heartbeats, each node then pings an address called the isolation response address. By default this address is the default gateway IP address configured for the Service Console. If the ping attempt receives a reply, the node considers itself valid and continues as normal. If a host does not receive a response, as Silo101 wouldn't, it considers itself in isolation mode. At this point, the node will identify the cluster's Isolation Response configuration, which will guide the host to either power off the existing virtual machines it is hosting or leave them powered on. This isolation response value, shown in Figure 10.39, is set on a per-virtual machine basis. So what should you do? Power off the existing virtual machine? Or leave it powered on?

The answer to this question is highly dependent on the virtual and physical network infrastructures in place. In our example, the Service Console and virtual machines are connected to the same virtual switch bound to a single network adapter. In this case, when the cable for the Service Console was unplugged, communication to the Service Console and every virtual machine on that computer was lost. The solution, then, should be to power off the virtual machines. By doing so, the other nodes in the cluster will identify the releases on the locks and begin to power on the virtual machines that were not otherwise included.

Figure 10.39 The Isolation Response identifies the action to occur when an ESX Server host determines it is offline but powered on.

In the next example, we have the same scenario but a different infrastructure, so we don't need to worry about powering off virtual machines in a split-brain situation. Figure 10.40 diagrams a virtual networking architecture in which the Service Console, VMotion, and virtual machines all communicate through individual virtual switches bound to different physical network adapters. In this case, if the network cable connecting the Service Console is removed, the heartbeat will once again be missing; however, the virtual machines will be unaffected since they reside on a different network that is still passing communications between the virtual and physical networks.

Figure 10.40 Redundancy in the physical infrastructure with isolation of virtual machines from the Service Console in the virtual infrastructure provides greater flexibility for isolation response.

Figure 10.41 shows the isolation response setting of Leave powered on, which would accompany an infrastructure built with redundancy at the virtual and physical network levels.

Figure 10.41 The option to leave virtual machines running when a host is isolated should only be set when the virtual and the physical networking infrastructures support high availability.

To support a redundant HA architecture, it is best to ensure that the Service Console port is sitting atop a NIC team where each physical NIC bound to the virtual switch is connected to a different physical switch.

Оглавление книги