Configuring Citrix NetScaler v10.5 VPX High Availability to Load Balance HTTP Traffic

Citrix Netscaler is an Application Delivery Controller (ADC), by Citrix Systems. Netscaler is a widely deployed appliance that is available in three forms, the MPX (physical appliance), the VPX (virtual appliance) and the SPX, the physical appliance running XenServer that can host multiple virtual instances of Netscaler. If am using Netscaler to load balance ordinary HTTP traffic between two Windows Server 2008 R2 servers, with the IIS 7.5 role installed.

The topology that is being adopted is the “Two-armed mode, multi-subnet” model as show below, this is a Citrix recommended design when deploying Netscaler.

You can download a trial of the Citrix NetScaler 10.5 VPX from Citrix. It is available for XenServer, Hyper-V and VMware vSphere. In this example I am using vSphere, when you download the vSphere version of the VPX it comes as an OVF file that should be imported into vSphere. This can be done from the local machine you are using the vSphere Console from, so there is no need to upload the OVF to a vSphere datastore.

In Citrix Netscaler there is a significant difference between Clustering and High Availability, for one Clustering requires a special "clustering" license, where as traditional High Availability is provided as part of all the Netscaler editions.

In my example I am configuring two Netscaler VPX's in a HA pair, the following facts should be noted with HA and Citrix Netscaler;

·       Setup in Pairs (max 2 nodes)

·       Primary Node owns the VIP, SNIP (only one per pair)

·       Heartbeat every 200ms over UDP/3003 (3 second threshold for failover to initiate)

·       TCP port 3010, 3008 is used for node sync, file sync TCP 22

·       Configuration made on the primary are replicated over TCP 3011, 3009

As this is only a test environment I have created two new vSphere Standard Switches, with no adapter uplinks connected. The External vSwitch represents a DMZ, and the Internal my local area network.

My TCP/IP configuration(s) are as follows;

• RB_Test_Internal (LAN Subnet) – 192.168.0.0/24
• RB_Test_External (DMZ Subnet) – 172.16.0.0/24
• NS01 (NSIP) is 192.168.0.20/24
• NS02 (NSIP) is 192.168.0.21/24
• HA Pair (SNIP) is 192.168.0.23/24
• Web Server 1 is 192.168.0.50/24
• Web Server 2 is 192.168.0.51/24
• NS HA Pair VIP is 172.16.0.100/24

If you have reviewed the Citrix eDocs on Netscaler, the physical topology and logical subnet configuration I am doing in this example is referred to as a “mutli-armed, multi-subnet” deployment.

In a production environment you would probably have several dedicated uplinks from each of these vSwitches to provide connectivity to the physical networks. These uplinks would be either access ports or trunk ports depending where you are doing EST, or VGT for VLAN tagging.

Once the OVF appliance is imported, open a Console Connection to the VPX to set the initial configuration at this stage this will be the address that is used to manage the Netscaler VPX from your web browser.

Once the initial management IP is set, you can use a browser to connect to the Netscaler. It would suggest using Google Chrome as it seems to have the least amount of issues with Java when you are making administration changes.

When you login the first screen you will be presented with will have four options, the Netscaler (NSIP) should already be configured and show a green tick indicating this. 

The next part to configure is the Subnet IP Address (SNIP), this is an interface that is used to communicate with servers on the backend. Click on the Subnet IP Address option to begin configuring it. 

The SNIP address should be on the same subnet and VLAN that your internal servers that you are trying to load balance are. The wizard also provides a simplified break down of how the SNIP is used to communicate with the backend servers.

Step 3 is to configure a hostname for the device along with a DNS server, call this whatever you want a point it to your local DNS server, which will typically be a Domain Controller. You should also remember to manually create an (A) record for the Netscaler pointing to the correct IP in your DNS Forward Lookup Zones. This is usually forgotten as Microsoft devices use Dynamic DNS to do this automatically.

You will be prompted to restart the VPX appliance once your click on done. Step 4 is where you configure the license for the VPX appliance, you can get a 90 trial from Citrix that should be ample for testing. The following blog post here http://blog.ryanbetts.co.uk/2014/09/downloading-licensing-and-basic.html covers licensing the Netscaler VPX in detail.

Once the reboot is completed you should be able to log back into the VPX, and you will be taken to the Configuration window. To ensure your license file has been imported correct click on Licenses, the trial license should allow Load Balancing, Content Switching and SSL Offloading. 

The next step is to configure the High Availability between the two Netscaler VPX's, to do this click System, High Availability, from there you should see the first node in the state UP. Click the Add button. 

You should now enter the NSIP of the secondary node into the Node IP field. The username and password to login to the Netscaler should be the same on both these devices, I have left these as there default nsroot/nsroot.

When you click Create the Netscaler will prompt you to restart the running configuration and reboot the device.

Once the restart has completed, under the High Availability section you should see both nodes. As heart beating should be operating between the devices the first Netscaler VPX should still be operating as the Primary. 

The Actions menu can be used to show details, Force Synchronization and Force Failover between the two devices. 

The next step is to define the Services (or Servers, that you want to load balance between), to do this expand Traffic Management, then Load Balancing and click on Services. Click on Add to launch the wizard.

Configure the settings to be in line with your environment, I have two Web Servers (192.168.0.50 & 51) that are inside the local area network. You must create a Service for each of these servers. 

The servers are still offline for me at the moment therefore they appear as DOWN. This will automatically change when the Netscaler can communicate using the SNIP over ICMP.

Once I brought the servers online and there was connectivity between the Netscaler and the Web Servers the State changed to UP, and the lights went green. It would be a good time to save the running configuration.

Also from the Load Balancing menu, click on Virtual Servers, a Virtual Server in NetScaler is a Netscaler entity that external clients can use to access applications hosted on the servers. A Virtual Server is represented by a hostname, Virtual IP (VIP), port and protocol. Click Add to begin creating a new Virtual Server.

The name of the Netscaler Virtual Server is only locally relevant, therefore it does not make much difference what this is called here. I have configured my Virtual Server with the IP address of 172.16.0.100, which is the subnet that is in use on my DMZ side of the network. The Netscaler VPX's have two NIC's, one on each side of the two networks, LAN and DMZ.

Once you click OK, you will be prompted to enable the feature "LB", click Yes to this.

After this completes you will see under Services and Service Groups, no Virtual Server Service Bindings, click on the arrow to begin configuring this.

This is where we bind the services (or servers to be load balanced) to the Netscaler Virtual Server, click the Plus button to open the console.

Select both of the services that you created in the previous steps, in my example I have named both of my web servers "iisx". Click OK once this has been done.

Click Bind.

Click Done.

You must now click on the Method button from under the Advanced menu, this will expand the configuration screen and allow you to choose a High Availability method. Netscaler supports a number of different load balancing algorithms, the most common ones being;

• LEASTCONNECTION (Which service currently has the fewest client connections. This is the default load balancing algorithm.)
• ROUNDROBIN (Which service is at the top of a list of services. After that service is selected for a connection, it moves to the bottom of the list.)
• LEASTRESPONSETIME (Which load balanced server currently has the quickest response time.)
• URLHASH (A hash of the destination URL.)

A full list of the supported algorithms can be found at the following Citrix eDocs article http://support.citrix.com/proddocs/topic/netscaler-load-balancing-93/ns-lb-customizing-lbalgorithms-wrapper-con.html

I am going to configure LEAST CONNECTION at this stage, once done click OK. You should review the eDocs page to determine which algorithm will suit your needs the best.

The Virtual Server still appears to be “DOWN”, this will come online when the configuration is applied and saved to the memory. Click Done.

Once a refresh has occurred, click the Save icon.

Click Yes to confirm.

Now if you browse to the external VIP IP address, you should be connected to one of the web servers, I changed the default IIS landing page to ensure it was working correctly. 

Exchange 2013 SP1: Client Access Servers (CAS) with Network Load Balancing (NLB) Connecting to the ECP Displays Error "HTTP 500 Internal Server Error"

Exchange 2013 SP1: Client Access Servers (CAS) with Network Load Balancing (NLB) Connecting to the ECP Displays Error "HTTP 500 Internal Server Error"

You have configured Exchange 2013 Client Access Servers (CAS) in a Highly Available cluster using Windows Network Load Balancing (NLB) when you try and connect up to the single namespace for the ECP (https://mail.domain.com/ecp) you are prompted to enter your credentials but when you login it times out and displays "HTTP 500 Internal Server Error", for me this also happened when I tried to connect to my CAS Servers via https://localhost/ecp therefore it caused quite a problem!

 

It seemed to happen straight after I configured the Network Load Balanced (NLB) cluster, so my first thought was to disable the cluster and then try connecting via https://localhost/ecp on each of the CAS servers, this worked!

To disable an NLB Cluster from PowerShell use the following command;

Stop-NLBCluster

 

I then decided to change the Cluster Operation Mode from the default of Unicast to Multicast, this can be done using the nlbmgr.exe GUI, by right clicking on the cluster and selecting Cluster Properties.

VMware Article on Windows Network Load Balancing Cluster Operation Modes http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006580

From the Cluster Parameters you can change it from Unicast to Multicast.

 

Then use the following PowerShell command to restart the cluster;

Start-NLBCluster

 

 

It's a good idea to check inside the GUI that the cluster has started successfully and both nodes are showing green and are in the Converged state.

 

Now if you connect back to the single namespace /ECP, enter your credentials and it should now work as expected.

 

Exchange 2013 Client Access Servers (CAS) using Single Namespace and Windows Network Load Balancing (NLB) "An error occurred while using SSL configuration for endpoint 0.0.0.0:443. The error status code is contained within the returned data".

Exchange 2013 Client Access Servers (CAS) using Single Namespace and Windows Network Load Balancing (NLB) "An error occurred while using SSL configuration for endpoint 0.0.0.0:443. The error status code is contained within the returned data".

You have recently just completed a new Certificate Request in Exchange 2013, the certificate seems to have bound correctly to the Client Access Server (CAS) in which you raised the request, although some users cannot connect to Outlook Web App (OWA) or Outlook. There are two Client Access Servers (CAS) in the environment, load balanced using Network Load Balancing (NLB). On investigating the certificates on the second CAS (not the server that the request was raised on), the Event Logs are filled with the following error "An error occurred while using SSL configuration for endpoint 0.0.0.0:443. The error status code is contained within the returned data" to get users working and to take the pressure of me to fix the issue I used nlbmgr.exe to Stop the problematic host, thus passing all Client Access traffic through only a single CAS.

 

The error "An error occurred while using SSL configuration for endpoint 0.0.0.0:443. The error status code is contained within the returned data" is caused by a conflicting binding in IIS. Therefore I opened an MMC to view the Certificates, it appeared the newly installed public certificate had not been installed correctly on the secondary CAS, the private key associated to the certificate was not found. I removed this certificate and used the MMC on the working CAS server to Export the working Certificate.

 

The wizard is self-explanatory but ensure the Yes, Export the Private Key radio button is selected, click Next.

 

I then used the MMC on the problematic server to import the Certificate with it's Private Key, you can do this by right clicking on the Personal/Certificates store and selecting Import.

 

Now that both of the Client Access Servers have the correct certificate with corresponding private key, it should be viewable by it's Friendly Name from the ECP. You will notice the IIS service is currently assigned to this certificate, and it was working correctly on the original CAS server.

 

 

The next stage was to manually reconfigure the Bindings in IIS on the problematic server, from the IIS Manager GUI click on Default Web Site and click Bindings... from the right hand menu. On the working CAS the bindings were are follows, please note 10.10.7.34 is the VIP of the NLB cluster. I therefore reconfigured the problematic server to match this configuration.

 

HTTP to 10.10.7.34 on Port 80 with No Hostname.

 

HTTPS binding to IP Address 10.10.7.34 on Port 443, with the Hostname mail.domain.com and the SSL Certificate bound should match the Friendly Name of your Certificate request viewable from the ECP.

 

HTTPS with No IP on Port 443, again using the same Friendly Named SSL Certificate.

 

Once this was completed I restarted the IIS Service, and the "An error occurred while using SSL configuration for endpoint 0.0.0.0:443. The error status code is contained within the returned data". was no longer filling the Event Logs. I also went back into NLB Manager and started the problematic CAS again, to test thoroughly I open Internet Explorer and browsed to OWA and checked the Certificates status.