Exchange Server 2013/2016 native Load Balancing & Failover options

Although NLB is a supported method for load balancing Exchange Server 2013/2016, it does have some limitations, especially for small to medium environments. If you are trying to architect a highly available Exchange solution you will almost certainly be using Database Availability Groups (DAG’s). DAG’s were introduced in Exchange 2010, and are used to replicate Mailbox Databases between Exchange Servers. The DAG technology is built on Windows Failover Clustering.

Exchange Server 2013 is broken down into two server roles the Client Access Server (CAS) and Mailbox Servers (MBX). The DAG technology only protects failure at the Mailbox Server level, however it does not failover incoming connections to Exchange.  In small networks you want to keep the number of servers to a minimum, it is possible to have both Exchange roles installed on the same server, therefore if you wanted to deploy a highly available Exchange setup you can get away with two servers in total.

Network Load Balancing and Failover Clustering cannot be installed on the same server. Therefore, you cannot use WNLB to load balance the CAS server if you are using dual roles (CAS + MBX) servers. In order for WNLB to be a compatible solution with DAG’s you must separate the CAS and MBX roles onto separate servers.

DNS Round Robin

Although DNS RR can be used to load balance traffic across multiple CAS servers it does not offer any kind of failover. This is mainly because the native Windows DNS Server does not offer any kind of DNS record weight or priority. This means you can have multiple records pointing to your Exchange namespace (i.e mail.domain.com) using the IP’s of the CAS servers.

For this to even offer load balancing you have to reconfigure the TTL of the DNS records at both the DNS Server level, and the end clients.

To reduce the TTL on the DNS A records enable the Advanced Features inside DNS, when you create an (A) record you will then have the option to set a TTL on the record.

In this example I have set it very low to only 15 seconds, this maybe a bit low for most production environments. Especially if you only have a couple of DC/DNS servers.

So I have DNS RR setup to load balance across 3 x CAS servers.

·         mail.ryanbetts.co.uk > 192.168.1.104

·         mail.ryanbetts.co.uk > 192.168.1.105

·         mail.ryanbetts.co.uk > 192.168.1.106

Although the TTL has been lowered at the server side it must also be done on the clients. By default Windows caches DNS lookups for 1 hour.  To configure the DNS TTL cache on a localling in Windows, open the Registry at Local Machine\System CurrentControlSet\Services\DnsCache\Parameters

Create a 32 bit DWORD called "MaxCacheTtl" , set this to a value in seconds (Decimal).

For the changes to take a affect you must restart the DNS Client service. Although making these configuration changes load balances the traffic across CAS servers, it does not offer a very good user experience.

DNS RR in action on a Windows 8.1 client, when the DNS cache expires and the client does another query to it's primary DNS server the Outlook client remains connected to Exchange.

However I have noticed (I'm guessing) when the cache expires the client is sometimes prompted to enter credentials again. Which is annoying for any users, especially if you set the TTL to somethin extremely low like 15 seconds.

DNS Round Robin no "failover"

For a test to prove this to myself, so that I really understand it. I failed over my only DAG to the 2nd Exchange server (192.168.1.105). 

I then turned the 1st server off (192.168.1.104) however, as I did not change the DNS RR configuration the client was still resolving mail.ryanbetts.co.uk to the IP address of the 1st Exchange server. When the TTL expired it started resolving to 192.168.1.105 which made a successful connection.

The only way to make this a "highly available" configuration is to effectively disable DNS RR in the event one of the CAS servers fails. In this scenario you would simple delete the mail.ryanbetts.co.uk record that was pointing to the failed server, therefore all clients would be directed to the surviving server. This might be acceptable for some businesses but as it's not active/active it won't fit all requirements.

I am going to look at the free Kemp Layer 4-7 Load Balancer to load balance Exchange Server, this is free but has a limit of 20 mpb/s throughput and a maximum of 50 concurrent sessions. 

WSUS for SCCM SUP install on Win2008 R2 fails with "The update could not be found. There may be a network connection issue".

When trying to install WSUS on Windows Server 2008 R2 Standard the following error is thrown "The update could not be found. There may be a network connection issue". This is when trying to install WSUS from the Server Manager console.

The problem here was that this server was configured with a Local Policy to use another WSUS server as it's source for Windows Updates. I have no idea why it was configured as a Local Policy and not as a Group Policy. However, to resolve the problem, simply open the Local Policy editor, and browse to Computer Configuration/Administrative Templates/Windows Components/Windows Update.

Double click the Specify Intranet Microsoft update service location in my environment, this was enabled, with a legacy WSUS server populated in the intranet update fields. I changed this to Not Configure.

To double check the Local Policy has removed the problematic setting open the Registry, browse to Local Machine\Software\Policies\Microsoft\WindowsUpdate the intranet service points should not be present in this list of settings.

SCCM 2012 R2 error generated as Client Push tries to install SCCM Client onto a Hyper-V Cluster Object

SCCM 2012 R2 error "Client Configuration Manager failed to complete the ConfigMgr installation on client "HVCLUST01". In the past 168 hours, CCM has made 175 unsuccessful attempts. The operating system reported error 53: The network path was not found. This was failing because SCCM was trying to push the client to a Failover Cluster object for a Hyper-V Cluster. This is just a computer account that represents the logical cluster in Active Directory. However it was being discovered as a new computer during an SCCM discovery. 

The error was not causing any negative problems it was just causing red flags to be logged, which nobody likes.

The fix was to put the Hyper-V Cluster objects host name into the excludes servers list in the Registry of the SCCM server for that particular site. Browse to Local Machine\Software\Microsoft\SMS\Components\SMS_Discovery_Data_Manager and open the Registry key "ExcludeServers"

SCCM 2012 R2 "Configuration Manager did not find a site to manage this client." on Windows 8.1 client

After troubleshooting some problems with deployment of the SCCM Client to Windows 7, 8.1 and 10 devices, the manual installation worked however when your try to connect the client to a site it fails with "Configuration Manager did not find a site to manage this client." In this environment the SCCM infrastructure had been rebuilt twice previously, as this was a test setup to simulate a real deployment of SCCM 2012 R2.

The problem was caused by two things, the first is that I had not get the default Site for the Boundary Group the problematic client was in. To check or resolve this click Administration/Hierarchy Configuration/Boundary Groups. Right click on the Boundary Group and select Properties. 

Click the Reference tab, click the option Use this boundary group for site assignment, select the correct site from the drop down. Then click the Add button and select the corresponding site server.

This only corrected some of the problems, newly built clients that had not had any visibility of the old installation of SCCM were fine, they could resolve the Site Code without problem. However old clients that were previously managed by the old SCCM installation still failed. This was because the old installation of SCCM was configured to deploy the client via GPO's. In the GPO's the Site Code had been statically configured, this is a setting that is tattooed onto the client machines Registry. Even after the GPO was removed from the computer the SMS Site Code is still in the registry, in my example this Site Code no longer exists, hence it causes the problem. To resolve this open the Registry on the problematic installation and browse to Local Machine\Software\Microsoft\SMS\Mobile Client. 

Inside this folder there was an entry "GPRequestedSiteAssignmentCode", in this instance this was set to the old Site Code. I deleted the entire entry and it resolved the problem.

AD Connect syncing msExchangeMailboxGuid object causes "This user's on-premises mailbox hasn't been migrated to Exchange Online. The Exchange Online mailbox will be available after migration is completed." for new Office 365 mailboxes

When you try to open a new mailbox for an Office 365 users you get the following error;

"This user's on-premises mailbox hasn't been migrated to Exchange Online. The Exchange Online mailbox will be available after migration is completed."

AD Connect is configured to sync users, groups and passwords from the existing Active Directory (SBS 2011), however the option for “Exchange Hybrid Deployment” was not selected on purpose. This is because in this particular case the migration was for 6 users, therefore a PST export/import was done to migrate the e-mails, contacts and calendars.

The root of the problem is because the Active Directory attribute msExchangeMailboxGuid is being synced to Azure AD in Office 365, when it’s not required. 

You have to edit a configuration inside AD Connect (it’s actually FIM 2010 R2 under the covers). To open the configuration panel for FIM, browse to C:\Program Files\Microsoft Azure AD Sync\UIShell and launch miisclient.exe.

Click Connectors and select the connector for your local Active Directory and choose Properties.

Click Select Attributes and scroll until you find msExchangeMailboxGuid, if you have the same problem as me this will be selected. Simply disable this attribute.

You then have to delete the old reference to the msExchangeMailboxGuid from the FIM Connector Space. To do this select the Active Directory management agent (also known as a “connector”), and choose Delete. Read the next part properly.

Ensure that Delete Connection Space Only is selected and click OK.

It will ask you to confirm you want to delete data from the connection space, click Yes. If you did delete the entire connector, you could provision again by running the AD Connector wizard. This is fine if you have not made any major modifications to your AD Connect configuration. 

Use the following PowerShell command to force an entire sync across AD Connect;

Start-ADSyncSyncCycle -PolicyType Initial

You should notice updates when the Operations complete.

Now if you return to Office 365, you should see the following.

Export Exchange mailboxes to PST from the server side

I was recently doing a small Exchange migration using the PST export/import method. Using client side Outlook to do the exporting sometimes works and sometimes it has issues exporting everything. 

In Outlook 2013 if you are doing an export ensure you have configured the profile to cache all the mailbox data and not just the last 12 months, which is the default;

However its easier to do an export from the server side using Powershell;

Create a new share with the following permissions;

• Share permissions - Everyone F/C
• NTFS permissions - Everyone F/C + Exchange Trusted Subsystem F/C

Open the Exchange Shell as an Administrator and type;

New-MailboxExportRequest -Mailbox "user" -FilePath "\\uncpath.pst"

To check progress;

Get-MailboxExportRequest

Post Exchange migration Outlook won't load with "Cannot start microsoft outlook. cannot open the outlook window. the set of folders cannot be opened. you must connect to microsoft exchange with the current profile before you can synchronize with your outlook data file (.ost)"

"Cannot start microsoft outlook. cannot open the outlook window. the set of folders cannot be opened. you must connect to microsoft exchange with the current profile before you can synchronize  with your outlook data file (.ost)"

Clear the Outlook profile folder on C:\Users\Profile\AppData\Local\Microsoft\Outlook

CTRL + R and enter "Outlook /resetnavpane"