Installing and configuring Active Directory entirely with PowerShell

Creating a new Active Directory for a test environment is something I seem to have to do multiple times a week. I hope to never have to do it the long way with the GUI again.

The following PowerShell commands installed AD DS and creates a new forest/domain called “domain.co.uk”. You are prompted for the AD Restore Password, the rest is automated.

Import-Module ServerManager

Add-WindowsFeature AD-Domain-Services

ADD-WindowsFeature RSAT-Role-Tools

Install-AddsForest -DomainName domain.co.uk -CreateDnsDelegation:$false -DatabasePath "C:\Windows\NTDS" -DomainMode "Win2012R2" -DomainNetbiosName "DOMAIN" -ForestMode "Win2012R2" -InstallDns:$true -LogPath "C:\Windows\NTDS" -NoRebootOnCompletion:$false -SysvolPath "C:\Windows\SYSVOL" -Force:$true

Azure VHD Upload Fails “An upload is already in progress on machine HOSTNAME with process ID”

Uploading VHD’s to an Azure Storage Account is straight forward, the following command;

Add-AzureVHD -Destination https://storageaccountcontainer -LocationFilePath “C:\Win7.vhd”

You must create your Storage Account and relevant containers prior to running the command. I tried to upload a VHD, which failed to complete successfully this was due to a network connectivity issue on the client uploading the VHD file.

I tried to run the command again and received the following error “An upload is already in progress on machine HOSTNAME with process ID” even though I knew it had failed.

The fix here was to manually delete the partially uploaded VHD file. This can be done using the Web-GUI if you enter the Container that the original VHD select it and Delete the disk.

Run the Azure command again and the VHD should begin uploading again. 

Configuring Multi-Site VPN for Azure vNet's

It is possible in Microsoft Azure to configure your vNet to terminate multiple IPsec VPN connections to enable you to create mesh-like connectivity between your on premise sites and Azure.

In my example I have two on premise networks Network_A and Network_B, it is very important if you have two networks sharing a vNet gateway that they do not have overlapping address spaces.

            Network_A 10.25.1.0/16 (10.25.0.1 – 10.25.255.254)

            Network_B 10.22.1.0/16 (10.22.0.1 – 10.22.255.254)

The overlaps must also include the Gateway subnet, as this is a network in itself.

One thing to note if you are provisioning your vNet for the first time, is that for a multi-site VPN to be possible your Azure Gateway must be configured to use Dynamic Routing. If you are like me and you are re-configuring an existing gateway you can delete it and recreate it with a Dynamic Routing gateway. You will need to re-run your VPN script on your firewall though, as the Azure Gateway will be assigned a new IP.

Dynamic Routing is also known as Route-based in the context of networking. It is important to ensure your VPN devices support route-based routing. The following link can be referenced to check this;

https://azure.microsoft.com/en-gb/documentation/articles/vpn-gateway-about-vpn-devices/

Before you start your configuration, confirm your VPN devices at each of the sites can support route-based routing.  Surprisingly devices such as Cisco ASA’s and Palo Alto PAN-OS devices are not compatible- they only support policy-based VPNs.

Confirm you want to create a new Azure Gateway. It takes anywhere between 10-30 minutes to provision a new Azure Gateway so be patient.

Before you export your network configuration to XML you must first define the additional local networks that are being used at your other on premise sites.

Enter a descriptive name for the new network, if you forget to do this before you export the network configuration to XML you will be an XML validation error when you do the re-import.

Define the Address Space being used within the site. It is best to ensure your Address Spaces to not overlap, this is an easy mistake to make if you enter a "catch-all" mask to encapsulate the entire Class A, B or C subnet. A work around for overlaps if you do use general network ranges is to manually enter in the exact network ranges used in each of the on premise networks.

At this point you should have all the on premise networks defined under the Virtual Network.

The next step is to export the network configuration to XML. This can be done using the GUI.

You then have to edit the XML file to include your additional Local Networks that have been defined within your vNet.

If you export the network configuration before you define the Local Networks they will not be defined as <LocalNetworkSites> and you will receive a general XML validation error.

In my example script below, I have two Local Networks "Network_A" and "Network_B". The XML should be set like it is highlighted below (the variable is the local network name).

<Gateway>     <ConnectionsToLocalNetwork>       <LocalNetworkSiteRef name="Network_A"><Connection type="IPsec"   /></LocalNetworkSiteRef>       <LocalNetworkSiteRef name="Network_B"><Connection type="IPsec"   /></LocalNetworkSiteRef>     </ConnectionsToLocalNetwork> </Gateway>

Save the XML file and return to the Azure Portal. Use the New wizard and select Network Services, Virtual Network and Import Configuration.

Browse for the edited XML file and click Next. The XML is validated before the import can complete, if Azure does detect a syntax error in the code it will state exactly what line is causing the problem.

Once the import is completed, the next step is to import the configuration script automatically generated by Azure. This script should be imported into each of VPN devices. It is important to read and understand the configuration script, as there are some assumptions made by Azure.

For example, the default configuration script creates an Internet Key Exchange (IKE) policy with a number of 10, if you have used this for another VPN tunnel you could cause a conflict on your firewall which could cause an outage.

Multi-Site VPN’s with Microsoft Azure and Hardware VPN Concentrators (Cisco ASA's)

Although it’s possible to terminate multiple VPN (multi-site) into a single Azure vNet, there are some limitation around the VPN hardware you use to do this. The following document from Microsoft outlines all of the supported VPN devices to work with Azure.

https://azure.microsoft.com/en-gb/documentation/articles/vpn-gateway-about-vpn-devices/

If your project requires multi-site VPN, the important column to review here is the Route-based, in short if you want to terminate multiple VPN’s into a single vNet your VPN device must support Route-based.

This is somewhat confusing as Route-based also means Dynamic Routing

·         Static Routing = Policy-based

·         Dynamic Routing = Route-based

The difference between the two is, Policy-based routing encapsulates and encrypts traffic and then forwards it out a specific interface according to an Access Control List. Route-based routing on the other hand forms dedicated tunnels with a neighbouring VPN device and forwards all of the traffic across this tunnel.

The Microsoft documentation to create a multi-site VPN states that the Azure vNet Gateway must be created as a Dynamic Routing gateway, or in other words a Route-based gateway.

https://azure.microsoft.com/en-gb/documentation/articles/vpn-gateway-multi-site/

This is to allow multiple VPN connection to be terminated into the vNet. If you are using the Cisco ASA for example this will not work, if you check the supported devices list above. The Cisco ASA does not support Route-based routing. In reality if your vNet is configured to use Dynamic Routing and you try and connect it to a Cisco ASA it simply does not work.

In short this basically means that if you have two sites with Cisco ASA’s you cannot create a multi-site VPN to Azure. There is a very small amount of hardware supported for such a topology by Microsoft.

Unless you are running one of the following you basically can’t do multi-site VPN to Azure (these are the only supported device for multi-site VPN);

·         Checkpoint Security Gateway

·         Cisco ISR, ASR

·         Dell SonicWALL

·         Fortinet

·         Juniper SRX, J-Series, ISG, SSG

·         Windows RRAS

The “work around” that doesn’t work

There are a number of forums on the Internet listing this exact problem with ASA’s, Palo Alto’s etc. and some people claim the ingenious workaround is to;

·         Create two separate vNet’s (one for each site you want a VPN from)

·         Create two VPN’s from each of the sites, terminating into their own vNets

·         Create a vNet to vNet VPN between the two separate vNets

I must admit before I did some research on this I did think this could be an option, but again if you review some of the Azure documentation you will notice that for any vNet to vNet VPN you must also create your Gateway using Dynamic Routing.

 

 

 

The bottom line

 

In summary as of January 2016, if you are running a Brocade, Citrix, Palo Alto, WatchGuard, F5, Barracuda or Cisco ASA firewall you cannot create a multi-site VPN to an Azure vNet.

Azure PowerShell "Your Azure credentials have not been set up or have expired, please run Add-AzureAccount to set up your Azure credentials"

Azure PowerShell throws the error "Your Azure credentials have not been set up or have expired, please run Add-AzureAccount to set up your Azure credentials" when you try to run a PS command against an Azure service. This is using both the Get-AzurePublishSettingsFile and the Add-AzureAccount methods to authenticate to the Azure tenancy.

This is because there are a number of stale entries related to your account cached on the administrative device you are using, to display all of the cached accounts use the following command;

Get-AzureAccount

The following command can be used to delete all of the cached Azure credentials from the device;

foreach ($AzureAccount in (Get-AzureAccount).ID) {Remove-AzureAccount $AzureAccount}

Then try Add-AzureAccount, enter your credentials.

And you should re-authenticate to the Azure tenancy properly. You can now run any commands without being faced with the same error.

ADFS & WAP "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)".

A very common problem when installing WAP servers for ADFS is the error "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)". 

Check the bindings for your ADFS servers http.sys file (only for ADFS 3.0, older versions use IIS). The following command can be used to display the http.sys bindings.

netsh http show sslcert

The fix for me was to add the static binding of 0.0.0.0:443, I have highlighted it below. Without this I was receiving the "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)" error.

To add this static 0.0.0.0:443 binding use the netsh utility, it fails if you do not enter it in stages like it is listed below. The certhash and appid used for this binding should be the same that is used in your ADFS service name binding that is displayed when you use the netsh http show sslcert command.

netsh http add sslcert ipport=0.0.0.0:443 certhash=insert hash appid={insert appid}

Now you can try to re-run the WAP configuration utility. I have found this works best by using PowerShell. The following command can be used to perform this

Install-WebApplicationProxy –CertificateThumbprint certthumbprint –FederationServiceName adfsservicename

You will then be prompted to enter administrator credentials on the ADFS servers. The GUI is misleading at this stage as it states you should use "local administrator credentials", I can confirm this is not the case as you will get an error unless they are domain credentials.

When you run the WAP configuration wizard it is worth knowing that you sometimes have to run the command 2/3 times before it works correctly (without changing anything). This first time I run the command after making the binding change return the error.

"The federation server proxy could not renew its trust with the Federation Service.  

Additional Data 

Exception details: 

The remote server returned an error: (400) Bad Request. 

User Action 

Ensure that the federation server proxy is trusted by the Federation Service. If the trust does not exist or has been revoked, establish a trust between the proxy and the Federation Service using the Federation Service Proxy Configuration Wizard by logging on to the proxy computer."

The message returned from the PowerShell window was "an error occurred validating the SSL certificate". The length of the certificate hash is invalid. Did you forget to add the preceding '0'."

Without changing anything I then rerun the command and it worked correctly.

ADFS Server Farm Join Fails with "ADMIN0011: InvalidRequestFault" & “There was an error doing synchronization. Synchronization of data from the primary federation server to a secondary server did not occur”.

I was recently working on a project with a customer to extend their ADFS server farm to Azure for a passive backup in the event of DR.

When I run the command Add-AdfsFarmNode command ADFS returned an almost programmatic error. The error was “ADMIN0011: InvalidRequestFault” which was not very helpful.

The Event Log’s on the server I was trying to join to the ADFS server farm was logging Event 344 “There was an error doing synchronization. Synchronization of data from the primary federation server to a secondary server did not occur”.

The installation with the Add-AdfsFarmNode fails at 80% at the stage “synchronizing local database”.

The first step was to check the health of the ADFS server farm and to check all the relevant configurations such as SPN’s, DNS etc were configured correctly for an ADFS farm join. The following PowerShell command is handy for checking all these at once;

Test-AdfsFarmJoin

For some reason the command does not run correctly unless you store the ADFS service account in a variable and then call it from the variable, so to run the command you must enter it like this;

$ADFScred = Get-Credentials Test-AdfsFarmJoin –ServiceAccountCredential $ADFScred –PrimaryComputerName SERVER

As you can see everything test successfully, one thing that is worth noting here as there is some conflicting information around, the ADFS service account SPN should be set to host/ not http/. The ADFS wizard should automatically sets the SPN for you.

The next stage in my troubleshooting was to enable ADFS logging, by default Verbose logging is not enabled for ADFS as every transaction such as a new claim being issued would be logged. Although this article relates to ADFS 2.0 it is relevant for 3.0 as well.

http://social.technet.microsoft.com/wiki/contents/articles/1407.how-to-enable-debug-logging-for-active-directory-federation-services-2-0-ad-fs-2-0.aspx

This particular ADFS infrastructure had a number of custom Relaying Party Trusts configured for home-grown applications. At this stage I had to open a support case with Microsoft.

Although I thought I knew ADFS fairly well, I did learn something new about Relaying Party Trusts. A Relaying Party Trust can only have one policy of each type.

PolicyUsage = 0 "Issuance Transform Rules"  

PolicyUsage = 1 "IssuanceAuthorizationRule"  

PolicyUsage = 2 "Delegation Authorization Rule"  

PolicyUsage = 3 "ImpersonationAuthorizationRules" 

PolicyUsage = 4 "StrongAuthenticationPolicy"

By querying the ADFS (WID) database directly Microsoft was able to identify that there was one of the custom Relying Party Trust had stored two entries for IssuanceAuthorizationRule and Delegation Authorization Rule.

To do this run the following PS command;

$stsWMIObject = (Get-WmiObject -Namespace root\ADFS -Class SecurityTokenService) $connection = new-object system.data.SqlClient.SqlConnection($stsWMIObject.ConfigurationDatabaseConnectionString); $connection.Open() $connection.Database $query = @“ SELECT  [ScopeId], [AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyUsage], [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies].[PolicyId] FROM [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies] INNER JOIN [AdfsConfiguration].[IdentityServerPolicy].[Policies] ON [AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyID] = [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies].[PolicyID] ORDER BY [ScopeId],[AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyUsage] ”@ $command = $connection.CreateCommand(); $command.CommandText = $query; $result = $command.ExecuteReader(); $table = new-object “System.Data.DataTable” $table.Load($result) $table

Identify the duplicate entries;

Scope ID                                                                                   Policy Usage                      Policy ID

7ae60ee4-d27b-e511-80c7-00505689310d                                          0   cd983eb1-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          1   3a317185-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          1   63dc770a-fc91-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          2   64dc770a-fc91-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          2   3b317185-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          3   7ee60ee4-d27b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          4   7ce60ee4-d27b-e511-80c7-00505689310d

Match a Scope ID to a Relaying Party Trust to identify which one is giving you problems;

$stsWMIObject = (Get-WmiObject -Namespace root\ADFS -Class SecurityTokenService) $connection = new-object system.data.SqlClient.SqlConnection($stsWMIObject.ConfigurationDatabase ConnectionString); $connection.Open() $connection.Database AdfsConfiguration $query = "SELECT * FROM IdentityServerPolicy.ScopeIdentities"; $command = $connection.CreateCommand(); $command.CommandText = $query; $result = $command.ExecuteReader(); $table = new-object "System.Data.DataTable" $table.Load($result) $table

The Scope ID translates to your Relaying Party Trust. Interestingly enough if you opened the Relaying Party Trust from the ADFS GUI the secondary Policy Usage entry was not visible. Luckily in this case the problematic Relaying Trust Trust was only for testing and it could be removed without much of a drama.

Although if you are experiencing this issue with a trust you do require, the following steps can be taken to resolve the problem

1.      Stop the ADFS service

2.       Export the Relying Party Configuration with " Get-AdfsRelyingPartyTrust -Identifier <url identifier> " 

(The <url identifier> can be any of the identifiers associated with the "Relying Party Trust" in the "[ScopeIdentities] " table)

Disabling the Relaying Party Trust does not work.

3.       From the Primary Server, remove the Relying Party causing the conflict.

4.       Start the ADFS service and attempt an ADFS farm join.

5.       Sync should work now.

Surprising there is no metadata clean up etc. required for ADFS, once you remove (or remove and then re-add) the trust and the duplicate Policy Usage statements are gone everything should sync properly.