Azure PowerShell "Your Azure credentials have not been set up or have expired, please run Add-AzureAccount to set up your Azure credentials"

Azure PowerShell throws the error "Your Azure credentials have not been set up or have expired, please run Add-AzureAccount to set up your Azure credentials" when you try to run a PS command against an Azure service. This is using both the Get-AzurePublishSettingsFile and the Add-AzureAccount methods to authenticate to the Azure tenancy.

This is because there are a number of stale entries related to your account cached on the administrative device you are using, to display all of the cached accounts use the following command;

Get-AzureAccount

The following command can be used to delete all of the cached Azure credentials from the device;

foreach ($AzureAccount in (Get-AzureAccount).ID) {Remove-AzureAccount $AzureAccount}

Then try Add-AzureAccount, enter your credentials.

And you should re-authenticate to the Azure tenancy properly. You can now run any commands without being faced with the same error.

ADFS & WAP "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)".

A very common problem when installing WAP servers for ADFS is the error "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)". 

Check the bindings for your ADFS servers http.sys file (only for ADFS 3.0, older versions use IIS). The following command can be used to display the http.sys bindings.

netsh http show sslcert

The fix for me was to add the static binding of 0.0.0.0:443, I have highlighted it below. Without this I was receiving the "Unable to retrieve proxy configuration data from the Federation Service. Status Code Unauthorized (401)" error.

To add this static 0.0.0.0:443 binding use the netsh utility, it fails if you do not enter it in stages like it is listed below. The certhash and appid used for this binding should be the same that is used in your ADFS service name binding that is displayed when you use the netsh http show sslcert command.

netsh http add sslcert ipport=0.0.0.0:443 certhash=insert hash appid={insert appid}

Now you can try to re-run the WAP configuration utility. I have found this works best by using PowerShell. The following command can be used to perform this

Install-WebApplicationProxy –CertificateThumbprint certthumbprint –FederationServiceName adfsservicename

You will then be prompted to enter administrator credentials on the ADFS servers. The GUI is misleading at this stage as it states you should use "local administrator credentials", I can confirm this is not the case as you will get an error unless they are domain credentials.

When you run the WAP configuration wizard it is worth knowing that you sometimes have to run the command 2/3 times before it works correctly (without changing anything). This first time I run the command after making the binding change return the error.

"The federation server proxy could not renew its trust with the Federation Service.  

Additional Data 

Exception details: 

The remote server returned an error: (400) Bad Request. 

User Action 

Ensure that the federation server proxy is trusted by the Federation Service. If the trust does not exist or has been revoked, establish a trust between the proxy and the Federation Service using the Federation Service Proxy Configuration Wizard by logging on to the proxy computer."

The message returned from the PowerShell window was "an error occurred validating the SSL certificate". The length of the certificate hash is invalid. Did you forget to add the preceding '0'."

Without changing anything I then rerun the command and it worked correctly.

ADFS Server Farm Join Fails with "ADMIN0011: InvalidRequestFault" & “There was an error doing synchronization. Synchronization of data from the primary federation server to a secondary server did not occur”.

I was recently working on a project with a customer to extend their ADFS server farm to Azure for a passive backup in the event of DR.

When I run the command Add-AdfsFarmNode command ADFS returned an almost programmatic error. The error was “ADMIN0011: InvalidRequestFault” which was not very helpful.

The Event Log’s on the server I was trying to join to the ADFS server farm was logging Event 344 “There was an error doing synchronization. Synchronization of data from the primary federation server to a secondary server did not occur”.

The installation with the Add-AdfsFarmNode fails at 80% at the stage “synchronizing local database”.

The first step was to check the health of the ADFS server farm and to check all the relevant configurations such as SPN’s, DNS etc were configured correctly for an ADFS farm join. The following PowerShell command is handy for checking all these at once;

Test-AdfsFarmJoin

For some reason the command does not run correctly unless you store the ADFS service account in a variable and then call it from the variable, so to run the command you must enter it like this;

$ADFScred = Get-Credentials Test-AdfsFarmJoin –ServiceAccountCredential $ADFScred –PrimaryComputerName SERVER

As you can see everything test successfully, one thing that is worth noting here as there is some conflicting information around, the ADFS service account SPN should be set to host/ not http/. The ADFS wizard should automatically sets the SPN for you.

The next stage in my troubleshooting was to enable ADFS logging, by default Verbose logging is not enabled for ADFS as every transaction such as a new claim being issued would be logged. Although this article relates to ADFS 2.0 it is relevant for 3.0 as well.

http://social.technet.microsoft.com/wiki/contents/articles/1407.how-to-enable-debug-logging-for-active-directory-federation-services-2-0-ad-fs-2-0.aspx

This particular ADFS infrastructure had a number of custom Relaying Party Trusts configured for home-grown applications. At this stage I had to open a support case with Microsoft.

Although I thought I knew ADFS fairly well, I did learn something new about Relaying Party Trusts. A Relaying Party Trust can only have one policy of each type.

PolicyUsage = 0 "Issuance Transform Rules"  

PolicyUsage = 1 "IssuanceAuthorizationRule"  

PolicyUsage = 2 "Delegation Authorization Rule"  

PolicyUsage = 3 "ImpersonationAuthorizationRules" 

PolicyUsage = 4 "StrongAuthenticationPolicy"

By querying the ADFS (WID) database directly Microsoft was able to identify that there was one of the custom Relying Party Trust had stored two entries for IssuanceAuthorizationRule and Delegation Authorization Rule.

To do this run the following PS command;

$stsWMIObject = (Get-WmiObject -Namespace root\ADFS -Class SecurityTokenService) $connection = new-object system.data.SqlClient.SqlConnection($stsWMIObject.ConfigurationDatabaseConnectionString); $connection.Open() $connection.Database $query = @“ SELECT  [ScopeId], [AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyUsage], [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies].[PolicyId] FROM [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies] INNER JOIN [AdfsConfiguration].[IdentityServerPolicy].[Policies] ON [AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyID] = [AdfsConfiguration].[IdentityServerPolicy].[ScopePolicies].[PolicyID] ORDER BY [ScopeId],[AdfsConfiguration].[IdentityServerPolicy].[Policies].[PolicyUsage] ”@ $command = $connection.CreateCommand(); $command.CommandText = $query; $result = $command.ExecuteReader(); $table = new-object “System.Data.DataTable” $table.Load($result) $table

Identify the duplicate entries;

Scope ID                                                                                   Policy Usage                      Policy ID

7ae60ee4-d27b-e511-80c7-00505689310d                                          0   cd983eb1-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          1   3a317185-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          1   63dc770a-fc91-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          2   64dc770a-fc91-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          2   3b317185-d47b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          3   7ee60ee4-d27b-e511-80c7-00505689310d

7ae60ee4-d27b-e511-80c7-00505689310d                                          4   7ce60ee4-d27b-e511-80c7-00505689310d

Match a Scope ID to a Relaying Party Trust to identify which one is giving you problems;

$stsWMIObject = (Get-WmiObject -Namespace root\ADFS -Class SecurityTokenService) $connection = new-object system.data.SqlClient.SqlConnection($stsWMIObject.ConfigurationDatabase ConnectionString); $connection.Open() $connection.Database AdfsConfiguration $query = "SELECT * FROM IdentityServerPolicy.ScopeIdentities"; $command = $connection.CreateCommand(); $command.CommandText = $query; $result = $command.ExecuteReader(); $table = new-object "System.Data.DataTable" $table.Load($result) $table

The Scope ID translates to your Relaying Party Trust. Interestingly enough if you opened the Relaying Party Trust from the ADFS GUI the secondary Policy Usage entry was not visible. Luckily in this case the problematic Relaying Trust Trust was only for testing and it could be removed without much of a drama.

Although if you are experiencing this issue with a trust you do require, the following steps can be taken to resolve the problem

1.      Stop the ADFS service

2.       Export the Relying Party Configuration with " Get-AdfsRelyingPartyTrust -Identifier <url identifier> " 

(The <url identifier> can be any of the identifiers associated with the "Relying Party Trust" in the "[ScopeIdentities] " table)

Disabling the Relaying Party Trust does not work.

3.       From the Primary Server, remove the Relying Party causing the conflict.

4.       Start the ADFS service and attempt an ADFS farm join.

5.       Sync should work now.

Surprising there is no metadata clean up etc. required for ADFS, once you remove (or remove and then re-add) the trust and the duplicate Policy Usage statements are gone everything should sync properly.

Query Windows for a specific Windows Update (KB)

I was recently working with MS Support and was asked to confirm if a specific KB patch was applied to my problematic servers. Instead of scrolling through the GUI's list of installed updates, hoping to notice the KB you can use the following command.

wmic qfe hotfixid | find "KBxxxxxx"

Simple but handy ;)

First look Azure File System Configuration

Just announced is the Microsoft Azure File System (https://azure.microsoft.com/en-gb/blog/azure-file-storage-now-generally-available/) which allows users to present Azure storage over the SMB protocol, much like what has been used in the enterprise for many years.

Full details can be found in the link above.

Data is stored in the following structure on Azure. Account\Share\Directory\File. 

The Azure File System is build upon the same technology as the existing Azure storage architecture which includes blobs, containers etc. Therefore Azure File System is backed by all of the same SLA's and guarantees for durability, resiliency and availability as you would have come to expect.

In this post I am going to outline the steps required to create a Storage Account with File Share, and how to present that share to an Azure instance. Firstly login to the Azure Preview Portal. The Storage Account must be an ordinary Storage Account, and not one labelled "Classic", depending on when your subscription was first set up will determine if you even have access to Classic VM's and Storage.

If you do not have a Storage Account click the Add button.

Enter a name for the Storage Account, this must be globally unique. Configure the Resource Group and Regional settings to suit your requirements and click Create.

Once the Storage Account has provisioned click on it, and expand Services. You will notice there is now an option for "Files", if your Storage Account is "classic" the option for Files is not present. Click the Files tile.

Click the + File Share button.

Name the File Share, this only has to be unique to your Storage Account. Also give the file share a quota if you want to. Click Create.

Now click on the File Share, and you will be faced with the following options. As per the Azure file structure, the next step is to create a Directory inside of the File Share. Click the + Directory button.

Once the directory is created, you then have the ability to upload files into the directory. 

Now that the Storage Account is configured with a Share, Directory and sample file. You can now map a client device to the SMB Share, at the time of writing the following versions of Windows with the outlined version of the SMB protocol were supported. You can connect certain versions of Linux/Unix to SMB Share, further details can be found in the article above.

In order to authenticate the Azure instance you are using to connect to your SMB Share, you must have the Storage Account access key, you can expose these from the Preview Portal. Copy Key 1 to the clipboard.

cmdkey /add:demotesting.file.core.windows.net /user:demotesting.file.core.windows.net /pass:y/Do/lCNNGkRiqivgtFlX+XXXXXXXXXXXXSbZ6VOuSkpexVGYCRaGa1x0xBaNtg12FzCQaKb/DhQ==

net use Z: \\demotesting.file.core.windows.net\smbshare /u:demotesting y/Do/lCNNGkRiqivgtFlX+Wt9aJ2B2lkRjIXXXXXXXXXXkpexVGYCRaGa1x0xBaNtg12FzCQaKb/DhQ==

Azure VM Active Directory Move NTDS Database Location

When extending an Active Directory to Azure, I stupidly forgot to install the Active Directory Database and Logs on to a dedicated Azure data disk. All of the Active Directory documentation that relates to Azure recommends doing this for integrity purposes

https://azure.microsoft.com/en-gb/documentation/articles/active-directory-new-forest-virtual-machine/

The AD was installed and configured, so I had to gracefully move the DB and Logs files from there default locations. If you have not already installed Active Directory on your Azure VM's, simply change the installation location when you are going through the wizard.

To create a new data disk in Azure for your VM, click the VM in the Preview Portal, and click All Settings.

Click on Disks.

Click the Attach New icon.

Select a Storage Container in which to provision the data disk into, label the disk something descriptive and also set a size. You should probably do some investigation into how big your Active Directory database is before you do this.

Its important Host Caching is set to none. Click Create.

Once the operation to create an attach the new data disk is complete, use Disk Management (or diskpart) to online, initialize and format the disk. Please note, as the Active Directory database requires some special permissions, the disk must be formatted in NTFS.

On a Domain Controller open an Administrative Command Prompt. The following command will stop the AD DS service, please hit Y to confirm that the AD DS service can be stopped along with the other dependent services.

net stop ntds

The following commands (and sub-commands of NTDSUTIL) should move your AD database and logs without any problems. You do not have to pre-create the folder on the new disk.

ntdsutil activate instance ntds file move db to E:\NTDS move logs to E:\NTDS integrity quit shutdown –r –t 1 (optional)

After the reboot the AD files should now be re-homed.

Azure External Load Balancer "Failed to join virtual machine to load balanced set. The operation failed Port 443 is already in use by one of the endpoints in this deployment."

When you try to create an external/public Azure Load Balancer for HTTPS traffic, you recieve "Failed to join virtual machine to load balanced set. The operation failed Port 443 is already in use by one of the endpoints in this deployment. Ensure that the port numbers are unique across endpoints within a deployment."

The key thing to understand here is that "deployment" actually means the Cloud Service, therefore this error is saying another application or service is currently using TCP port 443 within your current Cloud Service.

Problem spotted, if you click on your Cloud Service and review what is listed under Input Endpoints you will probably find there is a VM instance using TCP 443 behind your Cloud Service public IP address.

The best way to fix this (and only from a GUI) is to use the Preview Portal. Click on the VM that is listed under the Cloud Service as using port 443 and click All Settings then Load Balanced Sets.

As you will see I currently have two Load Balanced Sets, one for Internal traffic and one for External traffic. The public load balanced set is causing the issue here for me, the internal LB was created only a few minutes ago to load balance the internal "tier" of ADFS. The external LB is going to point to WAP endpoints on separate VM's.

So I clicked on the Public LB and it was currently using the Cloud Services IP on TCP 443.

After selecting the Public LB you must click Leave to remove the old endpoints from the LB, if this is the last endpoint within an LB the LB will be automatically deleted. As this was a stale LB instance that I had created weeks ago for testing I went ahead and removed it.

Hopefully deleting the load balanced set works first time, I believe some people have experienced problems doing this from the preview portal, and have had to revert to Powershell to complete the operation.

Now lets try re-creating our Public/External Load Balanced Set with the Cloud Service public IP and TCP port 443. Click the VM's you want to provision into the LB set and click All Settings/Load Balanced Sets, in my case this was the first of my WAP servers that were going to be internet facing. 

Click Join.

If you had the same problem as me and you have successfully removed everything that was conflicting the operation should complete successfully.

Azure AD DirSync Unable to update this object in Azure Active Directory, because the attribute [FederatedUser.UserPrincipalName], is not valid. Update the value in your local directory services.

You have your Active Directory configured for SSO with Office 365, therefore the DirSync (or whatever it’s called this week) tool configured. All of your user accounts are populated into Azure AD, with the exception of one.

In your environment you have a number of UPN Suffixes for various reasons, the proper object is throwing the following error from the DirSync (FIM 2010 R2) interface “Unable to update this object in Azure Active Directory, because the attribute [FederatedUser.UserPrincipalName], is not valid. Update the value in your local directory services.”

On the Connector Space Object Properties page, the error code 0x8023134a DataValidationFailed is also flagged.

This error is logged by DirSync every time the Export Management Agent operation is run, which happens by default every 3 hours. The user account in question recently had it’s UPN Suffixes changed via a full installation of Forefront Identity Manager that is synchronizing identities from Novell eDirectory.

This is a fairly well known issue to Microsoft as they have outlined in this workaround article;

https://support.microsoft.com/en-us/kb/2669550

What they recommend doing is to connect to the Office 365 via PowerShell using Connect-MsolService.

Set-MsolUserPrincipalName –UserPrincipalName user1@oldupn.com –NewUserPrincipalName user1@newupn.com

This command in theory should work, as all it is doing is altering the UPN Suffix, which cannot be done using the Office 365 Web-UI. Unfortunately the gotcha here for me was that the command failed to run, I had to run it twice. Once to change the user account to my .onmicrosoft.com domain, and then again to update the UPN back to one of the internal federated domains.

Set-MsolUserPrincipalName –UserPrincipalName user1@oldupn.com –NewUserPrincipalName user1@domain.onmicrosoft.com

Set-MsolUserPrincipalName –UserPrincipalName user1@oldupn.com –NewUserPrincipalName user1@newupn.com

Once I did this it completely correctly, and I could see the changes from the Web-UI.

Azure Cloud Service Connecitivity "Remote Desktop can't find the computer "name.cloudapp.net". This might mean that "name.cloudapp.net" does not belong to the specified network.

Azure Cloud Service Connecitivity "Remote Desktop can't find the computer "name.cloudapp.net". This might mean that "name.cloudapp.net" does not belong to the specified network.

When you try to connect to a VM in a Cloud Service (name.cloudapp.net) it was working until I provisioned an additional 6 VM's into the same Cloud Service. I then tried to RDP to any of the other instances to see if it was a particular VM, or the Cloud Service itself.

The fix was easy login to Azure using the Preview Portal, expand the VM in question and click on the Reset Remote Desktop Settings, and click Yes to confirm. Then reboot the instance, and you should be able to RDP.

Active Directory Ports/Protocols between Firewall's/L3/4 Boundaries

If for whatever reason you have to have Domain Controllers separated by firewalls you will require the following ports and protocols allowed between the DC's to ensure you can A.) join the domain and B.) maintain a healthy Active Directory with replication etc.

LDAP TCP-in - 389
LDAP UDP in - 389
LDAP for Global Catalog TCP in - 3268
NetBIOS name Resolution UDP in - 138
SAM/LSA TCP in - 445
SAM/LSA UDP in - 445
Secure LDAP TCP in -  636
Secure LDAP for Global Catalog TCP in - 3269
W32Time NTP UDP in - 123
RPC - RPC Dynamic
RPC Endpoint Mapper
DNS - TCP and UDP 53
Kerberos V5 UDP in - 88
Netbios Datagram UDP in - 137

There are multiple ways to have Active Directory extended to none-trusted networks. Since Windows Server 2008 R2 the option to have a Read Only Domain Controller (RODC) in an un-trusted network has proved attractive to business. This ensure that if the un-trusted network is compromised, the RODC only holds a replicated copy of the Active Directory database. Even if the RODC was taken offline and attacked, no "write back" to Active Directory is possible so the impact is minimal. 

Depending on your requirements options such as Active Directory Federation Services (AD FS), DirSync, FIM 2010 R2 and Microsoft Identity Manager 2015 (MIM) can all be good solutions for identity and access management in an Active Directory environment.