How to Backup and Restore Docker EE UCP

Its possible to back and restore Docker EE UCP if you are running this in a production environment. One thing to note, that if you are doing a backup of UCP you must also have a backup of the Docker Swarm configuration. This can be done following this guide:

https://blog.ryanbetts.co.uk/2020/05/how-to-backup-and-restore-docker-swarm.html

The first step is to find your UCP (id):

docker container run --rm \
  --name ucp \
  -v /var/run/docker.sock:/var/run/docker.sock \
  docker/ucp:latest \
  id

This command will push out a string like this 4tkiq0u898qlr0bvehx9g6ek2.

Once you have this id reference we can run the actual backup. Ensure to change any variables which might be different for you, such as the user profile path which is being used as a target for the backup.

docker container run \
  --log-driver none --rm \
  --interactive \
  --name ucp \
  -v /var/run/docker.sock:/var/run/docker.sock \
  docker/ucp:latest backup \
  --passphrase "Password" \
  --id 4tkiq0u898qlr0bvehx9g6ek2 > /home/it/ucp-backup.tar

Now we can run the restore using this command:

docker container run --rm -i --name ucp \

  -v /var/run/docker.sock:/var/run/docker.sock  \

  docker/ucp:latest restore --passphrase "Password" < /home/cloud_user/ucp-backup.tar

You might get faced with errors if you are trying to fix corruption in which case you have to clean up the old configuration before you attempt to restore the backup. The following command can be used to do this:

docker container run --rm -it \

  -v /var/run/docker.sock:/var/run/docker.sock \

  --name ucp \

  docker/ucp:latest uninstall-ucp --interactive

Deploying the Docker Universal Control Plane on Ubuntu (EE 19.03)

I was recently building up a Docker EE lab to go deeper in some of my weak areas after passing the Docker Certified Associate certification. The following commands walk you through how to install the Univeral Control Plane on a newly created Ubuntu 18.04 server.

docker image pull docker/ucp:latest

PRIVATE_IP=172.16.1.212

docker container run --rm -it --name ucp \
  -v /var/run/docker.sock:/var/run/docker.sock \
  docker/ucp:3.1.5 install \
  --host-address $PRIVATE_IP \
  --interactive

In this particular example I set a variable of PRIVATE_IP and for clarity this was set to the host of my Docker Swarm manager (leader) node. I think most of the time it pulls this automatically but I found this to be more stable if you hard code it.

Please note there is another switch which must be used if you are deploying this on Azure IaaS VMs. 

The following switch should be added --cloud-provider Azure

You may also want to review this article, for another pre step which should be done before deploying UCP on Azure VMs.

https://blog.ryanbetts.co.uk/2020/05/installing-docker-ucp-fails-with-unable.html

How to install and configure SSH on Ubuntu Server 19.03

If you elect not to install OpenSSH at the installation stage of Ubuntu Server, you must install and configure it once the server is deployed. The following commands can be used to achieve this:

sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh

Installing Docker UCP fails with "unable to run install step "Deploy Kubernetes API Server": unable to reconcile state of Kubernetes API"

I was deploying Docker EE with the Universal Control Plane to Azure VM's and was hit with the error "unable to run install step "Deploy Kubernetes API Server": unable to reconcile state of Kubernetes API" late on in the process to deploy the UCP.

The first step to fix this was to scrub all of the half baked deployment of UCP, this can be done using the following commands:

docker swarm leave --force
docker rm $(docker ps -a -q)
docker system prune -a
docker secret rm ucp-auth-key

The reason the installation fails here is because of an empty JSON file which is missing from /etc/kubernetes - the /kubernetes folder is actually not there either.

Use the following commands to create /kubernetes along with the empty azure.json file which will resolve the installation error:

cd /
cd etc
mkdir kubernetes
sudo chmod a+rwx /etc/kubernetes
sudo echo "" > azure.json

Hopefully this was helpful. 

Docker Universal Control Plane install fails with "unable to verify that no UCP components already exist"

You may hit this error if you have had a failed attempt at deploying the UCP and there is still orphaned references to it on the Docker host. I forgot to remove the old UCP Docker volumes so when I tried to deploy the latest verion of UCP this happened.

"FATA[0001] unable to verify that no UCP components already exist: the following volumes appear to be from an existing UCP installation and must be removed before proceeding with a new in                stallation: ucp-kv ucp-metrics-data ucp-auth-api-certs ucp-auth-store-certs ucp-client-root-c                a ucp-cluster-root-ca ucp-controller-server-certs ucp-auth-worker-certs ucp-auth-worker-data          
    ucp-kv-certs ucp-auth-store-data ucp-controller-client-certs ucp-node-certs"

Simply use the docker volume prune command to get rid of all the old volumes which are not associated to a running container. Tread with caution if you are using a production system.

How to install Docker Enterprise Edition on Ubuntu 18.04 LTS

To get a trial of Docker Enterprise Edition sign into Docker Hub and get your unique URL.

Set the URL as a variable. 

DOCKER_EE_URL=https://storebits.docker.com/ee/trial/sub-aa658aaa-ec3b-489p-8f91-20774175a085
DOCKER_EE_VERSION=18.09

curl -fsSL "${DOCKER_EE_URL}/ubuntu/gpg" | sudo apt-key add -

sudo add-apt-repository \
  "deb [arch=$(dpkg --print-architecture)] $DOCKER_EE_URL/ubuntu \
  $(lsb_release -cs) \
  stable-$DOCKER_EE_VERSION"

sudo apt-get update

sudo apt-get install -y docker-ee=5:18.09.4~3-0~ubuntu-bionic

sudo usermod -a -G docker AzureUser

How to Backup and Restore a Docker Swarm configuration set

It is possible to backup a Swarm configuration store, which by default is stored at /var/lib/docker/swarm any Docker host acting as a manager node. 

This directory must be exported or stored externally to ensure you can restore the Swarm configuration in the event of a failure, obviously it would be best to have multiple Swarm managers in the cluster to ensure high availability. However this will not protect against corruption to the configuration.

sudo systemctl stop docker - 1st stop the Docker service so that no new writes are being committed.

sudo tar -zvcf backup.tar.gz /var/lib/docker/swarm - this command outputs the directory to a tar zip.

The taz zip file is stored in the working directory. You could of course setup up an automated job with something like cron to take out the manual intervention required to achieve this. 

sudo systemctl stop docker //stops Docker on the host to allow a restore

sudo rm -rf /var/lib/docker/swarm/* //deletes the existing data in the Swarm directory

sudo tar -zxvf backup.tar.gz -C /var/lib/docker/swarm/ //unzip tar zip to the directory

sudo systemctl start docker //restart Docker


The Docker Swarm config has now been restored to the Docker host. 

Configuring Docker Swarm Hosts with Join Tokens

To configure a new Docker Swarm, its remarkably easy you run the following command on a new Docker host and the Swarm is created:

docker swarm init

A Swarm is of course a cluster of Docker hosts, so the next step after creating a new Swarm is to join manager and worker nodes to the Swarm. Once you run the docker swarm init command the console will display a join-token, which can be used to join a new host to the Swarm. 

It will look something like this. You will see it prepopulates all the details required to successfully join a new Docker host to this Swarm. 

docker swarm join --token SWMTKN-1-26w39jcflglpun070cl0qxwnbqwobwj68e4i1dxdi1w2n3we80-4vftzhty9xqmhmq13tgapnzyq 192.168.1.9:2377

Its important to understand that a reliable network connection is required between all Docker hosts in Swarm. You will also notice port 2377 is used by the manager and worker nodes to communicate, so this port must be open between the two servers. It is unlikely you will hit any problems here if they are on the same network segment. 

From the console of an existing Docker manager node its possible to generate a new join-token. When you generate a new join-token you state whether the new host is going to join the cluster as a manager or a worker node.

For example create a new join-token for a manager node by running

docker swarm join-token manager

or 

docker swarm join-token worker

How to create a new Docker Swarm on Debian-Buster10

A Docker Swarm is a collection of node working together in a cluster. A Swarm can be made up of manager and worker node. As with most clustering solutions the recommendation is to try and avoid having an even number of nodes, to prevent issues with voting rights and cluster quorum. 

The diagram depicts how a Docker Swarm looks. 

The command below should be run on the first node which you want to become a Swarm Manager, its possible (and recommended) to have more than one Swarm Manager. However, when you initialise the cluster the first node becomes the "manager leader" which is basically the node which conducts and manages the orchestration tasks of the cluster.

docker swarm init --advertise-addr "pirate ip of cluster manager node"

The -advertise-addr switch tells the cluster to be provisioned with the correct address. In many examples its possible to leave this out and the docker swarm command assumes the private address of the host anyway. It's more important to be explicit if the host belongs to multiple networks, perhaps in a public cloud provider where the node might have multiple interfaces on different network segments. 

The following command can be run to check to see if the cluster has provisioned correctly. 

docker node ls

After running the docker swarm init command, it will generate string which can be used to join worker nodes to your Swarm cluster. 

Understanding and Configuring the Logging Driver on a Docker Host

Logging Drivers on Docker are pluggable components which all you to access and pull log data out of containers running on a Docker host. Many Logging Drivers exist for Docker, but the default and most common is known as "json-file". This is set as the default on a newly installed Docker host.

It is possible to change the default Logging Driver by editing the following file:

/etc/docker/daemon.json

Please note that a new file will be created if this does not already exist with custom settings 

sudo vi /etc/docker/daemon.json

Add the following code block in JSON format to set the Logging Driver to "json-file" for the entire Docker host. 

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "15m"
  }
}

The "log-opts" switch is another way for us to control the Logging Driver, and the subsettings depend on the driver itself. Above we have set "log-opts" to "max-size" which sets a max log file size to 15MB's.

Once you have made the code changes to the daemon.json file, commit and save the file. You must restart the Docker services before the changes will be pharsed. To do this run the following command:

sudo systemctl restart docker

Depending on whether you are running Docker EE or CE edition will depend on the Logging Drivers available to you. 

In Docker EE you have the following drivers available: 

• syslog
• gelf
• fluentd
• awslogs
• splunk
• etwlogs
• gcplogs
• Logentries

However in Docker CE edition the options are more limited:

• local
• json-file
• journald

The information above demonstrates how to set the Logging Driver at a global level across the entire host. It is possible with Logging Drivers to use different drivers on a per-container basis.

To do this use the -log-driver switch with the docker run command. For example:

docker run --log-driver json-file --log-opt max-size=50m hello-world

Audit Docker Security with CIS Benchmark Script

The following Git Hub repo includes a script which checks against dozens of common best practices related to securing Docker. 

https://github.com/docker/docker-bench-security

It is worth running this to get an understanding of your Docker environments security posture. 


Step 1: Clone the repo on your Docker host

git clone https://github.com/docker/docker-bench-security.git

Step 2: cd to the directory

cd docker-bench-security

Step 3: run the script (this runs the entire script)

sudo ./docker-bench-security/sh

Step 4: review the output

It is also possible to target certain aspects of a Docker deployment, such as doing a targeted scan of the Docker host configurations.

To do this run the script with the following switches:

This command runs checks againest the Docker hosts itself. 

sudo ./docker-bench-security.sh -c host_configuration

The other targeted tests are shown below. Just substitued the test name into the above command. 

Windows Virtual Desktop: Create Custom Windows 7 Image for WVD

Windows 7 is still very much alive in the enterprise, regardless if today is the last day Microsoft officially supports the legacy operating system. Announced in September 2019, Windows Virtual Desktop allows customers to run a fully managed VDI environment from Azure. The underlying technology is similar to Remote Desktop Service we have had in Windows Server since 2012 but the new Azure service abstracts all of the actual components which make it work. In other words you no longer have to manage RD Session Hosts etc with WVD.

Customers who have not yet managed to migrate away from Windows 7 are effectively exposed as new security updates will no longer be released for Windows 7 (unless you have a side agreement with Microsoft). However, when WVD was announced it was made clear that Windows 7 as an operating system would be supported in WVD deployments. In addition to this Microsoft are providing free Extended Security Updates (ESU) to customers running Windows 7 as part of their WVD deployment.

Although it is very straight forward to get a Windows Virtual Desktop deployment up and going a few steps are required if you would like you WVD Host Pools to spin out VM's running Windows 7. By default, it is only possible to create Host Pools with later operating systems such as Windows 10 Enterprise Multi-Session.

The process to make Windows 7 available to your WVD users is straight forward. You must create a customer "managed" image in Azure to be used as a reference template when creating a new WVD Host Pool.

The easiest way to do this is to deploy a new Windows 7 VM directly into Azure, install your apps and do any customisation, then convert it to an image. It is possible to do something similar if for whatever reason this won't work for you. The process would be to build a reference Windows 7 image on premise, probably on Hyper-V and run through the steps outlined in this guide https://docs.microsoft.com/en-us/azure/virtual-desktop/set-up-customize-master-image

Step 1: Create a new Windows 7 VM

This is very simple, use the poral to provision a new Virtual Machine, selecting Windows 7 Enterprise as the source Image.

Windows 7 Enterprise is not in the quick drop list of operating systems, if you search for it though you will find that there is a single image available. 

Step 2: Install Apps and Make Customisations

Once the VM has deployed, login to it and install any applications, or make any customisations. It is worth noting that some applications need further configuration to make them user-ready at first launch, this would be addressed on a per-application basis.

Step 3: Run SysPrep to Generalise the VM

The SysPrep tool has been around in Windows since before WDS, I think it first appeared in Windows Server 2003 with the introduction of Remote Installation Service (RIS). Anyway it has not changed since, it is a tool used to generalise (strip any machine specific metadata away from a Windows installation) to aid Windows imaging. If you fail to SysPrep you VM images you will have nothing but boot problems and instability.

Open an elevated Command Prompt and cd C:\Windows\System32\sysprep

Select the options shown below Enter System Out-of-Box Experience (OOBE) and ensure Generalize is ticked. You also want to ensure the VM is powered off after this process has completed, if not it will boot and begin detecting all the hardware and run the OOBE process.

The SysPrep process should complete in a few minutes and leave you with a clean, generalised VM ready to be converted into an image. 

Once the process has completed you will see the VM has entered the Stopped (Deallocated) state from the Azure Portal.

Step 4: Convert Windows 7 Image to Azure Image

Once we have the template ready click into the Virtual Machine from the Portal and click the Capture button. This will begin the process of converting the VM to an Azure Managed Image. 

The wizard will walk you through creating a new image, ensure that you give it a descriptive name. The name of the Azure Managed Image will be required when creating your WVD Host Pool along with it's Resource Group. You will notice the option to delete the VM once the template has been created, in this example I have chosen to select this option. However, in production you might want the template in place so that you can do some online servicing of the image as time goes on. 

Once the process completes you will have a new image available from the Portal. You can view all images if you search for Images in the global search bar.

Step 5: Create Windows Virtual Desktop Host Pool from Windows 7 Managed Image

The next step is to create WVD Host Pool, this can be done by searching for WVD in the poral and selecting Create.

You must have a number of infrastructure components in place before you can deploy a WVD Host Pool. This includes a WVD tenant, AD DS domain, AAD tenant with M365 licenses with all the associated networking. When you create a WVD Host Pool the creation must be able to join your WVD Session Hosts to an Active Directory domain.

From the wizard I have labelled the Host Pool "win7-personal" and selected the type as Personal. This outlines that users will maintain a 1:1 mapping with a dedicated Azure VM running Windows 7, which is created from this template. It is also possible to create Pool WVD Host Pools but there is limited value in doing this with Windows 7. By default, Windows 7 can only support 2 concurrent login sessions. This has been true of all desktop operating systems from Microsoft until the release of Windows 10 Enterprise Multi-Session, with is designed to allow pooled desktops to be created from a Windows 10 image.

On the Virtual Machine Settings tab, instead of taking the default of selecting a Gallery image, click on Managed Image. This will then present two new fields for the Azure Managed Image name, along with the Resource Group that the image is in. 

Azure Network Watcher: the default "NetworkWatcherRG" Resource Group is just irritating (how to change it)

If you are like me and insist on keeping your Azure subscriptions nice and tidy, with consistent naming of resource groups the default “NetworkWatcherRG” resource group is bound to annoy you.

Network Watcher is a region level service which can be used to troubleshoot network connectivity between your Azure resources. If you do not want the “NetworkWatcherRG” resource group making things look untidy, the trick is to create the instance of Network Watcher manually using Azure CLI or PowerShell.

The example below creates a Network Watcher instance for UK South in a designated resource group.

az login

az network watcher configure --resource-group "rb-core-rg-1" --locations uksouth --enabled

How to publish RemoteApps on Windows Virtual Desktop using Powershell

To publish RemoteApps on Windows Virtual Desktop, you must create a dedicated Host Pool for RemoteApps. It is not possible to coexist RemoteApps with full desktops. This was the same in legacy Remote Desktop Services. I have already created a new Host Pool using the Portal called “hostpool2”, please note for RemoteApps you must create the host pool with a server operating system.

Use the following command to authenticate to the WVD tenant.

Add-RdsAccount -DeploymentUrl https://rdbroker.wvd.microsoft.com

Use the following command to create a new RemoteApp Group.

New-RdsAppGroup -TenantName "Tenant Name" -HostPoolName "hostpool2" -Name "Demo"

Use the following command to display all the available applications on the host pool VM’s. This command displays 3 variables which are required for the New-RdsRemoteApp command.

1 – App Name: the name of the application

2 – Icon Path: the icon path on the local system to be displayed as part of the published app.

3 – File Path: the raw file path of the exe of the app to be published.

Get-RdsStartMenuApp -TenantName "Tenant Name" -HostPoolName "hostpool2" -AppGroupName "Demo"

Take the info which was displayed in the last step to complete the New-RdsRemoteApp command.

New-RdsRemoteApp -TenantName "Tenant Name" -HostPoolName "hostpool2" -AppGroupName "Demo" -Name "Calculator" -FilePath "C:\windows\system32\win32calc.exe" -IconIndex "0"

Use the Add-RdsAppGroupUser command to grant permissions to users or groups.

Add-RdsAppGroupUser -TenantName "Tenant Name" -HostPoolName "hostpool2" -AppGroupName "Demo" -UserPrincipalName "user1@domain.com"

How to clean up Windows Virtual Desktop tenant deployment using PowerShell

If you have been experimenting with Windows Virtual Desktop you may notice that old tenants that were created still show under the WVD Tenant management portal. These show even if the Host Pool has been deleted from the Portal. The following set of commands can be used to delete the tenant so that it no longer showers in the management portal.

Use Get-RdsSessionHost to find the name of the old Session Hosts.

Get-RdsSessionHost -TenantName "Windows Virtual Desktop Betts" -HostPoolName "host-pool1"

Use Remove-RdsSessionHost to delete the Session Hosts, this needs done even if you have deleted the Host Pool from the Portal.

Remove-RdsSessionHost -TenantName "Windows Virtual Desktop Betts" -HostPoolName "host-pool1" -Name "hoa-wvd--0.domain.com" -Force

Use Remote-RdsHostPool to delete the Host Pool, again this need done even if it’s been deleted from the Portal.

Remove-RdsHostPool -TenantName "Windows Virtual Desktop Betts" -Name "host-pool1"

Use Remove-RdsTenant to delete the old tenant so that it no longer shows in the WVD management portal.

Remove-RdsTenant -Name "Windows Virtual Desktop Betts”

Windows Virtual Desktop - New-RdsTenant throws error "User is not authorized to query the management service." due to Azure AD permission error.

When you try to create a new Windows Virtual Desktop tenant you run the command

New-RdsTenant -Name "Windows Virtual Desktop Betts" -AadTenantId "xxxxx" -AzureSubscriptionID "xxxxxx"

And are faced with the error "New-RdsTenant : User is not authorized to query the management service.". This is due to a permission configuration problem on Azure AD. 

Before you get to the stage of creating a new WVD tenant you must complete the consent process to grand access to your AAD tenant, this can be done here https://rdweb.wvd.microsoft.com/

Once it is done you will notice two new objects under Enterprise Applications for Windows Virtual Desktop, click on the first one. 

You must add a new user account with TenantCreator permissions before you can create a new WVD tenant. Please note that the Global Admin account for the directory does not work, it must be TenantCreator. 

Once you have a TenentCreator, ensure you authenticate to your directory at the Add-RdsAccount stage using this account before you attempt to create a new WVD tenant. This is where you will be faced with "User is not authorized to query the management service." even if you use a Global Admin account. 

Setting Azure variables in Windows for Terraform authentication

It is possible to store the environment variables for your Azure in the Windows profile of the machine you are using Terraform from. This prevents the need to store sensitive variables in your Terraform code. The first step is to create new Environment Variables under Windows, in this example I'm using Windows 10 Enterprise. 

The important thing here is what you label the variables, the Terraform program looks inside the Windows profile for the prefix "TF_VAR_" and the suffix must be exact to match the syntax of what Terraform is expecting for example in Azure Active Directory the service principal is called an "application id", Terraform does not understand this as it's looking for "client_id".

Azure Value	Terraform Expects	Windows Variable String
Application ID	client_id	TF_VAR_client_id
Client Secret	client_secret	TF_VAR_client_secret
Tenant ID	tenant_id	TF_VAR_subscription_id
Subscription ID	subscription_id	TF_VAR_tenant_id

Use the following Azure CLI code to authenticate to Azure using the variables:

az login --service-principal -u %TF_VAR_client_id% -p %TF_VAR_client_secret% -t %TF_VAR_tenant_id%