Sunday 31 August 2014

Configuring StarWinds iSCSI SAN (Free) Edition for a VMware vSphere 5.5 Nested Lab

StarWinds offer a free iSCSI SAN that can be installed on a Windows Server to provided an iSCSI target for VMware ESXi 5.5 hosts. This can be good if you are labing vSphere, and you need some shared storage. The free edition does have some limitations, the most restrictive is probably the 128GB limit on your LUN's. This is OK for a lab, as you probably would not use a software-based iSCSI target in a production environment.
Once you have downloaded the iSCSI executable, install it on a Windows Server. As my environment is entirely nested I added another Virtual Disk to my vCenter Server (again you would never do this in production) to store the iSCSI Image File.

Click Image File.

Now use the ... button from the wizard, and point to your partition that is going to host the Image File. You must name the Image File, and ensure you have .img extension on the file. I have set the size to 75GB, click Next.

Tick Asynchronous Mode and click Next.

Select Write-back Caching and then click Next.
As this is a new installation choose Create a New Target from the drop down and give the Target Alias a name. You should ensure Allow Multiple Concurrent iSCSI Connections (Clustering) is ticked, if you are planning to connect up multiple ESXi hosts. Click Next.

Copy and paste the full target name to Notepad as it will be required in vSphere. Click Finish.

Click on the Devices menu and select the ImageFile that you just created. Right click on it and select Attach to Target...

Click Attach to the Existing Target click Next.

Select the target that you created, I named mine iSCSI. Click Next.

Click Finish.

The StarWinds iSCSI software is now configured, open the VMware vCenter console and select one of your ESXi hosts. Click on the Configuration tab, and click on Storage Adapters. Click Add.

You will be asked to Add Software iSCSI Adapter click OK. Once the software iSCSI HBA is installed on an ESXi host it cannot be removed.

Once the operation completes right click on the Software iSCSI HBA and select Properties.
Click on the Static Discovery tab, and click Add.

Enter the IP address of your iSCSI server, provided you have not changed the port it should be 3260, and copy and paste the iSCSI Target Name from your Notepad document. Click OK.

vSphere will then confirm that it is now going to rescan all of the HBA's, click OK.

Now click on the Storage tab, and use the Add Storage... button.

Select Disk/LUN and click Next.

Provided the software iSCSI adapter is configured correctly you should see ROCKET iSCSI Disk select this and click Next.

Depending on your requirements click VMFS-5. Version 5 offers many improvements over VMFS-3 for full details using this VMware Documentation link click Next.

Click Next.

Name the Datastore and click Next.

Choose Maximum Available Size and click Next.

Click Finish on the confirmation page.

The datastore should now appear as an available storage location for that host.

Right click on Cluster01 and select Browse Datastore, and you should be able to view the files and folders on the datastore.

If you return to the StarWinds Management Console you should be able to view the active iSCSI Session by clicking on Targets and then iSCSI Sessions.

Saturday 30 August 2014

Certain Users unable to connect via iPhone's to Exchange 2010: "An exception occurred and was handled by Exchange ActiveSync. This may have been caused by an outdated or corrupted Exchange ActiveSync device partnership."

You have a number of users complaining they cannot connect to Exchange ActiveSync using their iPhone/iPad. The standard, remove the account and re-add it trick does not resolve the issue. ActiveSync also appears to be working correctly for everyone else.

On investigation of the issue in the Windows Logs\Application there is a Warning related to Exchange "An exception occurred and was handled by Exchange ActiveSync. This may have been caused by an outdated or corrupted Exchange ActiveSync device partnership." This did not reference the problematic user.

The first stage was to view the ActiveSync DeviceID Associations with the users, this can be done by using the following PowerShell command;

Get-ActiveSyncDevice | ft UserDisplayName, DeviceID

This will display all of the users that have an associated ActiveSync device to their Exchange account. The user that was having problems had two entries.

I then logged into the Exchange ECP from a web browser, if it's enabled in your environment you can normally connect up to it using your namespace followed by /ECP. Then click on the Manage My Organization button and choose Another User....

Try the username of the problematic user, and click OK.

You then click on Phone and Mobile Phones from the left hand side menu. As you can see there is two entries for the iPhone. I selected them both and used the X button to delete the entries.

It will ask you to confirm if you would like to remove the device, click Yes. Please note this will not Wipe the device.

On the problematic iPhone I then tried to re-add the Exchange account and it worked correctly.

Thursday 28 August 2014

Exchange 2010 SP3: After Unexpected Power Outage "Active Manager failed to mount database MailDB1 on server exchsrv.domain.local. Error: An Active Manager operation failed. Error The database action failed. Error: Operation failed with message: MapiExceptionJetErrorRequiredLogFilesMissing: When You Try and Mount the Mailbox Database

After an unexpected power outage you are unable to mount your Mailbox Database in Exchange 2010.  When you try to manually mount the Mailbox Database you receive the error "Active Manager failed to mount database MailDB1 on server exchcas.domain.local. Error: An Active Manager operation failed. Error The database action failed. Error: Operation failed with message: MapiExceptionJetErrorRequiredLogFilesMissing: Unable to mount database. (hr=0x80004005, ec=-543)". Please note I copied the original database file to another location first, and performed all these actions before I did it on my live database, I advise you do the same!

I used the following ESEUTIL command to check the State of the Mailbox Database file. The /mh switch tells ESEUTIL to check the state of the database.
eseutil /mh “database path”

As you can see from the screen shot the Database State is reported as Dirty Shutdown, this makes perfect sense as the Exchange server lost connectivity unexpectedly.

I then attempted to use the following ESEUTIL command to repair the Mailbox Database file.
eseutil /p “database path”

It failed with "Operation Terminated with Error -1032 (Jet_errFileAccessDenied, Cannt Access File, the File is Locked or in Use) after 10.47 Seconds" I checked the Event Logs for more details.

The Event Logs state that ESEUTIL is trying to create a temporary database file at C:\ProgramData\Microsoft\Windows\StateMenu\Programs\Microsoft Exchange Server 2010\temp.edb. As you can see it was reporting an "Access Denied" error.

As it was trying to write to the C:\ drive I had to expand my C:\ so that it had enough space for the temp database

I then tried to run ESEUTIL /P again and it also failed, with the same error. The fix here was to change the ACL on the C:\ProgramData\Microsoft\Windows\StateMenu\Programs\Microsoft Exchange Server 2010\ directory to allow Everyone Full Control.

When I ran ESEUTIL /P again it began to work. It returned the following warning "You should only run Repair on damaged or corrupted databases. Repair will not apply information in the transaction log files to the database and may cause information to be lost. Do you wish to proceed?" as my database was offline and unaccessible I had no option. Please note I copied the original database file before doing any of this.

The ESEUTIL tool then started scanning the database, my database was 60GB's and this took approximately 30 minutes to complete.

At one stage it looked like the process had hung, so I checked the Event Logs. A Warning "eseutil (3616) The database enginer lost one or more bad columns of data in one record. It is highly recommended that an application-level integrity check of the database be run to ensure application-level data integrity". I decided to let ESEUTIL run it's course and it did complete.

When it completed it confirmed it by displaying "Repair completed. Database corruption has been repaired".

For good practice I then decided to run a defrag on the Mailbox Database. I used the following ESEUTIL command to do this;
eseutil /d “database path”

When the degrag completes you will get the following prompt to confirm it has happened successfully.

Now, as I was performing all of this on a copy of the production database I then copied the file to the original location. Not forgetting to change the name of the original database.

I then opened services.msc and started all of the Exchange Server, services that I had stopped at the beginning of the process.

I enabled Circular Logging temporarily. You can do this by right clicking on the Mailbox Database, clicking Properties and ticking the Enable Circular Logging option.

I then tried to mount the database and it returned the following error, this was different from before. The error had changed to ErrorAttachedDatabaseMismatch this was to be expected as I had copied the original file.

The fix was to create a new folder called old in the original locations for both the Mailbox Database and the Transaction Logs, then move all of the file into the old folder except the new copy of the mailbox database.

Now when I tried to mount the database it worked successfully.

When you are satisfied that the database is back to a healthy stage, I would disable Circular Logging.