Installing Virtual Server 2005 R2 SP1 on Windows Home Server 2011

Breaking up is never easy but I’m slowly getting over the pain of losing Drive Extender from Windows Home Server 2011 and have started evaluating if I’ll upgrade when the time comes.  There was another potential seperation for me when considering the move to the new version and that was the fact I have been using Virtual Server 2005 R2 SP1 to host virtual machines on my current WHS.  Microsoft have not included the Hyper-V role on WHS2011 which is understandable given that this is a consumer product, running virtualisation would require additional support, plus the fact that there are all sorts of licencing issues (each virtual machine needs it’s own licence of the Windows client or server that you install).

I’m one of those those people that get labelled as “enthusiasts” so if there is a way to get something to work the way it wasn’t quite intended to with what I determine is acceptable risk then I’ll have a good go at it. This is where I remind you that this really isn’t supported so if you decide to follow these instructions you do so at your own risk and neither myself or anyone associated with The Digital Lifestyle takes any responsibility for corrupted files, lost data, or personal injury (when your partner finds out you lost all the pictures of your kids growing up).

This is what you’ll need before you start (these are all free, it’s the guest operating systems that need to be licenced) and remember to get the x64 (64-bit) downloads:

• Microsoft Virtual Server 2005 R2 SP1 – Enterprise Edition – http://www.microsoft.com/downloads/en/details.aspx?FamilyID=BC49C7C8-4840-4E67-8DC4-1E6E218ACCE4
• Microsoft Virtual Server 2005 R2 SP1 Update – http://www.microsoft.com/downloads/en/details.aspx?FamilyID=A79BCF9B-59F7-480B-A4B8-FB56F42E3348
• Virtual Machine Remote Control Client Plus (VMRCplus) – http://www.microsoft.com/downloads/en/details.aspx?FamilyID=80ADC08C-BFC6-4C3A-B4F1-772F550AE791

The reason that VMRCplus is needed is because the Virtual Server Administration website doesn’t work and so we’ll exclude this from the installation.  It is also important to note that VMRCplus doesn’t support alternative credentials so unless you are running your own Active Directory you’ll only be able to use this tool over an RDP connection to your WHS2011.  There is one more hurdle to jump and that is WHS2011 is built upon Windows Server 2008 R2 and so will prevent you installing VS2005 as it knows it is incompatible.

This protection will need turning off and I found the details on TechNet:

http://social.technet.microsoft.com/Forums/en/windowsserver2008r2general/thread/77fa60d5-78f3-46e9-9b36-3fd7f1aa9a31

Use local group policy to turn off application compatibility to allow the install. In Local Policy go to Computer Configuration -> Administrative Templates -> Windows Components -> Application Compatibility, enable the settings here to “Turn Off Application Compatibilty Engine”. When installing Virtual Server 2005 R2 SP1 on Server 2008 R2 ensure that the Hyper-V Role is NOT installed either, they really don’t like each other and strange happenings will ensue.

To access the “Local Policy” settings open an RDP session to your WHS2011, open the start menu and type “edit group policy” and this should display a single item which you can run and then follow the instructions above.  This is what you should now see:

This setting is cached so you’ll need to restart your WHS2011 server at this point.  With the server restarted and an RDP session open you can proceed to install the Virtual Server software, I’m not going to include every step here, if you’re doing this and can’t figure out how to install software you should STOP now, but you should choose a “Custom” install with these options:

When the installation is complete you’ll need to stop the “Virtual Server” service so that you can install the update (this was to support additional host and guest operating systems but Windows Server 2008 R2 is not supported as I already stated) and reboot the server again if necessary.  You should also run Windows Update as there is an additional security update that applies to Virtual Server 2005 R2 SP1.  The server will need to reboot for that security update and then you can install VMRCplus.

The VMRCplus tool might take a little getting used to but it works just as well as the old Administration Website (some of you might have already used this tool anyway).  At this point you can go back and change that Group Policy to “Not Configured” or “Disabled” (this is the option I chose to flag up the fact I changed it) without any impact on running Virtual Server 2005 R2 SP1 (that name just rolls off the tongue doesn’t it?).

Enjoy. 🙂