Upgrading vSphere 5.5 before End of General Support

General Support for vSphere 5.5 will end on September 19, 2018. If you are still running vSphere 5.5 you have just a few months left to upgrade to a newer version. I am beginning to see many customers make the push to upgrade directly to vSphere 6.5 to take advantage of the latest features. If you are running vSphere 5.5 then you need to read the following KB article from VMware regarding the End of General Support for 5.5 (KB 51491). I’m going to focus primarily on upgrading the vCenter Server Appliance in this blog article and then focus on the ESXi hosts in an upcoming blog article that I will lump together with upgrading vSAN 5.5. So keep an eye out for that one.

Any time you are looking to perform an upgrade of vSphere you will have a numerous number preparation tasks. My recommendation would be to establish some sort of ‘runbook’ that serves as guideline and outlines a detailed list of tasks. For example, ensuring compatibility is a major step especially if you are looking to continue using existing hardware. Some tasks may involve…

Verify server hardware compatibility from your server vendor.
Ensure existing enterprise backup solution functionality.
VMware Solution Interoperability.

The VMware Compatibility Guide is a great place to start. Very easy-to-use and if for some reason you cannot find the information you are looking here then contact your hardware vendor directly. They will have an answer for you.

If you are using multiple solutions from VMware you will need to do a few more things to guarantee interoperability and stability during and after the upgrade. There are two great references available from VMware to help you down this path.

The VMware Interoperability Matrix is a great upgrade preparation tool to guide you down the correct path to safeguard compatibility between multiple VMware solutions that may be present in your environment. For example, lets say you are using Site Recovery Manager 5.8.1 and vSphere Replication 5.8.1 in your vSphere 5.5 infrastructure as part of your DR strategy. You need to find out which specific version of these two solutions must use to ensure compatibility once the environment is upgrade to vSphere 6.5. The Interoperability Matrix will quickly tell me which version(s) I can use as you can see below.

After you are finished with the verifying compatibility and interoperability you will then need to review the VMware upgrade path (option on the Interoperability Matrix) and by referencing the Update Sequence for vSphere 6.5 and its compatible VMware Products (KB 2147289).

Scroll down the page and reference the ‘Supported Update Sequence’ portion of the KB article and you will find a table with a scroll bar. Simply highlight the VMware solutions present in your environment at the top and then work your way from left to right.

Continuing with my example above

vSphere ESXi 5.5
vCenter Server 5.5
Site Recovery Manager (SRM) version 5.8.1
vSphere Replication (VR) version 5.8.1
vSphere Update Manager (VUM)
vSphere Operations Manager (vROPS) version 6.2

My vCenter 5.5 VCSA appliance is running SSO locally along with vCenter Server (both roles); they are installed on separate servers. If the SSO was running on a separate system from vCenter Server then I would need to upgrade/migrate the external SSO to an external 6.5 PSC first (reference Upgrade Guide for more info). The example that I will be demonstrating shortly will will have all roles running locally on my vSphere 5.5 appliance.

After vCenter Server is upgraded, what is next in line for upgrade? If you reference the table you will see VUM, vROPS and VR all have a number ’10’ for the sequence. This indicates they are all on an ‘equal playing field’ and therefore I can upgrade these three (3) solutions next in any sequence that I desire. I can take advantage of VUM running on my vCenter 6.5 appliance and therefore I do not have to worry about migrating that separately. I simply proceed upgrading vROPS and VR next. Once the solutions are upgraded I will then upgrade SRM, followed by my ESXi hosts and my virtual machines (VMware Tools and Virtual HW).

Another important reference you should be aware of is the Supported and Deprecated Topologies for vSphere 6.5 (KB 2147672). Review this KB article thoroughly and make sure your target environment is using a supported topology that fits your needs. This is very important if you currently use more than one vCenter Server or even plan to in the future. My recommendation is to “future-proof” your vSphere environment to avoid any major changes. Get it out of the way now instead of having to modify the environment later.

Last but not least, do not forget to get your licenses! Upgrade from your My VMware portal. Have them ready to go and make sure you have correct number of licenses for your resources.

Just a quick recap…

Be very thorough during the planning phase of your pending upgrade. Verify all hardware compatibility and come up with a remediation plan in the event you discover something is not compatible. Check the Interoperability Matrix and Upgrade Paths for your multiple VMware solutions to ensure you have the correct upgrade sequence to guarantee functionality through your process.
Create a runbook or workbook of some sort to track the entire upgrade procedure from start to finish. Even if you are running vROPS or other management solution I recommend executing an RV Tools inventory dump prior to the upgrade and cross-check everything in a spreadsheet. Double and triple check everything!
Plan the upgrade during off-hours just in case you experience any type of unplanned downtime. Things go awry at times and even though a lot of these tasks can be performed during normal operations I would not plan on doing so. Remember “just because you can doesn’t mean you should” would apply here. Cover your butt and do this off- hours.
Don’t forget to run your backups! Snapshots don’t count as backup. They’re quick points-in-time references that you can still use during your upgrade procedure but do not rely on them as your only means to fallback should something fail. Get those backups complete and verify them by doing a restore!

Upgrading vSphere 5.5 to 6.5

In this demonstration I am going to upgrade an environment from vSphere 5.5 to vSphere 6.5. Before doing so let’s review what my lab environment currently looks like.

Single vCenter Server 5.5 appliance w/ local SSO (Version 5.5 U3 – Build 7881688)
Two vSphere ESXi 5.5 hosts (Version 5.5 U3 – Build 3248547)
vSphere Replication (Version 5.8.0.1 – Build 2170514)

My target version will be vCenter Server 6.5 Update 1e (Build 7515524).

You should also disable DRS on the cluster(s) where your appliances are located. We need to do this to ensure the source and destination VCSA appliances are not vMotion’d to another host mid-upgrade. This would cause the upgrade to fail due to connectivity loss. We could disable DRS just on the source vCenter VM but the new target VM has not been created yet which is why we simply disable DRS temporarily at the cluster-level during the upgrade.

Review the upgrade paths listed in the vSphere Upgrade (PDF) guide for 5.5 to 6.5 that starts on page 36. The VCSA appliance that I am going to migrate here shortly with you has all of the vCenter 5.5 roles running locally and everything will be upgraded to a single vCenter Server appliance with an embedded PSC (below).

If your source vCenter Server 5.5 appliance is configured differently from what you see above then you definitely need to review the architectures to better understand the overall upgrade procedure for your environment. Pages 36-40 cover the various architectures so review that information over and over.

Common Upgrade Errors

Before we get started let’s cover a few common errors that I have come across in my recent upgrade experiences. These errors can easily trip you up in the beginning of the upgrade procedure and cause you to begin rollback procedures.

The first one deals with the default certificates on the vCenter 5.5 appliance still using the ‘localhost.localdom’ hostname in the default certificate. To resolve this you will need to regenerate the certificates on the VCSA 5.5 appliance to reflect the current hostname and IP address of the appliance. If you do not update the certificates correctly the upgrade procedure will fail! Follow the instructions in VMware KB Article 2110772 to correct this issue before proceeding.

The second error I encountered occurs as the new appliance begins copying over the source VCSA configuration, hostname, IP address, etc. and a failure message appears ‘failed to set network.’ Prior to upgrading you may need to check the ‘ifcfg-eth0’ configuration. If the ‘IPADDR’ field includes a ‘slash (/)’ value next to the appliance IP address you will likely encounter this error. Simply follow the instructions in VMware KB Article 2144310 to resolve this issue before upgrading. This configuration error most was common when upgrading directly from 5.5 U2. I did not encounter this error upgrading from 5.5 U3.

The third common error involved VUM and attempting to migrate VUM running on a separate Windows Server VM into my new vCenter 6.5 appliance. You will obviously run into this error if you are running the VCSA 5.5 vCenter Appliance because VUM must run on a separate Windows Server in that particular environment. I decided to “test attempt” the migration multiple times (requires running the ‘Migration Assistant’ on the Windows Server running VUM). Unfortunately during the final steps of the upgrade process it fails with an error…

“Error attempting Vcintegrity Export file does not exist or is corrupted, abort import”

This is a known issue and the only resolution is to remove VUM from the source vCenter Server appliance from MOB and restart the upgrade procedure. You can find more information about this particular error in VMware KB Article 2150982. You will basically rebuild VUM from scratch on the new VCSA appliance once it is upgraded to 6.5. Start fresh!

Now we can get started on the upgrade! There are two (2) stages for the upgrade process. Stage 1 involves deployment of the new virtual appliance; Stage 2 involves the configuration and migration of source data to the new appliance.

From my workstation I mount the downloaded vCenter Server appliance ISO image, navigate to the ‘Installer.exe’ located in the ‘vcsa-ui-installer\win32’ directory and select Upgrade to begin.
Click Next on the ‘Introduction’ page of the wizard.
Accept the License Agreement and click Next.
Enter source vCenter appliance FQDN or IP address and click ‘Connect to Source’ button. The wizard will change and then prompt you for more information about the credentials for SSO and root account as well as the source for the vCenter 5.5 appliance. I enter the ESXi host info where my VCSA is running and click Next. Click YES on the Certificate Warning.
On the ‘Select deployment type’ page I choose the vCenter Server with Embedded Platform Services Controller and then click Next.
Enter the information for the appliance deployment target; this can be an ESXi host or separate vCenter Server. Click Next and then click YES on the certificate warning.
Enter the name of the new VCSA appliance as you want it to appear in the VM inventory. Do not name the appliance the same as the current appliance as it will cause an error and the upgrade to fail. Choose something unique for now and you can change it back to the original later. Click Next.
Choose the deployment size and storage size and click Next when ready.
Choose the datastore for the target VCSA appliance; enable Thin Disk mode is optional depending on your deployment requirements. I am working in a lab environment with limited resources so I use this option. Click Next.
Configure the static TCP/IP information for your deployment. The IP address you enter here is only ‘temporary’ for the deployment. Once the deployment and migration is complete the legacy source VCSA appliance will be powered down and your new appliance will reconfigure its IP address to that of the source. Click Next.
Review the information on the ‘Ready to complete stage 1’ page of the wizard and click Finish when you are ready.
Stage 1 of the upgrade process will begin with deploying a new VM for the new VCSA 6.5 appliance. When the process completes, select ‘Continue’ to launch Stage 2 of the upgrade procedure. If you were to leave this interface prematurely you can always pick up where you left off by accessing the newly deployed appliance from a web browser using the following URL: https://<fqdn-or-IP>:5480/
Click Next on the Introduction page of Stage 2. A ‘Pre-upgrade check…’ will initiate followed by a results window. Review the results and click Close when ready.
Enter the VCSA 6.5 SSO (Single Sign-On) site information for the new appliance and click Next.
Select which upgrade data that you would like to migrate from the source to the target appliance. Notice an estimated size is shown; the amount of data will have an impact on the amount of time it will take to upgrade the appliance. Click Next.
Select or un-select the check box for CEIP and click Next.
Review the information on the ‘Ready to complete’ window of the wizard and then select the check box to acknowledge that you have backed up your source VCSA. Yes, you should have done this and no a snapshot is not the only thing you should have completed here :). Click Finish.
The migration of the source appliance data will begin. During this process the wizard will eventually initiate a shutdown of the source appliance (it won’t delete it) and reconfigure your new appliance with the same TCP/IP settings that the source was once using. Basically your new appliance will take on the persona of the legacy appliance when everything is complete.
This slideshow requires JavaScript.
Upon completion of the upgrade you will then access the vSphere Web Client (or HTML5 Client) from a web browser. Simply enter one of the following URLs into your browser.vSphere Web Client: https://<vcsa-fqdn-or-IP>/vsphere-client/vSphere HTML 5 Client: https://<vcsa-fqdn-or-IP>/uiBelow I have the flash-based Web Client open. Notice the alert at the top to apply my vSphere 6 licenses for vCenter Server.

That completes the step-by-step upgrade procedure of the vCenter Server (VCSA) appliance upgrade from 5.5 to 6.5. After the upgrade is complete you will need to begin performing a handful of post-upgrade tasks. Some of those verification tasks include…

Ensure the static IP address is correct.
Verify the domain is correct; including identity source integration (i.e. Active Directory or LDAP) and test log in capabilities for the vSphere Web Client using various accounts.
Verify NTP settings are correct and the time on the appliance is 100% correct.
Verify the certificates on the appliance.
Begin spot checking data that migrated during the upgrade:
- Performance chart history.
- Event History.
- All roles, users, groups and permissions.
Start checking logs! If you use vRealize Log Insight then you will be ahead of the game here.

Last but not least…RUN A BACKUP to protect your new appliance! You can run a quick snapshot on the appliance but please by all means run a FULL backup.

I can now begin upgrading my other VMware components. I was running vSphere Replication (VR) version 5.8.1 in my 5.5 environment, I can now begin upgrading this appliance to 6.5, 6.5.1 or 8.0 (all three are compatible with vSphere 6.5). Reference your upgrade runbook which should contain everything you gathered when you used the Interoperability Matrix and Update Sequence.

I will leverage VUM on my new 6.5 appliance and begin upgrading my ESXi hosts in a ice clean coordinated/orchestrated fashion! Check out Melissa Palmer’s blog on how to accomplish this task “How to Upgrade ESXi Using vSphere Update Manager 6.5 on the vCenter Server Appliance“.

Conclusion

A lot of organizations out there are feeling the pressure to upgrade ASAP because of the quickly approaching End of General Support date later this year. Time goes by fast so don’t procrastinate any longer. Start planning and testing this procedure in the next couple weeks and get that environment upgraded long before that EoGS date (September 19, 2018).

I outlined a majority of the major tasks for you above but most importantly READ THE DOCUMENTATION. I cannot stress this enough. VMware has put together a great upgrade guide which I provided a link to above for you and again below. Don’t forget the release notes as well.

Don’t forget those 3rd party plug-ins that you may use as well. Make sure they are compatible with vSphere 6.5 before upgrading, especially if the plug-in is something you rely on a regular basis. Lean on your vendor to help provide you information to properly restore plug-in functionality after a successful upgrade of vCenter Server.

Double and triple check your VMware compatibility and interoperability matrices! Those resources are diamonds in the rough when it comes to properly planning the upgrade. They will set the foundation and ensure success for your upgrade.

My upgrade blog for ESXi 5.5 and SAN 5.5 is coming soon! So stay tuned! 🙂