For some reason I'd overlooked the fact that the instance of Team Foundation Server I was running on my Hyper-V server was still the release candidate instead of the RTM version. Upgrading turns out to be relatively painless. I followed Johan's suggestions. To backup the data-tier I fired up sqlcmd and ran the following:

BACKUP DATABASE Tfs_Configuration TO DISK = 'Tfs_Configuration.bak'

GO

BACKUP DATABASE Tfs_DefaultCollection TO DISK = 'Tfs_DefaultCollection.bak'

GO

I uninstalled just the TFS component, then ran setup.exe off of the RTM and after allowing the setup to proceed, chose the 'upgrade' option. It's nice to see that they ask you to confirm that you have done a backup before the upgrade can continue. Coincidentally, like Johan I also had a problem with a TFS workspace being in use – though I believe this was because I changed the TFS Build process to run as a user account (previously it was using SYSTEM). I used a variation on the same command he used (different possibly because my servers are just in a workgroup rather than a domain).

tf workspace /delete "1_1_TFS;NT AUTHORITY\SYSTEM" /login:tfsserver\username,password

Note that the TF.EXE command comes with the TFS client bits – I ran it from the machine I run Visual Studio 2010 on as my TFS server just has the server stuff.