Last week I came into the office and had a colleague come up to me even before I took my coat off. (You know who you are!). A VM was turned off and wouldn’t let itself be turned on again. Ok..what’s the story of this VM?

I immediately sat behind my laptop and asked what happened with this VM over the last week.
Well on Monday, the VM was snapshotted because some changes had to be made to it. After those changes had been applied, the snapshot was removed. The first try at removing the snapshot failed (unknown error), the second attempt was successful  (or so it was believed).
I first checked if any space was left on the LUN. This was not the case,so I “cold-migrated” the VM of to another LUN with plenty of free space. (maybe a reason why something went wrong?)

Next up was to check VC snapshot manager. The Snappy-Manager says everything is ok, no snapshots are available. Let’s check the VM directory itself through SSH…hé, what are those *000001-delta files still doing in the VM folder?
I powered on the VM and saw that the date/time of the delta files changed…this is not a good thing if snapshot manager says this snapshot does not exist.
Ok, over to plan B.
Step 1: I went into the datastore browser and copied the complete directory of the VM over to a backup LUN.
Step 2: I tried removing the still present snapshot with:
vmware-cmd /vmfs/volumes/1234/VM004/VM004.vmx removesnapshots
This command returned the following:
VMControl error -3: Invalid arguments: Virtual machine has no snapshots
Well that doesn’t help…
Step 3: I went into VC and created a snapshot (test) and flipped on the powerswitch of the VM.
Step 4: I checked the VM folder and a new file named 000002-delta. was created!
(so at least somebody knows there’s a file named *1-delta!!!!)
Step 5: Turned the VM off and went into snapshot manager to delete the snapshot….

15 nail-biting minutes later, snapshot manager said it was done…and it was not lying this time..all the delta files were gone! So far (almost 2 weeks now), no damage to the VM has been reported which makes me even more impressed with the robustness of VMWare!
If this post helped you out, help me out keeping this site alive and visit some of my sponsors on the left or right.. Thanks..!
TAGS: VMWare, snapshot, no space on LUN