We have been hit by the balloon driver many times even though the company that looks after our hardware swore that they never over commit resources. After months of begging I finally got access to the VMWare console and saw for myself that it was the balloon driver causing the issue so we now make memory reservation part of the standard build for production servers. I have lost access to the console again now but recently noticed some VMWare counters in Perfmon e.g. [VM Memory - Memory Ballooned in MB] so if you have access to the OS you could try keeping an eye on those. There are ones for VM processor as well e.g. [VM Processor - CPU Stolen Time]

Another thing that we have suffered from in the virtual world is DRS causing server migrations and it seems to pick the busy server to migrate for some reason rather than moving off the more lightly loaded VM's.