“IO IO, It’s off to disk we go!” was in Mike Walsh’s blog post that introduced this TSQL Tuesday and I thought this would be an ideal opportunity for me to get my feet wet with a TSQL Tuesday contribution of my own, my very first. This kind of got me thinking….IO what a vast and varied subject to write about, so I started looking back through my career about the some of the IO related issues I have come across and I realised that I had a couple to chose from that are IO related.
My most recent issue/problem was a combination of IO, Virtualisation, method’s of working. The problem was caused as a result of snapshot taken by a sys admin against a VM that housed a SQL Server that I support and that snapshot was not removed when it should have been. When my SQL disk backups (Full and Transaction log backups) were being written to disk, then tape, then deleted some days later, the snapshot recording all these disk changes and filled up all its available space, This in turn caused the server to fall over.I will state here that this was a test server and no major business issues came about as a result of this problem, so you could argue that it is not really a major troubleshooting crisis in my career but it did identify to me a couple of things that I needed to change in my daily checks routine, and a few additional skills I needed to learn as we moved some of our production instances to the virtual world , so this ‘little’ problem potentially prevented a much nastier one down the road.
Normal disk space checks need to be changed for SQL Servers running in virtual land – I have scripts, jobs, alerts and manual checks that check the size of my databases and the drives they reside how much free space each drive in my SQL Servers has remaining. These checks did not show anything to be concerned about. The snapshot on my server was not using any of that space, but it had run out of space allocated to it. I added to my daily check list, a simple manual check – Check for snapshots and its size on the VM.
It also identified a more general area for me to improve, in order to check the physical machines hosting the VM’s and their I needed to become more familiar with VM tools (VMWare’s infrastructure client in my case) as they will be necessary tools for troubleshooting VMs down the road.