One thing I learned while working as a database administrator over 17 years is the importance of teamwork across departments. Many times, we have to rely on network and SAN administrator to make changes to their environments in order to make SQL Server run more efficiently. There are times where the storage and network create bottlenecks for SQL server and after doing all the possible tuning you can do with your code and configuration you must turn to them for modifications to their hardware. Knowing how to effectively work with other departments without placing blame is crucial.
As most do, we take our work personally. After all it’s your hard work, decisions and efforts that make the environment run as it does. Telling network admins and SAN admins their environment is running poorly or misconfigured can cause a very adverse reaction. At times it can even cause them to refuse to work with you or make an effort to help fix the issue. I often refer to this as, telling them their babies are ugly. You need to learn how to nicely do this without pointing out the flaws in their genetics.
How do we do this?
The most important thing you can do is make friends with the other departments approaching them in a respective and friendly professional manner. Developing a cohesive relationship right off the bat is key, especially in a new job. It’s important to establish that you know their work is important and that you also appreciate their work. I tend to do this with food as a sign of appreciation. I bring in donuts or pizzas to other departments to show my appreciation. It helps to get me on their good side if I am ever in a position where I need to call their babies ugly. Having a friendly relationship with them helps soften the blow. Food always seems to do the trick when making friends.
Next you need access to see their babies, which are their environments, so you can get concrete numbers to show that the network or storage is causing a bottleneck for SQL Server. If your environment is using VMware, ask for read-only access to vSphere so that you can see how it’s configured and monitor your virtual machines. Get to know the environment and do your research so when you have to ask questions you are knowledgeable in exactly what you are questioning. I find being familiar their environments and knowing the proper vocabulary before going into discussions helps aid in conversations.
In addition to that, start gathering solid performance metrics with trends so that you can present proof that those systems are causing SQL Server issues. Numbers don’t lie. By doing this you are presenting a fact base issue and not an opinion. This is very important, as it is not us against them. Many times, charging at someone with allegations, especially without concrete numbers, that their systems or hardware is problematic can be like personal attack.
Take a look at the below performance monitor counters. These are just a few counters will help support your allegations, there are many more out that should also be utilized but I start with these.
- Avg. Disk sec/Read
- Avg. Disk sec/Write
- Avg. Disk Queue Length
- Database Pages
- Page Life Expectancy
- Avalable MBYTES
- Processor\% Processor Time
- Processor\% User Time
- Processor\% Interrupt Time
Next you need to show them that you have made efforts on the SQL Server side to tune your environment as much as you can. Keep a running list this of the improvements you’ve made. Have you done anything to reduce the amount of IO SQL is creating, i.e., tuned queries, added indexes, re architected data loads, optimized configurations to fit best practices? By doing this it shows you have made an attempt to do everything possible to rectify the issue on your side.
Lastly, always remember they too may be struggling with a non-optimal environment. They may have their hands tied with old equipment, such as an old spinning disk SAN and no budget for improvements. They may be battling too many VM’s on their hosts because they just can’t buy any more hosts to balance their demand and that’s what could be causing your SQL Server bottlenecks. Sometimes other administrators have no control over how ugly their babies are and have little options for making them prettier. It’s important to keep this in mind.
Getting your SQL Server environment as optimized and running at peak performance is always the goal of the database administrator. To do this often requires teamwork with other departments. Keeping the things in mind that I have mentioned can help make that easier. Remembering to not call their babies ugly or blaming SQL Server problems on them personally is essential to working cohesively to rectify the bottlenecks.