I once talked about the DBA team building about 3 years ago, now when I look back, I still believe my original idea is a good start, however, I feel it is not enough, so I’d like to explore some other areas for DBA team building, that is how to design a team, from team structure to work responsibility and along the way, I’ll give real examples to support my argument.
There are lots of theories about how to structure an organization, such as function-oriented design, like HR, Accounting, Sales etc. or process-oriented design, like assembly line work, or skill-oriented design, which is commonly seen in some consulting companies that have grouped the staff into Center of Excellence teams for different subjects.
To me, a good DBA team should have high efficiency with high productivity.
To achieve high efficiency, we need to look at communications within the team and more importantly outside of the team. Communication is the key to generate new ideas and to coordinate efforts.
To achieve high productivity, we need to look at skillset / knowledge spectrum within the team. Skillset and knowledge is the cornerstone to any high productivity.
As such, I come up with the following two principals for building an ideal DBA team
1. A DBA team should have the capability to solve a typical database issue from the beginning to the end within the team.
2. A DBA team should have the capacity to maximize its output from HR / accounting perspective.
For Principal 1, this is driven by efficiency consideration. Take a performance issue as an example, many times, we have to get DBA team, network team, storage team and QA team together to figure out an issue and verify its solution. This is a big hurdle to efficiency mostly due to the communication and also because that different teams have different managers with different priorities in their mind.
For Principal 2, this is driven by productivity consideration. No matter how much automation we have, we need manual involvement to take different tasks and we have to admit that different tasks have different level of difficulty, so putting a senior resource to tackle an easy job is obviously not a good investment. (well, an easy job does not mean it is not an important job). So the point here is that a DBA team needs different level of DBAs to work on different tasks so you can avoid a senior resource to feel bored while on the other hand, to train a junior one to grow up.
Back to my experience, we once had a DBA who is also inside a system team, he reports to the system team manager but with his knowledge of database administration, he is logically a DBA team member, and we consider him as our representative speaker in the system team (who manages network, user accounts, OS, and hard-ware related work). At that time, the work efficiency, from solving a problem perspective, is great, because we just need to talk to him, for example, applying a Cumulative Update, and he will do it because he has all the authority / access permission to do that. On the other hand, if there is any problem in the eyes of the system team, like history backup files not deleted and thus taking huge amount disk space, he has the authority from the DBA team to do it. Nowadays due to some organization change, he is no longer with the system team, and we have to go through another long path to do our work, i.e. from our DBA team manager to the system team manager, and then to the system team to get the work done. This no doubt has its own merit, such as, task is assigned more officially with audited trace, but efficiency is compromised here, especially when two teams have different priorities, and there is no authority (or the authority chain is too long) to coordinate the priorities.
To support my principal 2, here is my experience. I once had to do lots of QA environment refresh using production backups. Due to the frequency and the number of QA environments, I was kind of putting about 50+% of time doing the refresh work, and have no time to think about how to improve / enhance the overall production administration framework. We later outsourced this QA refresh work to another dedicated team, and they did much better than myself, from planning the work, communicating with other stakeholders to delivering and managing the lifecycle of the QA environments. On the other hand, I was relieved from this work and put all my energy to optimize my DB administration framework. I feel this is a win/win scenario, and as a contractor, I do not feel guilty to my client any more.