May 20, 2024 at 8:00 pm
I had one job where I was the only DBA, so on-call 24 x 7 x 365. We created and sold software that was run on-prem by clients, so I was mostly a development DBA, so not many emergencies. If they called me outside normal working hours, I was paid for my time with a minimum of one hour. Calls were rare, because almost everything could wait. Great job, except the base pay was awful (very small company selling to non-profits).
I had another job where I was the only DBA, so on-call 24 x 7 x 365. I was both a development and production DBA. One application was a back-end process for a web store, so production was also 24 x 7 x 365. There was very little discipline about making changes to code in production (usually ordered by the company owner, so there was no way to stop it) and no spare capacity. I would get called a couple of nights per week and around one weekend per month. It was mostly quick and simple issues, but there were occasions where I worked for several hours to clean up a mess. I found a job elsewhere because I felt this became abusive.
In my current job, I started as the only DBA for development and production, so on-call 24 x 7 x 365. We now have a second DBA (I am in the US and she is in Greece) so we have better coverage. We have no formal rotation between us. Even when I was the only DBA, I would get called less than once per year even though the applications I support are in use around the clock. Two main reasons are it is a strict ITIL shop so no one makes code changes in production without going through QA and it being part of a scheduled deployment, and the other is that the Service Desk goes to our infrastructure group for a second level and only they (or upper management) can involve the DBAs (third level) directly. The infrastructure team is very good at determining the difference between application, network, and database issues. We are all-in on Azure SQL Database, so if something goes crazy they generally just scale it up and have the DBAs deal with it in the morning. They have an on-call rotation and are scattered across 8 time zones, so even they don't get off-hours calls often. It's usually work hours for at least one of them, except on the weekends. This is paradise for me compared to that second job.
May 21, 2024 at 2:26 pm
@m60freeman your description of your current job sounds fantastic, compared to the second job. I've got a question or two I'd like to ask.
First, what sort of work is the company you work for? You mentioned ITIL, which I am only familiar with by reputation. It is something I would think the organization I work for (a large state department) would be all in on. But for reasons I don't know, we don't do ITIL. Was ITIL something that had to be adopted, or has it always been applied there?
Second concerns Azure SQL. I played around with Azure SQL four or five years ago. I got a small database uploaded into an Azure SQL database. I liked the way it worked, but it was pretty small, so didn't exhibit well the potential of Azure SQL, I think. IT management here has opted for lift-and-shift, when we put anything into Azure. Which means it costs more than following a cloud native approach. So, when the bills started to arrive they were always higher than they could be, but they would never consider making some necessary changes to make it more cloud native. I'm guessing the company you work for may have approached Azure (or at least SQL Server in Azure) in a similar way. How did you get them to change their architectural approach?
One of the tasks I've had to take on is TFS Administration, when the previous TFS Admin left years ago. We have an old, out-of-support on-prem TFS server. I argued, for years, that we needed to go to Azure DevOps Services or GitHub. After 5 years we've finally migrated our source code to GitHub. I'm glad that we did that, but WOW, I never thought it would take so long!! We had many false starts in trying to migrate out of TFS. I confess I'm tired of having to press this issue so many times, and each of those times it being defeated, most often without any explanation. ("Sorry Rod, we're not going to migrate out of TFS now.", "But why?", silence.)
Kindest Regards, Rod Connect with me on LinkedIn.
May 21, 2024 at 3:06 pm
First, what sort of work is the company you work for?
I work for a Europe-based IT company that is dedicated to servicing, and effectively part of, a global network of financial services companies.
You mentioned ITIL, which I am only familiar with by reputation.
ITIL came out of the UK but has had some level of world-wide acceptance. I've worked within that framework at previous (US-based) employers as well. It can be a PITA, but it really does prevent or solve a lot of problems. From Wikipedia:
"The Information Technology Infrastructure Library (ITIL) is a set of practices and a framework for IT activities such as IT service management (ITSM) and IT asset management (ITAM) that focus on aligning IT services with the needs of the business.
ITIL describes processes, procedures, tasks, and checklists which are neither organization-specific nor technology-specific."
How did you get them to change their architectural approach?
I didn't. This IT organization was relatively young and most projects were built specifically for Azure SQL Database. One major one was rewritten for Azure SQL Database and that new version replaced the legacy one (which ran in cloud VMs). The company never owned an on-prem server of any kind. Everything was either in a third-party cloud and migrated to Azure, or started in Azure.
We use Azure DevOps, supplemented with Octopus Deploy, although I don't know why. It must add some value over the native DevOps deployment process, but I don't know the specifics.
May 21, 2024 at 3:44 pm
One nice thing about ITIL is a company can implement it gradually/incrementally. In fact, it would be horribly disruptive and almost impossible to do all at once. Often the first step is to create a configuration management database (CMDB), which catalogs and defines the relationship between between hardware, software components, and networks. That lets you build configuration and change management and almost every other part of ITIL on top of that. For example, the service desk can relate tickets to items in it, and it can contain information about who maintains each item, the product manager for software items, etc.
May 22, 2024 at 5:00 pm
My story - I am getting ready to retire this year, after 40 years in IT, 25 of that as a DBA. I have been 'on call' for 40 years! Sometimes jobs had a list for primary\secondary rotation, but none of them ever formally compensated. Usually just took time off and told my boss not to call me. The worst job was just previous to this one - on call 24x7x365, no backup (3 dba's, but we each supported different systems, so no cross-over), AND we were expected to work every Saturday night patching various servers in rotation. We each got calls at least once a week. The other 2 DBA's had no interest in learning each other's systems, to share on call. After many discussions on how to better manage all of this, I went to my manager and said, this isn't working for me. He offered me 4x10 schedule. I said OK, I will schedule Saturday as one of those, Sunday off, what days during the week should I schedule as off days. He said, no, I need you in the office M-F. I told him that's not how 4x10 works. We looked at each other in silence. He finally said - well, what do you expect me to do about it? So I walked back to my desk, wrote my resignation letter, and handed it to him. He was mystified as to why I would quit. Clueless didn't begin to describe him.
I've been on this last job for 10 years. Been the lone DBA for much of that. But my boss says - I don't expect you to answer the phone outside of 8-5, M-F, and he has stuck by that. Sometimes business mgmt makes a fuss, but when we present the budget for going 24x7, for both systems and personnel, they shut up. I'm sure it will change at some point, but I will not be around to deal with it.
May 23, 2024 at 2:22 pm
I've got one more observation about how sometimes management can be clueless. I am one of the TFS and GitHub Administrators. Our TFS server is years out of support and finally last year management decided to migrate our source code out of TFS to GitHub. GitHub has the ability to import TFVC repos into GitHub repos. I proposed this to management, but they didn't want to spend the money to do it. Instead they told me and a colleague who is the other TFS/GitHub Admin, to migrate all the hundreds of TFVC repos to GitHub, manually. And management wouldn't listen to an other arguments. So, it took the two of us about 9 months to migrate all of those TFVC repos to Git repos in GitHub. I'm sure that the agency I work for spent more in our combined salaries, then it would have to use GitHub Import. And this isn't an isolated case. This sort of behavior of trying to save money in the short term, only to spend more in the long term, is a recurring habit.
Kindest Regards, Rod Connect with me on LinkedIn.
Viewing 9 posts - 16 through 23 (of 23 total)
You must be logged in to reply to this topic. Login to reply