May 23, 2026 at 1:16 am
Comments posted to this topic are about the item Even When You Know What You're Doing, You Can Screw Up
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 23, 2026 at 12:18 pm
I've got a situation at present where a BI product has an API and a UI that uses the API.
The API has "features", as in looks like a bug, behaves like a bug, stinks like a bug, results in downtime like a bug.
We wanted to elevate an existing login to have admin capabilities. From the UI this was a select box and apply. No hints as to the mayhem this would cause.
It dropped and the user, removing object ownership and all privileges then recreated the user. Sure they were an admin but an admin of sweet FA.
As objects now had no owners no-one could access data sources, or the reports or dashboards that needed them. What a fun month end support call that was! Bye bye weekend.
Another cloud DB BI product did something worse. Resizing switched off the auto-suspend setting so instead of shutting down after 1 minute of activity we got billed 24/7 for several days before that was noticed. The resize was a UI drop down box option with an OK button.
The people who made the change did nothing wrong. They used the product as it was supposed to be used.
We now have a shed load of monitoring on things that are out-of-the box bugs. Not faulty input detectors, outright crap machinery dectectors
May 24, 2026 at 3:24 pm
Grant I know what you mean. And this article's message is a good one! But I wonder how far one can take this?
For example, I am writing this on May 24, 2026, which is the Sunday of a Memorial weekend. I took the Friday off, to make it a 4-day weekend for me. My family and I wanted to go see the movie "Mandalorian and Grogu" which came out this weekend. We wanted to see if at one of those theaters which also offer meals during the movie. You have to purchase tickets and select your seating in advance. So, Thursday night I got into their website and purchased and tickets with seating for Saturday.
Or so I thought.
Mid Friday afternoon I got a reminder on my phone that the movie had occurred! I don't know how I did it, but I saw Friday as Saturday and made all the arrangements for the wrong day. And I've done this before, in other situations.
But how is that system going to know what I really wanted to do? How could any system know?
My situation isn't as bad as dropping a production database or exposing secrets to the Internet. But it is similar. I think that if I were to double or triple check every decision I make, I will slow to one third my normal pace. Which isn't acceptable.
Kindest Regards, Rod Connect with me on LinkedIn.
May 24, 2026 at 6:22 pm
Amen to that !
Johan
Learn to play, play to learn !
Dont drive faster than your guardian angel can fly ...
but keeping both feet on the ground wont get you anywhere :w00t:
- How to post Performance Problems
- How to post data and code to get the best help
- How to prevent a sore throat after hours of presenting ppt
press F1 for solution, press shift+F1 for urgent solution 😀
Who am I ? Sometimes this is me but most of the time this is me
May 26, 2026 at 1:15 pm
About 15 years ago, I was hired as the first DBA at a small medical records company. My first project was to assess the overall environment and make suggestions/changes to ensure best practices, tighten up security, etc. I noticed the AD group "All Users" had sysadmin rights in our main production server. Once my eyes stopped rolling and I picked my jaw up off the floor, I dove into this one. Long story short, there were appropriate security groups with the correct users in the groups and seemingly proper permissions set in the various databases on the server. After a couple of weeks of investigation, I felt confident enough to remove "All Users" from the sysadmin role.
Within 3 minutes of removing the group from the role, I could hear employees throughout the floor complaining the app no longer worked. A couple of managers paid me a visit and I quickly put the group back into the role. The app started working again.
Well, to this day, still don't know what I missed, but I'm thinking there were deeply hidden calls to xp_cmdshell or the OA procs. This server was the wild west, so it could've been anything. Before I arrived, the entire production system was created and maintained by developers who knew SQL, but were clearly not DBAs. I still use experience as an example for several scenarios to this day.
May 26, 2026 at 1:20 pm
I've got a situation at present where a BI product has an API and a UI that uses the API.
The API has "features", as in looks like a bug, behaves like a bug, stinks like a bug, results in downtime like a bug.
We wanted to elevate an existing login to have admin capabilities. From the UI this was a select box and apply. No hints as to the mayhem this would cause.
It dropped and the user, removing object ownership and all privileges then recreated the user. Sure they were an admin but an admin of sweet FA.
As objects now had no owners no-one could access data sources, or the reports or dashboards that needed them. What a fun month end support call that was! Bye bye weekend.
Another cloud DB BI product did something worse. Resizing switched off the auto-suspend setting so instead of shutting down after 1 minute of activity we got billed 24/7 for several days before that was noticed. The resize was a UI drop down box option with an OK button.
The people who made the change did nothing wrong. They used the product as it was supposed to be used.
We now have a shed load of monitoring on things that are out-of-the box bugs. Not faulty input detectors, outright crap machinery dectectors
I mean, yikes!
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 26, 2026 at 1:24 pm
Grant I know what you mean. And this article's message is a good one! But I wonder how far one can take this?
For example, I am writing this on May 24, 2026, which is the Sunday of a Memorial weekend. I took the Friday off, to make it a 4-day weekend for me. My family and I wanted to go see the movie "Mandalorian and Grogu" which came out this weekend. We wanted to see if at one of those theaters which also offer meals during the movie. You have to purchase tickets and select your seating in advance. So, Thursday night I got into their website and purchased and tickets with seating for Saturday.
Or so I thought.
Mid Friday afternoon I got a reminder on my phone that the movie had occurred! I don't know how I did it, but I saw Friday as Saturday and made all the arrangements for the wrong day. And I've done this before, in other situations.
But how is that system going to know what I really wanted to do? How could any system know?
My situation isn't as bad as dropping a production database or exposing secrets to the Internet. But it is similar. I think that if I were to double or triple check every decision I make, I will slow to one third my normal pace. Which isn't acceptable.
Right there with you. And, again, I'm positive, I didn't do anything wrong. I read from the radio first, which should have got the correct settings (and the settings were correct, see Chicago and Wroclaw). I then changed a few channels, but not the APRS settings. Wrote back to the radio. Everything should have worked. But it didn't. And I'm not sure what the problem is, or the solution. But I know I've got one more thing to check.
BTW, I've booked flights & hotels on the wrong days. I do a double check, each month, on the next month of travel to be sure there are flights, and hotels, on the right days. It slows me down. But I get where I need to go.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 26, 2026 at 1:25 pm
About 15 years ago, I was hired as the first DBA at a small medical records company. My first project was to assess the overall environment and make suggestions/changes to ensure best practices, tighten up security, etc. I noticed the AD group "All Users" had sysadmin rights in our main production server. Once my eyes stopped rolling and I picked my jaw up off the floor, I dove into this one. Long story short, there were appropriate security groups with the correct users in the groups and seemingly proper permissions set in the various databases on the server. After a couple of weeks of investigation, I felt confident enough to remove "All Users" from the sysadmin role.
Within 3 minutes of removing the group from the role, I could hear employees throughout the floor complaining the app no longer worked. A couple of managers paid me a visit and I quickly put the group back into the role. The app started working again.
Well, to this day, still don't know what I missed, but I'm thinking there were deeply hidden calls to xp_cmdshell or the OA procs. This server was the wild west, so it could've been anything. Before I arrived, the entire production system was created and maintained by developers who knew SQL, but were clearly not DBAs. I still use experience as an example for several scenarios to this day.
Ooh, I worked at a place like that.... for six months.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 26, 2026 at 1:26 pm
Amen to that !
Cheers!
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 26, 2026 at 1:49 pm
About 15 years ago, I was hired as the first DBA at a small medical records company. My first project was to assess the overall environment and make suggestions/changes to ensure best practices, tighten up security, etc. I noticed the AD group "All Users" had sysadmin rights in our main production server. Once my eyes stopped rolling and I picked my jaw up off the floor, I dove into this one. Long story short, there were appropriate security groups with the correct users in the groups and seemingly proper permissions set in the various databases on the server. After a couple of weeks of investigation, I felt confident enough to remove "All Users" from the sysadmin role.
This story reminds me of the last place I worked...
EVERYONE was a local Administrator on their PC. At least what SQL Servers were used, didn't have everyone in sysadmin!
May 26, 2026 at 1:56 pm
Ooh, I worked at a place like that.... for six months.
I lasted 18 months at this place, mostly because there were some very promising opportunities once it was pointed out how bad the app was. That said, it really messed with my mental health, so I jumped to get out of there when I had the chance.
May 26, 2026 at 6:46 pm
Grant Fritchey wrote:Ooh, I worked at a place like that.... for six months.
I lasted 18 months at this place, mostly because there were some very promising opportunities once it was pointed out how bad the app was. That said, it really messed with my mental health, so I jumped to get out of there when I had the chance.
Nice work. I just couldn't hack it and took the first decent opportunity that came my way.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 31, 2026 at 1:31 pm
<snip />Right there with you. And, again, I'm positive, I didn't do anything wrong. I read from the radio first, which should have got the correct settings (and the settings were correct, see Chicago and Wroclaw). I then changed a few channels, but not the APRS settings. Wrote back to the radio. Everything should have worked. But it didn't. And I'm not sure what the problem is, or the solution. But I know I've got one more thing to check. BTW, I've booked flights & hotels on the wrong days. I do a double check, each month, on the next month of travel to be sure there are flights, and hotels, on the right days. It slows me down. But I get where I need to go.
That is a great idea! My faux pas with the movie tickets would have put me out about $70. But plane tickets! Dang, you could be out hundreds of dollars!!
Kindest Regards, Rod Connect with me on LinkedIn.
Viewing 13 posts - 1 through 13 (of 13 total)
You must be logged in to reply to this topic. Login to reply