May 6, 2024 at 12:00 am
Comments posted to this topic are about the item The DBAccountant
May 6, 2024 at 1:04 pm
My one-and-only experience in being involved in the financial side of resource decisions was spending eleven years as IT manager in a company as we implemented their first IT technology. In those days the hardware/software vendors had personnel who were very actively involved in helping select systems according to needs and budget and we were advised by a consulting firm, and systems were pretty much a 'package deal' of hardware, OS, and development language. The first selected system (IBM hardware/software/programming language) only lasted through about the first two years of growth. Then in order to continue increasing growth of newer technology for the company we decided to do a total change of hardware, OS, and programming language from IBM to Burroughs (Unisys). Then just a couple years after that change, we did another total system change, keeping the first Burroughs system and adding a larger system, and doing the first database implementation and another total programming language change. Going through all of this in eleven years was what made me decide 'never again' and that I would never be involved in the financial side of things again.
Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )
May 7, 2024 at 11:00 am
I think financial monitoring of cloud resources needs to be something that DBAs are aware of and participate in but not something that is their sole responsibility. I think there are enough tensions where DBAs have to push back on things without becoming the window of NO for something else as well.
I'm not a DBA anymore. I have had to keep a keen eye on how long Github workflows take to run and therefore cost, how workflow jobs should be organised because each job is billable at a minimum grain of 1 minute. 10 jobs of 6 seconds gives a bill of 10 minutes, not 1 minute.
When DBs such as BigQuery are run you are charged for every query. If you query INFORMATION_SCHEMA you instantly incur the cost of querying 10MB. If you have 1 table and your query INFORMATION_SCHEMA.TABLES then that is charged as if it were 10MB.
If someone uses a BI dashboard with an auto-refresh enabled then those underlying queries will keep executing and incurring cost as long as that dashboard is open.....and if you forget you have it open in your millionty-billionty browser tags.....OUCH.
Then there is auto-scaling. You can set a minimum and maximum number of machine instances. Sound reasonable, however when you attempt to deprecate something you have to remember to take that something out of the auto-scaling group, otherwise it will keep coming back and whatever that undead thing is will incur costs. Start up costs can be more intense than background running costs so if it keeps resurrecting over and over again...OUCH. Similarly, anything that causes an unintended failure of something in an auto-scaling group can be a moneypit.
Serverless technology can be great for short, infrequent activity. If you running something continually or frequently enough then serverless is a very expensive option.
Cloud systems can be architected and engineered for incredible resilience. Just make sure your business actually needs and can afford the level of resilience.
May 8, 2024 at 3:15 am
I think that the people who made the decision to move to to the cloud should have the sole responsibility of monitoring the costs to prevent all the "I told ya so"s from the DBAs. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
May 8, 2024 at 10:37 pm
I think that the people who made the decision to move to to the cloud should have the sole responsibility of monitoring the costs to prevent all the "I told ya so"s from the DBAs. 😀
The problem with that is that the people looking at the costs start telling you to not run as many queries, or to use less storage, even though that's not really feasible.
Once our financial analysts quit looking at the detailed costs of cloud usage and just compared the total costs to having on prem servers, they quit complaining. The hard part was getting the true cost of the on prem services, much of which was buried in multiple departments.
May 9, 2024 at 3:38 am
Jeff Moden wrote:I think that the people who made the decision to move to to the cloud should have the sole responsibility of monitoring the costs to prevent all the "I told ya so"s from the DBAs. 😀
The problem with that is that the people looking at the costs start telling you to not run as many queries, or to use less storage, even though that's not really feasible.
Once our financial analysts quit looking at the detailed costs of cloud usage and just compared the total costs to having on prem servers, they quit complaining. The hard part was getting the true cost of the on prem services, much of which was buried in multiple departments.
From what I've seen and heard (and, as a mere observer, could certainly be mistaken but don't know) there's a similar problem in measuring true cost on the cloud compared to on-prem because the cloud does not have the same throughput as on-prem. From what I've heard, slower systems are frequently selected to "save on costs" and I wonder how much that's actually saving.
Anyway... if someone that moved us to the cloud wanted to put a throttle on how many queries we were running, I'm thinking that there'd be a 4 course pork-chop dinner involved where the appetizer would be an up close an personal reminder that they said the cloud would be cheaper and that didn't include any throttling.
--Jeff Moden
Change is inevitable... Change for the better is not.
May 9, 2024 at 7:37 am
The problem with that is that the people looking at the costs start telling you to not run as many queries, or to use less storage, even though that's not really feasible.
Once our financial analysts quit looking at the detailed costs of cloud usage and just compared the total costs to having on prem servers, they quit complaining. The hard part was getting the true cost of the on prem services, much of which was buried in multiple departments.
This is so true. An ex-colleague on the operations side saw the cloud costs and said "I have nightmares knowing what I could have given you on-prem for 10% of that"!
An awful lot of on-prem IT costs are hidden as "just the way we do business" and accountants robbing Peter to pay Paul.
A profitable cloud provider has to get their costs right if they are to remain profitable. Its harder for cloud consumers to hide and/or spread the costs. That's why I recommend regularly reviewing the cloud vendor's equivalent of the AWS Well Architected Framework.
The cloud vendors have a number of tools and strategies to help you manage costs and/or alert on them. However, using them and acting on them is the consumer's responsibility. This is 2nd nature to people who have spent their career on the operations side but less so for those on the Development side.
May 14, 2024 at 4:30 pm
Actually, running less queries is possible. A lot of apps move to the cloud and are very chatty. We don't notice as much on a LAN/WAN with a local server, though I bet many of your clients complain about performance.
We can't reduce all queries, but often we can reduce them. But it takes work to rewrite or clean up apps.
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply