The Technical Debt Anchor
-
Comments posted to this topic are about the item The Technical Debt Anchor
-
I was given the task of migrating an application that hadn't been touched or patched in years. The app relies on AWS Lambdas written in Python 3.8, and AWS issued a notice that Python 3.8 was being deprecated. The app would continue to run until AWS removed the Python 3.8 runner. The 1st stage of deprecation was for AWS to block any changes to existing Python 3.8 apps.
The App was deployed using an early version of Terraform. How hard could the upgrade be? We ended up in dependency hell.
- The version of Terraform had a cap on the version of the AWS provider (think of a Terraform provider as an API).
- The capped version of the AWS provider only supported up to Python 3.8
- Upgrading the Python code locally to 3.13 revealed that more than expected Python changes were needed.
- Upgrading Terraform revealed deprecated resources (think in terms of API function calls) and alterations to surviving resources
- Other Terraform providers were out of date with similar upgrade woes
Additionally, our company had written several reusable Terraform modules, many of which had been deprecated in favour of more up-to-date alternatives. The old app still used the deprecated versions, so the migration scope ballooned further to include swapping out old modules for their replacements.
What should have been a couple of weeks' worth of work turned into months of effort. That effort took us away from things the business wanted us to achieve. The silver lining to this cloud upgrade is that it has quantified and demonstrated the business impact of tech debt.
Sweating the asset has resulted in us paying with our sweat (at usurious interest) and blood pressure.
The approach we follow now is that all our apps quietly auto-patch and upgrade. If anything breaks, in almost all cases, it happens before it is deployed to the production environment. The breakage will be small and relatively easy to fix.
We have tests for data quality, but more importantly, we have business processes and SLAs attached to those tests.
It is all about "a stitch in time saves nine".
- David.Poole wrote:
I was given the task of migrating an application that hadn't been touched or patched in years. The app relies on AWS Lambdas written in Python 3.8, and AWS issued a notice that Python 3.8 was being deprecated. The app would continue to run until AWS removed the Python 3.8 runner. The 1st stage of deprecation was for AWS to block any changes to existing Python 3.8 apps.
The App was deployed using an early version of Terraform. How hard could the upgrade be? We ended up in dependency hell.
- The version of Terraform had a cap on the version of the AWS provider (think of a Terraform provider as an API).
- The capped version of the AWS provider only supported up to Python 3.8
- Upgrading the Python code locally to 3.13 revealed that more than expected Python changes were needed.
- Upgrading Terraform revealed deprecated resources (think in terms of API function calls) and alterations to surviving resources
- Other Terraform providers were out of date with similar upgrade woes
...
Ouch, having been involved in Terraform / version issues I can really feel that one...
-
There's a lot in today's article, but at the moment I can only touch on a few of them. I believe that the place where I work has a lot of technical debt. There has been, for many years, a tendency to ignore technical debt. Often because we had other pressing demands put on us, so we had to get something new done due to regulatory requirements or politicians. (I work in State government here in the US.) But another contributor to our big technical debt is managers who didn't want anyone doing anything to existing software, if it is running. My former boss was one of these managers. He would ignore things like security risks, just because the app was running.
There's two sentences from the article which I want to call out. The first is, "Don't expect to do the same job you've done for years.". That is precisely what a lot of people who work here are expecting to do. It's what people who've worked here for decades have done. They've come here with a certain skillset, and due to the culture, they haven't had a need to change.
The second sentence follows from the first, "Learn to use new tools and learn to use them effectively in your position." Many people here resent having to learn anything new. They came here expecting never to need to learn anything new and for many people who've worked here 20 to 30 years, they haven't needed to learn anything new.
I think some things are changing, due to our adopting the cloud more (not a lot, but some more). There's a lot of resentment, as this isn't what they signed up for. Oh well, maybe this will have a positive impact upon our reducing our technical debt.
Kindest Regards, Rod Connect with me on LinkedIn.
-
The third one is a fun one - open source dependencies. We have a tool that often has "free" plugins available for it but after a few years those free plugins get switched to paid plugins. The business has build workflows around the "free" stuff and it turns into a situation where we have to buy it.
The other fun one is when the open source project stops getting updated and goes into an archived state. Then if you are lucky, you can find a fork that keeps the project going, but sometimes once it is archived you have to migrate to some new tool.
And one debt that I consider technical debt but not sure it qualifies is cloud migration. Moving software from on prem to the cloud (via vendor, so going on prem to SaaS) is often easy. The vendor does what they can to make it as simple as possible. BUT then you are locked into their cloud. If they decide to double the price, you just have to eat that. AND migrating OFF of the cloud from a lot of vendors can be very challenging. One SaaS tool we had bought into and tried to migrate from we had to buy a 3rd party tool to get our data out of the system. They offered no data extraction tools - once it was in the cloud, it was theirs. A different tool we migrated to the cloud with has no good data extraction tools, so once you migrate to the tool you are kind of stuck with it. As they say "if you think the marriage was expensive, wait until you see the cost of divorce". Now this doesn't apply to ALL cloud tools. SOME have nice tools to migrate to and from the cloud, but if you don't do your research first before migration, that becomes technical debt when upper management says we need to cut costs and your license for the tool went up.
The above is all just my opinion on what you should do.
As with all advice you find on a random internet forum - you shouldn't blindly follow it. Always test on a test server to see if there is negative side effects before making changes to live!
I recommend you NEVER run "random code" you found online on any system you care about UNLESS you understand and can verify the code OR you don't care if the code trashes your system. -
So ... nowadays still having win2003 OS with SQL2005 or worse comes into the picture
Johan
Learn to play, play to learn !Dont drive faster than your guardian angel can fly ...
but keeping both feet on the ground wont get you anywhere :w00t:- How to post Performance Problems
- How to post data/code to get the best help[/url]- How to prevent a sore throat after hours of presenting ppt
press F1 for solution, press shift+F1 for urgent solution 😀
Need a bit of Powershell? How about this
Who am I ? Sometimes this is me but most of the time this is me
-
I see many DBs that would run perfectly happily on SQL Server 2000. I've seen various pieces that put the average data warehouse size at below 250Gb.
I can see why businesses are slow to upgrade. It's not just an IT phenomenon. In the 1970s, British Rail received their 1st HST 125 locomotive. Intended as an interim solution and with a design life of 25 years, these were only withdrawn from mainline service in 2021. They are still in operation in some regions of the UK, 50 years later.
The UK have had problems with electricity substation transformers exploding. Precisely why is still a bit of a mystery, but what is known is that these were installed in the early 1960s with a 40-year design life. The transformers example is relevant because they lack the technical and architectural protections that are mandatory in their modern equivalents. There is a suspicion that the oil in the radiators degrades with age, leading to failure.
Hardware does not last forever. There comes a point where you have a lot of ancient, mission-critical artefacts that are expensive to replace, especially when the replacement process must minimise downtime.
Back in the world of IT, systems work until they don't. When they stop, there are limits to what technical necromancy can achieve.
Viewing 7 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply