Technical debt is a huge topic, but I'm glad you brought it up, Steve. Everywhere I've worked I've witnessed technical debt. Often, it is as you said, developers (and DBAs) not fully understanding the system they're to develop and the requirements for the requested software solution. i.e.: they develop something that doesn't match what's needed. Blame goes between developers and BAs not grasping it correctly, to users not communicating what's needed well.
However, I've also noticed that the technical debt in the organization I currently work for is significantly large. To me it seems disproportionally larger than any other organization I've worked for. But I might be wrong. It might be because this organization has been around for many decades and the IT group is exceptionally large (at least by the standards I've experienced). I work for a large state government agency in southwestern US. This state has been a state for over 100 years, so some of the software systems have been around longer than anyone here has worked in the IT field.
Certainly, the ever-present pressure to work on the next urgent thing pushes addressing technical debt to the back burner. Heck, I think sometimes technical debt has even been taken off the stove. But one thing you didn't mention in your post, Steve, is a phenomenon I've only seen here. One of the first projects I was put on was to make modifications to an old ASP.NET WebForms app written by someone who left years before I came along. The fellow who wrote the app used techniques from old Windows apps development, rather than anything related to web development. I told my boss about this. My boss's response was, "All developers think every other developer's work is junk. So, the first thing they want to do is re-write it. Don't re-write this app. Just fix the small task that needs fixing and move on." I was shocked by this response. I wonder if this thinking is commonplace among the managers where I work. I wonder if this type of thinking is commonplace in other places as well. I get the feeling that if a software solution satisfies more than 75% of the requirements, then the temptation is to just call it good and move on to the next high demand project. And maybe that's the way in many places.
Kindest Regards, Rod Connect with me on LinkedIn.