November 18, 2020 at 12:00 am
Comments posted to this topic are about the item Collecting Toxic Waste
November 18, 2020 at 9:03 am
I read this and it took me me back to my software development days (over 25 years ago). I worked in a team of three developers, who also provided support for the application we were writing.
I wasn't the best programmer but I was good at administration, implementation, and support. We evolved a "development process" with written specifications for new features, code fixes, changes, etc. When a change was needed we had to present the change specifications and the other two would play "devils advocate" and ask questions / challenge any assumptions / suggest alternatives. It worked really well - certainly for me as I used to do a lot of the testing and implementation.
I found the process of explaining something to someone else incredibly useful, it meant I had to clarify my thoughts - which often identified issues before I presented it. Over the years as I moved away from development and into support and administration, I use the same thought process. I've lost count of the number of times I've looked at something for extended periods of time, shown it to someone else and either they've pointed out the flaw or I've spotted it as soon as I start explaining it!
I'm a huge advocate of following this sort of process with anything you do in IT - be it development, support or administration.
November 18, 2020 at 1:26 pm
Good article, Steve. There are a few things I could respond to, but I'll focus upon your second question. No, we don't track technical debt. We accumulate it, but we never track it. In fairness to my current employer, I have never worked any place where they track technical debt. Because of that I'm not sure what tracking technical debt would look like. Whatever it looks like, we don't do it.
As a corollary to this, I'll mention something related to it. I've mentioned before how much I dislike the commute I must normally go through, to work. They insist that I travel a long distance and long travel times, just so that I can work in my state's capital. This pandemic has, at least for me, revealed an interesting aspect and that's what it is like to run our applications near my home. I live near the largest city in my state, so naturally there are a lot of state employees that live and work here. I'm now seeing, firsthand, how slow our applications are. For example, I'm now working on an application which pulls down a little over 18,000 records every time it opens this one dialog window. It must do so; my boss requires that it do so. However, that query runs very slowly. I've timed it and found that it takes up to 30 second to retrieve that data and display it. 18K+ isn't that much, but this query is still slow. If you work at the headquarters in the capital, all such queries are instantaneous. But not so when you're 70 miles away. I'm hoping that seeing this for ourselves, will cause us to try and optimize the query.
Rod
November 18, 2020 at 1:35 pm
I have to agree with Gamberoni in that explaining a design, code, test, etc. to someone else is very useful. I've often found that when I get stuck, if I explain it to someone else, even if they don't have an answer, quite often helps me to better understand the issue and arrive at a solution.
I also agree with Steve about "toxic waste" that builds up over time. Too many people (at least those I've worked with) think that Agile means no design is necessary and they just jump in and start coding with a focus only upon the current sprint. Then when the invariable bugs and patches come, they are given about as much consideration as the initial code. Then the system becomes overly complex, error prone, and difficult to manage and maintain.
Time spent in thoughtful and careful design is less time spent in development and far less time spent in maintenance. Plan your implementation well with future features baked into your design and you'll have an overall shorter development cycle, lower maintenance, and greater satisfaction. Time well spent now is future time saved.
LinkedIn: http://www.linkedin.com/in/sqlrv
Twitter: http://www.twitter.com/sqlrv
Website: http://sqlrv.com
November 18, 2020 at 8:41 pm
Another form of technical debt is needless personal identifying information (PII) that accumulates in our databases and file systems. I've sometimes analogized it to toxic waste, because of the way it's produced on an industrial scale and it's potential to spill over into the public domain causing harm to individuals. Also, just like toxic waste, government regulators are imposing penalties when it's not properly contained. So, let's keep our databases clean.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
November 18, 2020 at 9:10 pm
that toxic PII is something I have some customers rethinking. It's a liability if they lose data, and one that often isn't even generating revenue.
November 19, 2020 at 12:30 am
You got that right, Steve. Some of our data has PII. In a few of these cases I can't figure out why they want to collect it.
Rod
November 19, 2020 at 12:50 am
I don't worry too much about collection. At least not as much as why do we keep it. If we can get rid of it an reduce liability, do that.
If we need it, great, but if not, it's a liability.
November 21, 2020 at 7:34 pm
Continuing down the PII = Technical Debt side track, I'm thinking that in an ideal scenario, an OLTP databases, data warehouse, data lake, etc. should not contain PII - only something like a sequential or random personally unique identifier (a surrogate key) that links back to an isolated and highly secured "Data Vault". This could be Azure Key Vault, or perhaps hosted by a third party who specializes in this type of solution. Integration between on-prem or cloud hosted transactional databases and data vault can be facilitated by an API. I've never worked for an organization that even attempted to take PII protection this far, and the concept is relatively new, but anyone in the process of building out a new database enterprise should consider it.
I found an article on the topic:
https://www.linkedin.com/pulse/gdpr-pii-data-vault-20-dan-linstedt
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply