Jeff Moden (7/1/2013)
Great Editorial. Now all we need is a link that teaches managers and developers how not to over-promise and one that teaches developers how to get over the "just get it off my plate" syndrome. ;-)
It all boils down to the simple rules that "Good enough usually isn't" and that "If you want it real bad, that's the way you'll usually get it". :-P
I would disagree on one point there, mostly because you need some nested quote marks but also because I don't agree with your "never": good enough always is good enough - but "good enough" is sometimes a lie, sometimes a mistake, and only rarely good enough; and I've sometimes written stuff which I knew would have a lifetime of only a few days, so that honing the performance would have been a waste of resource - good enough was good enough those times.
The really big problems are as you say, over-promising and the "just hand it over" syndrome. In my experience over-promising is almost always committed by junior/middle management or by salesmen, and is the result of timescale pressure from the top and/or from prospects. Bonus schemes for salesmen make them very vulnerable to such pressure, especially when the penalties of their overpromising damage development bonuses, not sales bonuses, and of course the fact that the company has been committed and will lose face leads to top management refusing to accept realistic dates. Training salesman to promise nothing not already committed by development/QA/Release teams and persuading those teams to do proper estimating instead of going for "what's the earliest we could possible do this if everything goes just right" might solve the problem if the people concerned are trainable. "Just get it of my plate" should be a firing offence; developers indulge in it, so do QA types, and so do development/QA management. In a developer it is usually the worst sort of laziness: I can't be bothered to do this properly because I want to get on to the next interesting thing instead of finishing this boring old stuff. In managers it's often a pass-the-buck strategy: maybe if I sign this off and hand it over to Fred he'll get blamed for any delays, not me.
The bottom line is that it takes time to learn how to write good code and unit test it with enough data to know if it will perform well. Too many people won't take the time to do either. That's why SSDs and things like Hekaton are such a draw... they promise the gift of performance for crap code.
SSDs and in-store databases change the game somewhat. Using that sort of thing to try to make lousy code run faster is usually futile, and I don't much like it even when it works.
However, just as large scale random access storage devices allowed performance that was unimaginable using serial mag tape, and necessitated new algorithms/designs to exploit them properly, devices with zero seek time and effectively zero search time (SSDs) will allow us to do new things and may mean that we need to learn new tricks to exploit them properly.
The same applies even more strongly to in-memory databases, which aren't really particularly new - Hekaton is in a sense old-hat. The cache algorithms in SQL 2000 were such that certain of the NEOS databases were effectively resident on our customer sites (while others on the same sites were non-resident) giving a big performance boost: of course the NEOS databases were designed to behave like that, without using any of that silly table-pinning stuff. What Hekaton brings is elimination of a lot of overheads in buffer handling, because no page of the database can be dropped from the buffer. Back in the late 80s/early 90s I was working on massively parallel systems with in-store database (using at different times our own fully declarative destined to be in-store database system and Oracle OMPS) and the projects I was involved in were certainly not the only ones addressing in-store databases in those 10 years. We probably need to learn new tricks to exploit in-store databases, just as we do to exploit SSD storage - and many of those new tricks may be common to both technologies.