SQLServerCentral / Editorials / SQLServerCentral.com / Another Bug Hunt / Latest Posts

RE: Another Bug Hunt

Chris-232075 — Mon, 30 Jan 2006 07:57:00 GMT

Excel has bugs in it?

I'll let you have the last word, except to say that my answer to both questions is "yes".

Thanks!!!

RE: Another Bug Hunt

Richard Sisk — Thu, 26 Jan 2006 08:00:00 GMT

Chris, no offense taken, I am enjoying the debate.

Let me put my point of view in better perspective. A software company would not survive if it released software where basic functionality did not work, or even if the majority of the functionality didn't work. The market would quickly tire of this and not buy it. On the other hand, if a company that builds a complex application with many different applications interfacing with each other, that also interfaces with other applications, and has tons of functionality, if they waited until the software was bug free (which is a very high hurdle) they would never release software and would also be out of business. As I've said in the past, it is a matter of balance based on business decisions. Reality dictates that the software company has to decide at what point between these two extremes that they will release software.

To address the ethics question, the company should disclose known issues either through release documentation or a portal that the customer has access to. Of course this is an after the purchase kind of thing but then if a sales person came into a sales call and started out by saying "Here are all the problems with our software", they would be kicked out of the competition. Plus the competitors are always there to help point out the issues with the other vendor's software, so there is some knowledge of where the known issues are.

Would you want to wait until Excel was bug free before it was released? Do you think it was unethical to be sold Excel before it was bug free? I use Excel, but insert the name of your favorite application or operating system in its place. My answer to both of those is "no", otherwise I would not have bought the software and would still be tracking my stocks with paper and pencil.

RE: Another Bug Hunt

Steve Jones - SSC Editor — Wed, 25 Jan 2006 18:54:00 GMT

Chris makes some great points and I think there should be some recourse when an application doesn't do what it promises.However I think there is also a question of scale. Larger pieces of software that perform general functions are harder to debug and deliver bug free. The latest press on Oracle says something like 50 million lines of code that have to scanned and not many automated tools can even handle something that large. Delivering bug free software in an OS or large piece of software is hard.In a more specialized applicaiton, it's easier. TurboTax is a very specialized piece of software and I've rarely heard of issues in it.

RE: Another Bug Hunt

Chris-232075 — Wed, 25 Jan 2006 18:29:00 GMT

I get it that you believe that there are legitimate business reasons for releasing software with known bugs. I'm sorry, but I'm afraid I can't agree with the arguments made to support this logic.

However, assuming I'm mistaken about the business logic ... what about the ethics of such an approach?

Even though the dba/development community is conditioned to expect bugs in early releases of software products they purchase...does that really release MS and Oracle from responsibility from delivering a finished (bug-free) product when collecting our money? Isn't there a significant cost to our employers or customers when such bugs exist (consider how much time is spent researching the causes and applying any fixes or workarounds)?

What about the general public or even a purchaser of a high-end business application who hasn't been conditioned as we have? Would management put the "cat2/cat1 error handling policy" in the product brochure or in the purchase agreement? Probably not. But would you argue that it's also "understood" by these buyers that the product will be "incomplete" when version 1.0 is delivered? If so, I'm not sure sure. I've been around a little, and that hasn't been my experience at all. My experience is that such buyers plan and expect that the product will be entirely consistent with oral and written representations made when selling the product (which is likely to exclude any mention of known bugs in the product).

Depending on statements made in the sales process, buggy software could even be illegal. Most states have "Fair Business Practice" laws that say it's against the law for businesses to make false representations to generate a sale. I'v read about software companies that have been sued and lost under such laws when, in essence, the delivered application was "buggy".

By the way...enjoying (or enjoyed) the discussion, and I apologize for any potentially abrasive sounding comments -

RE: Another Bug Hunt

Richard Sisk — Wed, 25 Jan 2006 15:42:00 GMT

Condoning it is not the right way to describe my point of view. I see it as a business decision that must be made before the software is released.

I'm saying that a business has to make a decision as to what point it is right to release software. If we were to wait until our application was bug free, we would never release software. Imagine in MS waited until SQL 2005 was bug free before it was released, we would be getting it sometime in 2015, and would we even want it at that point?

What they and I would suspect every other software company do is determine where the point of diminishing returns is met, based on many variables, and then they release the software.

You choose to purchase software based on many factors. Some people will choose to buy the 1.0 version of software, even though it may be buggy, but it really does something that they need, Windows Mobile 5.0 on my iPAQ comes to mind. Others prefer to wait until a later version comes out because what they currently have is sufficient and they don't want to fight the hassles of buggy software.

In the same way, different software companies have different criteria for releasing software. Some may decide to release software because their customers are screaming for certain functionality, so they will reduce the time spent in QA in order to get the release out and follow up with another release soon after to clean up bugs that have been found. Other companies will spend more time testing their software because their customers have a lower tolerance for buggy software. These are generalities, but it points out that there are many different market requirements for software, and in order to remain in business, the software company has to decide what is the best time for the software that they sell for it to be released.

RE: Another Bug Hunt

K. Brian Kelley — Tue, 24 Jan 2006 10:57:00 GMT

No, I don't condone it, but I do understand the realities of the situation. My mom is Japanese. I was taught to do it right or don't do it at all... lest you face a bamboo stick.

RE: Another Bug Hunt

Chris-232075 — Tue, 24 Jan 2006 10:27:00 GMT

Maybe I misunderstood, but I thought you and Richard were doing more than highlighting this reality regarding what many businesses settle on...I thought you were condoning it. In earlier posts, others seemed to be apologists for this reality. If so, that's where we disagree. If not, I apologize for the confusion.

RE: Another Bug Hunt

K. Brian Kelley — Tue, 24 Jan 2006 09:42:00 GMT

Sure, and I think ideally we'd all agree with Rudy. However, realistically we know what many businesses settle on, which was the point Richard was making and also what you elaborated on.

RE: Another Bug Hunt

Chris-232075 — Tue, 24 Jan 2006 09:15:00 GMT

Rudy can correct me if I’m mistaken, but when he wrote “do it right the first time”, I suspect that he meant bug-free. The quality (fix the bugs) vs. profits considerations are, in my opinion, short-term considerations only. In the long-run, continuing to make such decisions in favor of profits is likely to actually reduce profits.

For example, if you’re in your forties like me, then you’re likely to remember that, when growing up, your parents purchased American-made cars ONLY. Try to gauge the ratio of foreign cars to American cars driven today the next time you walk through a supermarket parking lot. 8 to 1? Greater?

Why? One can try to over-complicate it, but the answer is that the Japanese auto makers have significantly higher standards for quality. The Japanese strive to build bug-free automobiles. American auto makers ask themselves, “How many bugs can we release in our products before the backlash begins to erode profits?”

(My Ford was recalled five times while I owned it and, with all the needed repairs, was more expensive to maintain than the one Honda and two Nissan’s I’ve owned combined.).

Richard’s “cat3 errors vs. cat1 errors” approach is, I’m afraid, common in our industry. That’s my complaint. In addition to being bad for business in the long-run, one can also argue that such an approach is also unethical (unless you disclose to your customers that your product is being delivered with “cat2 and cat1” errors).

RE: Another Bug Hunt

K. Brian Kelley — Tue, 24 Jan 2006 07:22:00 GMT

Agreed. To business, "doing it right the first time" may mean, "Get the product out as soon as possible, bugs or not, so long as the public backlash for the bugs doesn't seriously erode the profit margin.

RE: Another Bug Hunt

Richard Sisk — Tue, 24 Jan 2006 07:14:00 GMT

Ah yes, but it is the definition of "doing it right the first time" that makes this debate interesting.

RE: Another Bug Hunt

rudy - Doctor "X" — Mon, 23 Jan 2006 12:47:00 GMT

It's really quite simple, it boils down to:

'pride of ownership' --> doing it right the first time
' rationalization' --> political-speak for talking your way out of responsibility for something
'MBA Mentality' --> looking and and worshipping the almighty dollar, damn the costs later

Every pro/con/middle of the road statement point made thus far in this discussion thread can be put in these three categories.

Just remember this credo --> there is never time to do it right, but always time to do it again !

RE: Another Bug Hunt

Richard Sisk — Fri, 20 Jan 2006 10:42:00 GMT

The decision of when to release software is a balancing act between many different things. How many features, how many bugs are acceptable, time to market, price, available resources, market demands, and competition. The people at the software company have to balance these forces and decide when it is the right time for software to be released.

If we were to wait until our software was bug free (relatively speaking) then we would be out of business. We do have an objective standard that we use to determine if software is ready for release and it has to do with the severity and quantity of bugs. We divide bugs into three severities 1, 2, and 3 with criteria for what those categories are. We will not release software with cat 1 errors, cat 2 errors have to be justified to the release team (which includes representatives from all aspects of our organization) and we fix as many cat 3 errors as possible.

Comparing our software to that developed by NASA, or air traffic control software is not a good comparison. First, you can see a difference in the prices, also there is not the same competition for NASA software that we face in the ERP business, expectation must also be higher for NASA software because of how it is going to be used, and NASA has far more resources available for developing their software than we do. Those are all factors in deciding how many bugs can exist in the software when you go to market.

It's difficult to simplify this discussion to things like greedy corporations, or poor development standards because it is much more complex than a one or two dimensional issue. There are many factors involved, but the biggest factor is market forces. Software developers are in business to make money and stay in business, the only way they can do that is to produce a product that customers will buy.

RE: Another Bug Hunt

Chris-232075 — Thu, 19 Jan 2006 17:45:00 GMT

Thanks for your comments. First let me say that I appreciate your site, your editorials and the tremendous resource that SQL Server Central provides. I use it daily, and I really appreciate you.

I also want to apologize for my self-righteous tone. I'm usually a bit more diplomatic, but this is an issue I feel strongly about.

I can't control governments, courts or the insurance community. Nor can I control unreasonable expectations of my customers with regard to issues of timing (for example, expecting a 12 month product to be delivered in 12 days).

However, like all developers and DBAs, I can control the quality of my own deliverables, and I am not sympathetic to the "all software has bugs" take on things. A co-worker once told me that database relationships are "textbook" stuff, not “real world”. Another co-worker tried to make the case that source control is needless bureaucracy. A 3rd party component I recently purchased for $500 only worked after three days of trial and error because their support documentation didn't bother to mention a couple of necessary "workarounds". A colleague told me just today that normalizing his table to 3NF (41 fields) was unnecessary (1 table normalized to 15). And, last week, I had to uninstall and reinstall SQL Server 2000 (Enterprise edition) to get Full-Text Search installed (it was not installed during the initial installation and I could not get it to install during a customized install).

You are correct that delivering bug free software for business applications is unrealistic. But I believe that the primary reason is not because of the complexity of our work, but because of the fact that many (if not most) in the development community have seriously low standards for quality with apologists to defend them. In my view, that’s not acceptable. Not from MS, not from Oracle and not from me. I honestly believe that we software/database developers can do better ... much better.

Yes ... bug free. I'm very serious. NASA software engineers' deliverables must be bug free when used in the space shuttle. Software used by air traffic controllers better be bug free before the end users put it to use. Electronic voting machines ought to be bug free before November. Am I being unrealistic here? Lives and democracy don't depend on my deliverables, but then again, my products are not nearly as complex as the space shuttle.

Thanks for listening and letting me vent.

RE: Another Bug Hunt

Steve Jones - SSC Editor — Thu, 19 Jan 2006 16:14:00 GMT

Chris,I'm sure you do a good job, or your clients would not be happy. But bug-free? c'mon, get real. If a few hundred thousand people started hitting your software would it still be bug free?I know MS can do a better job and without a doubt the marketing/financial people get in the way of the quality of the products, but the same pressures are everywhere. Until the government or courts or insurance industry starts to hold people accountable, they will not do a much better job.Don't forget. This software is used in millions of DIFFERENT environments. I've seen people complain about lots of different pieces of software, MS or not, that have worked flawlessly for me.

RE: Another Bug Hunt

jasonhatfield01 — Thu, 19 Jan 2006 14:53:00 GMT

Maybe somebody needs to offer 3rd party support for software that companies retire. NT is an example of that could be supported by 3rd party for price.

Interesting idea and one that I've never seriously considered. But how do you patch an OS or application without the original source code? I'm by no means a developer, am I missing something?

RE: Another Bug Hunt

Chris-232075 — Thu, 19 Jan 2006 09:16:00 GMT

I want to respond to Steve's comment that "every software has bugs".

I'm not perfect, but I bend over backwards to provide deliverables that are bug free. Do I succeed? Most of the time. When I don't succeed, are my bugs resolved quickly? Almost immediately. Am I smarter or more knowledgeable than my colleagues? No, but I care about the quality of my deliverables, and I do whatever is necessary to deliver bug free products.

C'mon Steve. The development community needs to quit delivering crap (defined as bug infested software that needs to be supported for weeks, months or years) and raise the bar.

RE: Another Bug Hunt

K. Brian Kelley — Wed, 18 Jan 2006 23:27:00 GMT

Probably a better analogy is this:Take a car from 1930...it may run, but new cars are safer, more efficient, and have new features. Do you expect Ford to install seatbelts, airbags, and antilock breaks in your 1930 Ford?From http://digg.com/security/Microsoft_to_release_WMF_patch_at_2PM_PST_todayThe differences in OSes go beyond a basic paint scheme. They are different systems altogether under the hood and therefore the analogy to retrofitting a car of yesteryear is probably a more reasonable one.

RE: Another Bug Hunt

Richard Sisk — Wed, 18 Jan 2006 22:53:00 GMT

Well, upgrading software is not exactly like selling a car with a new paint scheme so I'm not sure the analogy is correct.

Just in the 11 years that I have worked at this company, we have upgraded from DOS, to Windows 3x, WIN95, 98, to NT operating systems. We've modified our installations from floppy based, to CD, to web based distribution. The develoment system has gone from 16 bit, to 32 bit, to .NET based. In addition we have fixed bugs and added features to the software. Also upgraded support for the database from Sybase to SQL 4x, SQL 6x, 7x, 2000 and 2005. So there are other external factors which force us to upgrade the software to keep pace with changing technology.

Another difference from the car analogy is that our customers pay a maintenance fee which covers the upgrade of the product. This helps us and them stay current and add new features. Thats a totally different model than the auto industry uses.

All that said, there are customers that decide to stay behind on older versions of our software, and we do end up supporting them. We just do our best to encourage them to upgrade. Also, there is a risk for us as a software vendor that when they are faced with a decision for an upgrade, there is always a chance that they will select to go with a competitor. So we are not in the business of forcing customers to upgrade just to get more money out of them, but that is the way this business works (by the way, we sell ERP software and most deals are in the 100K-1M dollar range, just to give you an idea of the business I am in).

Now, I do agree with you that there may be a market for 3rd party support of retired products. In fact we have a channel of 3rd party vendors that resell and support our product and they sometimes do sell support for retired versions.

You know, there may be better ways to sell software, but for now this makes the most sense for our business.

RE: Another Bug Hunt

Bob Hoffman-209065 — Tue, 17 Jan 2006 11:10:00 GMT

So does that mean if I buy a new car this year and the next years version has a different paint scheme but is the same vehicle under the skin, I should buy another new one, just to keep current?Nice try, but no sale here.I purchase software based on lots of criteria. One being will I get my investment worth in time. Other than OS's, most software is designed to perform a specific task. In a business environment, if you purchase today what you need and it works now and 5 years from now, why should you upgrade? Not everybody needs the "latest and greatest" software versions. We still use Office 97. Why? Because it works and most of the users don't even use 75 - 80% of the features it has. I have been to M$ seminars where the speaker admitted they made that version too good and up to Office12, there has not been any real advancement in the program. Certainly not to justify the cost to upgrade to gain usability that won't be appreciated. We shall see what O12 offers. Same stuff, new dressing? Software better last longer than a few years or become so cheap that it just becomes better to upgrade based on financial reasons alone. Maybe somebody needs to offer 3rd party support for software that companies retire. NT is an example of that could be supported by 3rd party for price.Some software companies appear to be following the plan of a local contractor here back in the 80's. He built cookie cutter design strip malls on a lot of properties in town. Which led him to build some of the larger malls in town. Problem was, he had to keep starting new projects to acquire new funding just to complete the other projects already under construction. In the end, he ran out of new projects and it caught up with him. Now the properties are owned by others that purchased them for pennies on the dollar.If software companies want to sell new enhancements to a base product to generate new revenue, that's fine. They can always sell new versions to new customers and upgrade current customers a long the way, if they want. Just state the programs shelf life on the box when its support will expire. Same as shelf life for the food you buy. Then I can decide if I want it or not.

RE: Another Bug Hunt

Richard Sisk — Tue, 17 Jan 2006 09:11:00 GMT

Being in the commercial software development business myself, I can attest to the fact that it makes little business sense to support the older versions of the software. We are constantly encouraging customers to upgrade to our latest version, that way our development teams can focus on fixing issues in only a few code bases, not everything we ever sold. This frees up resources for development of better functionality and staying up with the latest technologies.

Companies have limited resources and have to decide how best to apply those resources not only to increase profits, but usually just to stay in business in a competitive environment. Just because MS is huge does not mean that they do not face the same kind of pressures that the rest of us do in the software business.

So excpecing MS to update older versions of their OS does not make sense to me.

RE: Another Bug Hunt

Greg Hartlaub — Tue, 17 Jan 2006 06:31:00 GMT

Hi Steve,I agree with you. Although I doubt Microsoft will make resources available to look at older versions and provide support/upgrades even for a fee. That activity will not improve the bottom line.I believe pressures from Wall Street will prevent MSFT from fixing past versions. As a small investor I see the pressures on Microsoft from the likes of Google, Yahoo, Sony, etc. And I know, you don't like Sony. I certainly hope Bill Gates and crew will make many improvements on current and future products down the road but I think they'll let sleeping dogs lie.

Another Bug Hunt

Steve Jones - SSC Editor — Mon, 16 Jan 2006 16:49:00 GMT

A good thing that SQL Server 2005 is complete because it looks like Microsoft is again shifting resources. Based on the security news of the past few weeks, Microsoft is on another bug hunt, scouring code for problems and potential vulnerabilities. Does anyone remembers the great patch year of 2002 with SQL Server, where we got lots (too many) patches being released? It seemed like every few weeks we were applying some "critical" SQL Server patch.

It has gotten much better, though I hope this is because most of the problems were found and not because people aren't paying attention.

Yesterday I wrote about the analysis of slow fixed by Microsoft, which may be true, and seems like human behavior to me. I'm not giving MS a free pass, however. I think that they have gotten much better in fixing things and they are doing a good job. But they can, and should do better.

I know they have been training people to write better code, and I know it costs money to look back at older OS's. And I'm not saying they should be supporting Windows 98, but I do think they should be scanning ALL their code for the "types" of issues that have been found (buffer overflows, malformed inputs, etc) and proactively issuing patches during the regular release cycle for these things.

And they should be going back to Windows 98 at least, just to be sure that those code bases can be patched. Maybe they would charge for those patches, but I think it would be a great PR move to just release those patches. Hiring 10 people to be bug catchers couldn't cost much, especially compared to the positive advertising they'd get.

Every software has bugs and the OS's are no exception. Even Linux and its core packages are constantly releasing patches for issues they find. Both camps do a good job of working on and releasing patches, just on different schedules. But they can do better.

I just hope Microsoft bundles these patches up and limits the burden on admins with a set schedule. And maybe a month off here and there.

Steve Jones