Blog Post

Technological Movements and the Herd Mentality....


It's amazing to me how the herd mentality works in Technology (and many other things) sometimes. I get the idea that since there are so many new things happening in Technology all the time, it is automatically assumed that "new = good" and "old = bad". I believe that this mentality has done much to unjustifiably hurt the database in many people's eyes. After all, it is very likely that the oldest technological component used in your applications is the database. Application languages come and go all the time, it seems, but the database has stuck around for a long time, prompting many people to look distastefully at the database wondering why this piece of technology is still even around. I know that such a mentality has been at work in some of the latest Technological innovations, especially in the recent NoSQL movement. Now, I don't want to disparage said movement, and I get the idea that the people responsible for these technologies are likely not as anti-Relational as are many of the followers, but it is those followers that make lots of noise and can generate lots of confusion. At the end of the day the Relational Database is a MASSIVELY popular platform and for good reason; it's very scalable and flexible and has been in use in a wide variety of applications for decades. Nonetheless, in my experience it is not very well-liked, in spite of its large user base, and several myths have appeared which try to debunk the Relational Database:

Myth #1: "Relational Databases are not scalable" This one is the rallying cry of some NoSQL pundits, who point at the incredible volumes generated by the Web 2.0 applications like Facebook or Twitter. There are 2 points to be made in reply to this. First, in reality the majority of the applications in the world don't have 600 million users. Such an application can most accurately be described as an "edge case", with special performance requirements. Second, Facebook does use MySQL with the InnoDB engine, which is a relational database, even with those huge volumes. So in fact Relational Databases have truly scaled in the real world for a long time, and continue to do so. So where does this accusation come from? I think it comes from the fact that it is not easy to scale OUT when using Relational Databases. It's easy to scale web servers out by adding more servers, but with databases it's not that simple. I think that many developers WANT the database to be more like what they are used to. While this is understandable, it is also not realistic. After all, web server scale-out doesn't have to deal with the issues inherent in a persisted data store like database servers do. Also, while Relational Databases don't scale out, they have other ways to scale. However, proper design is paramount; you can't just blindly add another server and have the problem go away.

Myth #2: "The database should be mated to the application" One of the great strengths of the Relational Database is its very flexible and easy to use SQL query language. This allows the database to be "decoupled" from the application and used as an independent data store. You can import data into it, export data from it, and have your users, support professionals, developers and testers run queries against the database all day long using a simple SQL Editor. Many of the NoSQL databases coming out now don't have this; the data design of these databases is not easily mapped to real world concepts (such as when you model entities after real-world business concepts). Some of these databases don't have easy query languages like SQL, and this is fine, but what is interesting is how few people talk about this; they just gloss over it as if it didn't matter. Now, I don't know about you, but nearly all the applications I've ever worked with need easy access to the data in many different ways, and not having it is a deal-breaker. So while it would be nice to have a database that integrates seamlessly with the application, it is much more important to have a flexible and easy to use independent data store. As a matter of fact, the recent trend towards Master Data Management points to the concept of a decoupled data store that many systems can use as the single, correct data store for critical pieces of information.

Myth #3: "SQL is an "old" language, and old is bad" SQL is certainly old, but there are advantages to this. If you know that some languages last longer than others, wouldn't it be interesting to have as much business logic as possible in that language so that you're not forced to upgrade it every few years? So, "SQL" = "old", but "old" <> "bad". The same could be said of some other old "technologies" such as breathing and eating 🙂

Myth #4: "The database should be disposed of" There is a dream among Application Developers that they should never have to deal with databases. They should just be able to persist their data structures as they exist in the application, and that's it. It sounds great, and people are always talking up the Object Oriented Databases because of this. There is no doubt that this is a great advantage of Object Oriented Databases, but the question is, why hasn't it ever taken off? What has stopped it? Why do so few people talk about the downsides? Is it once again a matter of bias? I believe it is, and it causes lots of confusion and misguided expectations when you expose one side of the story and ignore the other side.

Myth #5: "It's all about Technology." Wrong. It's all about the Business. The business is what creates wealth, and Technology is used to support the Business. The minute you get away from the Business and focus only on Technology you have the makings of a useless, failed project. And the reality is that the Database does a great job at supporting the wild and crazy world of business, not the neat-and-tidy world of theoretical application design.

Myth #6: "Databases are not easily testable." If there is any truth to this it is because the database is generally de-emphasized in favor of application technologies. It is certainly possible to write tests for database code, just like any other code, but it's just not as popular. Also, there is an interesting paradox at work regarding this topic: Database Constraints, which have been around for many years, are a GREAT form of tests, yet they are very unpopular. How is it that so many people are in favor of automated tests but so few people are in favor of Database Constraints? Again, I believe it's a matter of bias in favor of the known versus the unknown.

So, there is my rant for two pennies. What are your thoughts? Do you believe that databases have been victimized as described above?