Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 

No reason to abandon SQL

By Phil Factor,

I’m told by friends who provide enterprise-level hardware for large applications that they are seeing a significant trend away from the huge ‘database-scale’ hardware behemoths towards commodity hardware. NoSQL Database systems such as MongoDB are being selected despite their relative immaturity, purely on the fact that they are so easy to justify on grounds of cost-saving. They can use shedloads of commodity hardware clusters, so can scale up so much more cheaply. It can cost ten times as much to expand capacity of a relational system.

Of course, reality is different, because there are other costs. The typical NoSQL solutions have been refined for web applications rather than a busy OLTP task. They are best with offline processing and analysis of large-scale data that doesn’t change much, and where the underlying schema hasn’t been figured out. It doesn’t have the facilities required for commerce such as transactionality. All the services that SQL provides, if they are required, need to be written now at the application level by the developers; whether that is aggregation, joins, scheduled jobs, ETL or transactions.  This is all likely to add costs into the equation.

Abandoning SQL can be painful for other reasons. Working with MapReduce, for example, is a cultural shock to any SQL developer because it is more like writing execution plans by hand. Whereas, with SQL, you just declare what result you want, no matter what the size and distribution of the data, MapReduce required hand-crafted strategies that would need to be refined as the data changes. I stopped doing that sort of thing over twenty years ago.

Could the answer be to add SQL to ‘BigData’ systems such as Hadoop? There are already many SQL On Hadoop solutions such as Hive, Impala, Stinger or Hadapt but . In a new twist, the Hadoop stack is now host to a full-featured SQL database called ‘Splice Machine’, capable of running transactions and analytics simultaneously.

The clever thing about the Splice Machine, which uses Apache Derby as its database engine, is that the SQL planner and optimizer has been extended to create execution plans that can make use of Hadoop's parallel architecture so that processing is done at the individual node. The intermediate data is then spliced back together to create the result.

The early signs are that we can now run a SQL-based relational database with distributed execution plans over commodity hardware, leaving just the task of splicing together of the result to the engine itself, then why can’t Microsoft or Oracle do it? The dream of a linear cost for scaling a relational database now seems attainable, and it looks like the open source community may have somehow got there first.

Total article views: 154 | Views in the last 30 days: 4
 
Related Articles
ARTICLE

A Beginners Look at Hadoop

Hadoop has been making a lot of noise in the Big Data world. Despite my lack of Linux experience I ...

BLOG

Introduction to Hadoop

Hadoop was created by the Apache foundation as an open-source software framework capable of processi...

ARTICLE

Scaling Out the Distribution Database

Replication is a great technology for moving data from one server to another, and it has a great man...

ARTICLE

Scaling Out

Kumar discusses the differences between scale up and scale out, then does a very good walk through o...

ARTICLE

The Power of Hadoop

Hadoop is an open source framework for working with data, and one that Microsoft has adopted. Is it ...

Tags
database weekly    
editorial    
 
Contribute

Join the most active online SQL Server Community

SQL knowledge, delivered daily, free:

Email address:  

You make SSC a better place

As a member of SQLServerCentral, you get free access to loads of fresh content: thousands of articles and SQL scripts, a library of free eBooks, a weekly database news roundup, a great Q & A platform… And it’s our huge, buzzing community of SQL Server Professionals that makes it such a success.

Join us!

Steve Jones
Editor, SQLServerCentral.com

Already a member? Jump in:

Email address:   Password:   Remember me: Forgotten your password?
Steve Jones