Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
Log in  ::  Register  ::  Not logged in

One Database to Rule Them All

By Steve Jones,

This is what you build to juggle 6,000 tweets a second. That's the headline that caught my eye and it's about the challenges of Twitter and the data that they handle. Twitter definitely has a tough problem, one that few of us have, but perhaps they can help us learn to better deal with our own data from their experiences on an edge case.

The story is journalistic, not so technical, but it is interesting. Twitter has struggled with a blend of data that is partially crucial and must be consistent now (usernames) and other data that can be a bit out of date (tweets). They also have lots of unstructured data (photo/video) that is combined with more traditional, structured data. They've used a few different database platforms to store this data and assemble it with their application. That's the same things that most of us also do when we deal with many different types of data.

However Twitter is trying to find away around dealing with disparate systems. They've had a number of engineers working on Manhattan, their database designed to handle both structured and unstructured data. And because they work for Twitter, this platform is designed to manage all of this data with very high workload demands at scale.

It will be interesting to see if they come up with any innovative ideas. Certainly SQL Server already has options for managing structured and unstructured data, though perhaps not at the scale Twitter needs.

Total article views: 280 | Views in the last 30 days: 1
Related Articles

Unstructured Data in SQL Server

More and more of our data does not fit neatly into a structured, relational model of rows and column...


Unstructured Data

I remember when Windows 3.1 started to gain widespread deployment in businesses. With all it's WYSIW...


Andy on Twitter

Just prior to the 2009 PASS Summit I posted about giving Twitter a try, and thought I’d report back ...


What is Twitter?

I saw an interesting blog post from Andy Leonard (@AndyLeonard) recently on what he thought Twitter ...


Twitter Your Issues

Twitter is one of those interesting Web 2.0 ideas that is hard to analyze. Is it valuable? Steve Jon...


Join the most active online SQL Server Community

SQL knowledge, delivered daily, free:

Email address:  

You make SSC a better place

As a member of SQLServerCentral, you get free access to loads of fresh content: thousands of articles and SQL scripts, a library of free eBooks, a weekly database news roundup, a great Q & A platform… And it’s our huge, buzzing community of SQL Server Professionals that makes it such a success.

Join us!

Steve Jones

Already a member? Jump in:

Email address:   Password:   Remember me: Forgotten your password?
Steve Jones