• xsevensinzx - Saturday, January 6, 2018 9:32 PM

    Jeff Moden - Saturday, January 6, 2018 7:22 PM

    Good article but I really don't understand why you say that relational databases are really only good for small scale stuff.  Is there no one that you would consider "large" that uses SQL Server, Oracle, or some other RDBMS?

    I think he is being relative here. I could be wrong there, but that's how I read it in the sense, small in this context may be large for you.

    There are plenty of case studies that show both SQL Server and Oracle can solve very large problems. However, as the article highlights, when you don't really have a data model or don't attempt to define a proper one with or without the proper expertise to tune the tech you have, then you're left with a pretty large problem that does not just mean an increase in data size. It could just mean poor business process. Therefore, taking a piece of tech with the assumption that it has great ease of use that your business can easily pivot to that you can just throw large quantities of data at, seems like a good choice until you realize it's not that simple.

    Kind of reaffirms the notion that it may not actually be a tech issues as much as a business process issue. I.e.: maybe your RDBMS could handle that large problem afterall, you're just doing it wrong or don't want to do it right. Which is sadly very true in a large amount of cases I bet.

    Regardless, it all translates to using the right tools for the job with the added excerpt -- the right tools your team knows how to use. Not what some hot new marketing blog is telling you to use.

    Heh... of course Steve is being "relative" here and so was the article he cited.  I absolutely get that what people think is large (for example, some of the larger health insurance  companies) really isn't compared to some of the juggernauts of the computational world (NASDAQ, for example, which suffers "Billions of transactions per day and has "Multiple Petabyte Databases" and single tables with "Quintillions of records" https://www.youtube.com/watch?v=AW87RzZJ0Z0).  Certainly, most of the stuff that most of us (denizens of SQLServerCentral.com) deal with is, indeed, quite "small to medium scale" compared to NASDAQ or even companies contained in the following list of companies ( https://customers.microsoft.com/en-us/search?sq=&ff=&p=0&so=story_publish_date%20desc )

    I also understand what is being said in the following quote from the article...


    There's an article that I think illustrates this well, and might be worth passing along to your developers. It's called Why Amazon DynamoDB isn't for everyone and it's worth a few minutes of your time to read. The gist of the article is that a NoSQL system, like DynamoDB, isn't as simple and easy as you expect. It also says that for many applications, a relational database is a better choice because it's often better understood and will solve most problems at small scale. For most of us, we don't really get past what I'd consider small to medium scale, and so we should stick with relational systems. 

    The cited article starts off by saying that...

    By 2004, Amazon’s business was already stretching the limits of their Oracle database infrastructure. In order to scale the growing business, AWS designed an award winning internal key-value store — Amazon Dynamo — to meet their performance, scalability, and reliability requirements.

    And, yes... despite the fact that there were "only" ""hundreds of millions of products" from Thanksgiving through Cyber Monday", there were also a huge number of transactions for each product sold that control the warehouse robots and "smart" conveyer systems ( http://money.cnn.com/2017/11/29/technology/amazon-cyber-monday/index.html ) very likely (IMHO) bringing the number of related data transactions to the "billions" per day that the NASDAQ sees.

    That cited article and Steve's good article on the same subject seem to be suggesting that when you actually do approach or achieve truly "large" scale, that an RDBMS isn't the way to go and, yet, there are huge RDBMS's, such as what is powering NASDAQ, that have been using an RDBMS for a long time. 

    That's my only point of contention with the article. Size doesn't actually matter.

    That being said and what DOES matter, I very much DO agree with the main thrust of the article.  We have computational juggernauts like NASDAQ and Amazon that have similar scale (again, IMHO).  The former uses an RDBMS and the latter uses something that (apparently) doesn't.  Why is that?  The answer is indeed "The Engineers" but, again, to highlight my point of contention with the article, it has nothing to do with scale as to whether or not an RDBMS was used.  It was the know how of "The Engineers" that allowed one giant of the industry to use an RDBMS and the other to use something else.

    Heh... and to rub it in a bit for the people (and especially the self proclaimed "engineers") that I've had to interview in the past, I'm thinking that the Engineers that built both the NASDAQ and Amazon each knew how to get the bloody current date and time from the system being used . 😉

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)