Big Data for SQL folks: The Technologies (Part II)

  • Comments posted to this topic are about the item Big Data for SQL folks: The Technologies (Part II)

    Frank Banin
    BI and Advanced Analytics Professional.

  • regarding the "is ETL dead?" question:

    every time a new shiny dev toy comes out, the silo'd tunnel vision ADD kiddiez are always quick to scream "zomg everything else is obsolete now!".

    it hasn't been true even once yet; i don't imagine it'll be true this time either - and for the same reason as usual: the invention of the screwdriver didn't eliminate the need for hammers. you just need better tool users to decide which is best in a given situation.


  • Would like to add to the ETL discussion. One of the primary reasons for ETL is to extract data from OLTP systems and minimize the impact of long running reports that may scan billions of rows. Offloading that load into a data warehouse is essential to keeping the OLTP system running at peak and not impact end users relying on your transactional application. Also, most disparate OLTP database instances are not configured to support these types of query profiles and have been setup to optimize random I/O vs long sequential reads. I believe the LDW definitely has it's benefits and place but not at the cost of impacting transactional applications. My bet is that the LDW approach will be adopted to combine data from Hadoop, MongoDB etc and your traditional structured data in the near future.

    Luke C
    MCSE: Data Platform, MCP, MCTS, MCITP - Database Administrator & Database Developer

  • "We have seen that the Big Data solutions out there are not one-size-fit-all. The technologies are also evolving very quickly, making it a quagmire for most of those charged with the responsibilities of Big Data implementations, especially on the heels of well publicized initial Hadoop adoption challenges."

    How is a person suppose to educate themselves on this stuff? Things are changing so fast, and all this technology is used by only a few giant companies. Reading about it is fine, but getting hands on seems a daunting task.

  • Be great if the author could publish a part 3 to this series of articles. a lot can happen in 2yrs in this big data world. especially with Sql Server latest releases and advent of a mature azure platform.

    Go Frank!

  • Just read this. Amazing work Frank!

    Looks like SQL 2016 is going to be very Big Data-Centric from what I have read.

    "I cant stress enough the importance of switching from a sequential files mindset to set-based thinking. After you make the switch, you can spend your time tuning and optimizing your queries instead of maintaining lengthy, poor-performing code."

    -- Itzik Ben-Gan 2001

  • Thanks for a great article.

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply