Big Data for SQL folks: The Technologies (Part II)

  • Frank Banin

    Ten Centuries

    Points: 1323

    Comments posted to this topic are about the item Big Data for SQL folks: The Technologies (Part II)

    Frank Banin
    BI and Advanced Analytics Professional.

  • sherifffruitfly

    Ten Centuries

    Points: 1198

    regarding the "is ETL dead?" question:

    every time a new shiny dev toy comes out, the silo'd tunnel vision ADD kiddiez are always quick to scream "zomg everything else is obsolete now!".

    it hasn't been true even once yet; i don't imagine it'll be true this time either - and for the same reason as usual: the invention of the screwdriver didn't eliminate the need for hammers. you just need better tool users to decide which is best in a given situation.

    -sff

  • Luke C

    Right there with Babe

    Points: 790

    Would like to add to the ETL discussion. One of the primary reasons for ETL is to extract data from OLTP systems and minimize the impact of long running reports that may scan billions of rows. Offloading that load into a data warehouse is essential to keeping the OLTP system running at peak and not impact end users relying on your transactional application. Also, most disparate OLTP database instances are not configured to support these types of query profiles and have been setup to optimize random I/O vs long sequential reads. I believe the LDW definitely has it's benefits and place but not at the cost of impacting transactional applications. My bet is that the LDW approach will be adopted to combine data from Hadoop, MongoDB etc and your traditional structured data in the near future.

    Luke C
    MCSE: Data Platform, MCP, MCTS, MCITP - Database Administrator & Database Developer

  • sql.arizona

    SSC Rookie

    Points: 29

    "We have seen that the Big Data solutions out there are not one-size-fit-all. The technologies are also evolving very quickly, making it a quagmire for most of those charged with the responsibilities of Big Data implementations, especially on the heels of well publicized initial Hadoop adoption challenges."

    How is a person suppose to educate themselves on this stuff? Things are changing so fast, and all this technology is used by only a few giant companies. Reading about it is fine, but getting hands on seems a daunting task.

  • Siderite

    Valued Member

    Points: 69

    Be great if the author could publish a part 3 to this series of articles. a lot can happen in 2yrs in this big data world. especially with Sql Server latest releases and advent of a mature azure platform.

    Go Frank!

  • Alan Burstein

    SSC Guru

    Points: 61067

    Just read this. Amazing work Frank!

    Looks like SQL 2016 is going to be very Big Data-Centric from what I have read.

    "I cant stress enough the importance of switching from a sequential files mindset to set-based thinking. After you make the switch, you can spend your time tuning and optimizing your queries instead of maintaining lengthy, poor-performing code."

    -- Itzik Ben-Gan 2001

  • akljfhnlaflkj

    SSC Guru

    Points: 76202

    Thanks for a great article.

Viewing 7 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply