Azure Data Factory : Your First Data Pipeline

  • ramakant

    SSC Veteran

    Points: 275

    Comments posted to this topic are about the item Azure Data Factory : Your First Data Pipeline

    Ramakant Dadhichi
    Technology Enthusiast

  • rgoerwit

    SSC Enthusiast

    Points: 188

    This is a very good article.

    I'd be interested in your view of Azure Data Factory overall maturity and utility, compared to, say, other ETL tools.

    My shop has had a mixed experience with it.  The very first time I tried to create an ADF pipeline came in response to a need for a very simple exchange of information with an outside vendor.  All they wanted was for us to upload a CSV file with "people" data in it, generated off of one of our administrative systems, to an Amazon S3 bucket.  Anyone working in a modern, multi-cloud shop is familiar with this sort of integration.

    Normally I'd have used on-prem SSIS (+CozyRoc) or Jitterbit for this.  But to expand my horizons, I tried, in turn, an Azure Logic App and ADF.

    Shockingly, neither the logic app nor ADF could copy data into an S3 bucket.

    It was hard, after that, for me to think of ADF as anything but a toy - or a tool for people who live in a fairly monolithic shop.

    Have things changed recently?  Can ADF connect to a broader set of tools and services that in the past?  Is it yet possible to do simple things like drop a CSV file into an S3 bucket, without going on some sort of coding adventure or buying a third-party plug-in?

     

    • This reply was modified 2 months, 2 weeks ago by  rgoerwit.
    • This reply was modified 2 months, 2 weeks ago by  rgoerwit.
    • This reply was modified 2 months, 2 weeks ago by  rgoerwit.
    • This reply was modified 2 months, 2 weeks ago by  rgoerwit.
  • ramakant

    SSC Veteran

    Points: 275

    Thank you so much 🙂

    No S3 bucket is only available as source as of now. Its a shame that it is not available as sink which would make it a complete solution. Probably in future they will have it as well, hopefully.

    Attachments:
    You must be logged in to view attached files.

    Ramakant Dadhichi
    Technology Enthusiast

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply