"Low Code" pains in Azure Data Factory

  • I've inherited an azure environment that relies heavily on ADF pipelines created in the portal. I'm finding it extremely difficult to troubleshoot, or do any input/output mapping. I wish I could just see the code being run and how it's being called.

    Does anyone have any suggestions for getting an ADF environment into shape, or resources for using ADF in a more code oriented manner applinked?

    Thanks!

    • This topic was modified 1 month, 2 weeks ago by  oplewini.
  • Have you got an example of your pain points?

    When in the designer you can preview the data for some of the activities and when it has run you can see the input and output of each activity.

    You can also drill into the pipeline runs via the Monitor area in Data Factory and see the same.

     

    • This reply was modified 1 month, 2 weeks ago by  DNA_DBA.
    • This reply was modified 1 month, 2 weeks ago by  DNA_DBA.
  • Conversely, go the other direction. Monitor where the data is being moved to in order to identify which processes are causing pain. Also, look at where the data is being moved from. Both, of course, depend on what & where you're actually moving data, but it gives you a good guess anyway.

    ----------------------------------------------------
    The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood...
    Theodore Roosevelt

    The Scary DBA
    Author of: SQL Server 2017 Query Performance Tuning, 5th Edition and SQL Server Execution Plans, 3rd Edition
    Product Evangelist for Red Gate Software

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply