SQLServerCentral Article

An Introduction to Flowlets in Azure Data Factory

,

Introduction

Flowlets is a new feature of Mapping data flows introduced in Azure Data Factory and Azure Synapse Analytics. Flowlets can be used to extract portions of an existing data flow and save them as separate "Flowlet" objects inside your data factory or workspace. It lets you reuse implemented logical blocks. We will discuss various features of Flowlets below.

Adding Flowlets as a source

Inside the data flow design surface, the add source button will now allow for optional Flowlets as a source:

Adding Flowlets as a transformation

Flowlets can be used as a transformation in the data flow.

Create a new Flowlets from an existing dataflow

To create new Flowlets from existing dataflow, open the existing data flow and click on turn on multi-select option (lower right in the image below):

Then, select the items in the data flow design surface:

Now, right-click on the selected objects and choose to create a new flowlet option:

A new Flowlets will be created, note that the source changed to input and sink changed to output respectively.

The Flowlets is available under Data flows category:

We can remove unnecessary columns from the Flowlets if required:

As the Flowlets input does not have any input data set, the data preview option is not available:

 

Debugging a Flowlets is a little different than debugging the data flow as Flowlets do not have any input data, we need to provide the data manually. Click on the debug settings:

Let's add input data manually:

Now we can reuse the same Flowlets Now we can preview the data in the output:

We can reuse the same Flowlets multiple times without creating it again. To use the Flowlets in other dataflow add the Flowlets like below:

Conclusion

Flowlets are very useful to reuse the complex dataflow logic multiple times without creating it again. As of writing, Flowlets is available as a public preview in the Azure Data Factory and Azure Synapse Analytics, The General Available will be announced in the coming months.

Rate

5 (1)

Share

Share

Rate

5 (1)