Creating New Azure Data Factory Data Flow File Data Sets

Are you looking to reuse file connections you have set up in Azure Data Factory prior to using Data Flows? Maybe you have a case where you’re building some pipelines and you’re pulling data from files, etc. and you want to use reuse those for a Data Flow. When I tried this I discovered that you cannot reuse these existing text file data sets, but you must recreate them. In today’s post, I want to share a quick tip on how to recreate these file connections to use in Data Flows.

Step 1 (Screenshot below): Create a new Data Flow in Azure Data Factory using your work canvas. Under the settings pick a data set and point it towards the file that you have previously set up. On the left side, you should see your previously made data sets. I have a couple previously created that are still there, but not available as selections in my Source Data set list within the Data Flow, so they will need to be recreated. 

Step 2:  When I create a  a new data set and point it towards my Azure Data Lake linked service, I’ll now see the menu shown in the screenshot below. There is an option for delimited text, which I was using in my case. I choose delimited text and set up the data set. If you look at the JSON beneath it, you can see they have made some changes in the formatting.

When I move back to the Data Flow (see below), you will see the drop-down for source data and the newly created data set is now an option available to be used in my Data Flow. Note the files you previously made will not be available, but only the ones you recreate as new are available for selection.

While you may not be able to reuse previously existing connections, I hope this quick tip will make your process of recreating these files easier. If you're interested in learning more about file connections you have set up in Azure Data Factory using Data Flows, Azure Data Factory or Data Flows in general click the link below or contact us. We’re here to help you with any Azure product or service and take your business from good to great.

We all know life can get hectic. Here at Pragmatic Works, we're no different. But one of our goals is to learn something new about Azure every day, as things are constantly changing and being updated. Many people are still learning all the amazing things they can do within the Azure cloud and we want to help. Our posts in our Azure Every Day series are a great way to learn more about Azure each week.