Azure Data Factory V2 Visual Design Tools

In today’s post I’d like to tell you about Azure Data Factory Version 2 (ADF V2). If you’re familiar with the first version of Data Factory, you know that designing anything meant that you had to write JSON. Getting started with ADF was hard for many as there were no visual tools. Data Factory is a powerful product for doing data integration and the cloud, but it could be a challenge to get it up and running.

The good news is that, with the new Azure Data Factory V2 (currently in preview), there are some great new visual design tools available. All the concepts available in V1 with your data sets, pipelines and activities are still there, but with new Gooee driven design tools.

As you start with defining your data sets, there are visual tools for doing this and connecting your data strings, all available through the Azure portal – no client tool installation needed. Once you’ve defined your data set, you can define a pipeline and then add activities to that pipeline.

This can now all be done with a drag and drop style interface. You can drag an activity onto your workspace and write the logic for conditions under which to fire the next action; all done through some powerful tools. Activities and features that are available in ADF V2 visual tools are:

• Invoke Data Lake analytics through a drag and drop and some configuration.
• Invoke Azure Machine Learning batch processing with hooks in to do this with visual tools.
• There are tools for HDInsight loads, such as running Hive or Pig jobs.
• They recently added a new activity for SSIS package executions, giving us the ability to execute SSIS packages from Azure Data Factory with visual tooling to do this.
• The ability to schedule pipelines through creating a trigger. If you’ve ever set up SQL Server Agent schedules, the scheduling that’s available with triggers will feel very familiar and you’ll have flexibility for how often you want to run it, for how long and to retry logic and such.

If you’ve used the first version of Azure Data Factory, I highly encourage you to look at these new visuals tools in V2. If you’d like to learn more about the new capabilities available in V2, we’d be glad to help. Click the link below or contact us to get help with this or any other Azure product or service.