Related Articles
Newsletter
Join our blog
Join other Azure, Power Platform and SQL Server pros by subscribing to our blog.
Start with the FREE community plan and get your lifetime access to 20+ courses. Get Instant Access Now!
Need help? Talk to an expert: (904) 638-5743
Private Training
Customized training to master new skills and grow your business.
On-Demand Learning
Beginner to advanced classes taught by Microsoft MVPs and Authors.
Bootcamps
In-depth boot camps take you from a novice to mastery in less than a week.
Season Learning Pass
Get access to our very best training offerings for successful up-skilling.
Stream Pro Plus
Combine On-Demand Learning platform with face-to-face Virtual Mentoring.
Certification Training
Prepare and ace your next certification with CertXP.
Private Training
Cheat Sheets
Quick references for when you need a little guidance.
Nerd Guides
Summaries developed in conjunction with our Learn with the Nerds sessions.
Downloads
Digital goodies - code samples, student files, and other must have files.
Blog
Stay up-to-date on all things Power BI, Power Apps, Microsoft 365 and Azure.
Community Discord Server
Start here for technology questions to get answers from the community.
Career Guides
Breaking into the field? Let these guides help get you started with a plan.
Affiliate Program
Earn money by driving sales through the Pragmatic Works' Training Affiliate Program.
Reseller Partner
It's time to address your client's training needs.
Foundation
Learn how to get into IT with free training and mentorship.
Management Team
Discover the faces behind our success: Meet our dedicated team
Contact Us
How can we help? Connect with Our Team Today!
FAQs
Find all the information you’re looking for. We’re happy to help.
Today I’m excited to talk about one of the new releases in Azure that gives you continuous integration and deployment using Azure Data Factory. This new release is an Azure Data Factory visual interface that allows you to export any of your Data Factory components as an Azure Resource Manager (ARM) template.
When you do these exports from your Data Factory, it will generate 2 files. The template file, which will contain all the Data Factory metadata for the pipelines, data sets, etc., as well as a configuration file, which will contain environment parameters that will be different for each of your environments. So, if you’re going to create a development, a test and a production environment, each one will be different.
You also can specify things like storage containers, Databricks clusters, etc. After you’ve deployed this, you’re going to create a new factory for your environment. You’re also going to associate your Visual Studio team services get repository to that Data Factory, enabling source control versioning and collaboration uses.
Next, you’ll set up your Data Factory with VSTS. This is where all the developers can author data factory resources, such as pipelines, data sets and other components. Once you have this development area set up, developers can modify the resources and debug them right in the interface, along with checking performance. They’ll also have the option to create a PR from their branch to master or create a collaborative branch to get the changes reviewed by peers.
Once they are satisfied with the changes and are ready to go to production, they set it in the master branch and can then publish it to the development Data Factory. Or they can promote each of those environments through exporting those ARM templates when they’re ready from the master branch, or any other branch.
So, you export the template and it gets deployed with different environment parameters to test and production environments. From there, you can also set up VSTS release definitions to automate the deployment of your Data Factory to multiple environments.
The benefit with this is it opens the opportunity to bring your true dev test and production environments, that you’re used to in your local environment using SSIS or other ETL tools, to Azure. This tool offers a tremendous amount of power and it’s getting better all the time.
I recommend you investigate this further and if you have any questions, or any input to share with us, we’d love to talk about it. Whether it’s about my topic today or anything Azure related, click the link below to contact us – we’d love to help.
ABOUT THE AUTHOR
Free Trial
private training
Newsletter
Join other Azure, Power Platform and SQL Server pros by subscribing to our blog.
Leave a comment