Sharing Integration Runtimes Among Azure Data Factories

In this post I’ll talk about self-hosted integration runtimes and the ability to share them across Data Factories and tell you about a new feature that was announced in the Azure Data Factory space.

The integration runtime is essentially the connector that allows you to connect back to your on premises environment and safely and securely move data between Azure and that on-prem environment for Data Factories. This is a dedicated application for Azure Data Factory that’s similar to the on premises Data Gateway.

Here’s where this new feature helps. Up until now, Data Factories could not share integration runtimes. So, if you needed to set up different Data Factories to connect back to on-prem data, databases or flat files, etc., you would have to set up individual integration runtimes for those various Data Factories or pipelines going across multiple Data Factories.

With this newly announce feature comes some new terminology: shared integration runtime and a linked integration runtime. Shared integration runtime is basically the standard integration runtime you’re used to, however now it can be shared. And when it is shared, it will have a subtype that’s shared with other Data Factories.

So, you’ll have your main shared integration runtime and on top of that you’ll have a linked integration runtime, which is a linked integration runtime that references the infrastructure of another self-hosted IR. That link points back to a shared IR and allows you to share among multiple Data Factories.

With this straightforward process, you install the integration runtime in your environment, set up your linked service within your Azure Data Factory and then connect it through that linked service. Then you’re ready to pull the data that you need into the cloud and do transformations and push it out to Azure Data Warehouse, Azure Data Bricks, etc.

This cool new technology allows you to get your data to the cloud much easier and more efficiently. If you have questions around integration runtimes or Azure Data Factory or anything Azure related, you’re in the right place. Click the link below or contact us—we’d love to help.