How to Create a Data Lake Shortcut in Microsoft Fabric

In this tutorial, Austin Libal, a trainer at Pragmatic Works, walks viewers through integrating an existing Data Lake into a Microsoft Fabric environment. This step-by-step guide demonstrates how Power BI developers and Azure users can efficiently access external data without migrating large datasets.

Introduction

Austin begins by introducing himself and highlighting the importance of Microsoft Fabric, a platform for modern analytics and data integration. He references his previous “Learn with the Nerds” session, which covered an end-to-end Microsoft Fabric solution and the concept of a Lake House. For those unfamiliar with a Lake House, he recommends watching that detailed session before proceeding with this tutorial.

Objective

The main goal of this session is to integrate an existing Data Lake into a Microsoft Fabric workspace. This allows users to access pre-existing data stored outside of Fabric, leveraging it within the Fabric environment for analytics and reporting.

Step 1: Navigating the Power BI Service

• Austin opens the Microsoft Power BI “Learn with the Nerds” workspace.
• He highlights the importance of understanding different objects within the workspace, including Delta tables and the star schema setup with fact and dimension tables.
• Options for data acquisition include Dataflow Gen 2, data pipelines, shortcuts, or manual file uploads.
• The tutorial specifically focuses on creating a shortcut to integrate an existing Data Lake.

Step 2: Selecting the Data Source

• Users can connect to internal or external data sources.
• In this example, the Data Lake is hosted in Azure, so Austin selects the Azure Data Lake Storage Gen 2 shortcut connector.
• The connector requires the Data Lake URL, which can be retrieved from the Azure portal.

Step 3: Accessing the Azure Portal

• Austin demonstrates navigating the Azure portal to locate the Data Lake Storage account.
• Within the settings, he identifies the correct endpoints and selects the Data Lake Storage endpoint containing ".dfs.core.windows.net".
• This URL is then copied to the clipboard for use in Power BI.

Step 4: Creating the Shortcut in Power BI

• The URL is pasted into the Power BI connection setup.
• Austin authenticates using single sign-on to ensure proper access permissions.
• The shortcut is named after the Data Lake (“PW YouTube Lake”) and points to the appropriate subpath, which in this example is the “training” folder.
• Users can verify subpaths by checking containers in the Azure portal, ensuring they connect to the correct folder containing CSV and Parquet files.

Step 5: Finalizing the Connection

• Once authentication and paths are confirmed, the shortcut is created.
• This allows seamless access to external data directly within the Lake House environment without migrating large datasets.
• Users can view files, rename, or delete items directly from the Fabric workspace.

Benefits and Best Practices

• Integrating an external Data Lake avoids unnecessary data duplication and storage costs.
• Shortcuts provide a simple UI method to access large datasets efficiently.
• This method ensures continuity of data access for organizations with pre-existing lakes while leveraging Microsoft Fabric’s modern analytics capabilities.

Conclusion

Austin wraps up the session by emphasizing the ease of integrating external Data Lakes using shortcuts and encourages viewers to explore more Pragmatic Works content on Microsoft Fabric. He highlights that the product is still in preview, but the tools and techniques shared provide a strong foundation for analytics projects. For deeper learning, Pragmatic Works’ on-demand learning platform offers an extensive video series on Microsoft Fabric for Power BI developers.

Don't forget to check out the Pragmatic Works' on-demand learning platform for more insightful content and training sessions on Fabric and other Microsoft applications. Be sure to subscribe to the Pragmatic Works YouTube channel to stay up-to-date on the latest tips and tricks.