Master Azure Databricks for data integration with our course that provides an exploration of connecting to various Databricks sources, made to give you the skills to use this powerful tool for data processing and insight sharing.
Dive into the world of data integration with our specialized class on Databricks Sources. This course is designed to demystify the process of connecting to diverse data sources using the robust, cloud-based platform of Azure Databricks. You'll gain practical knowledge on navigating the Azure portal to set up essential components such as resource groups, Databricks instances, key vaults, and data lakes. We'll walk you through the steps to access both public and private data repositories, ensuring you have the skills to leverage Databricks Sources effectively for any data project or challenge you face.
Course Outline ( Free Preview)
What You'll Need to Get Started for Databricks Sources
Module 00 - Class Introduction
This video is an introduction to the Azure Databricks Sources class, which teaches you how to connect to various data sources using Azure Data Bricks. Manuel explains and lists the prerequisites for the class, such as having access to the Azure portal, creating a resource group, a Databricks instance, a key vault, and a data lake.
Module 01 - External Databases (Azure SQL DB)
In this video, Manuel shows you how to connect to an Azure SQL Database from a Databricks notebook using two different connectors: JDBC and SQL Server. He explains the syntax and options for each connector, and how to use variables, queries, and Key Vault to make your connection more secure and flexible. He also demonstrates how to read the schema and display the data from the database table or query in a data frame.
Module 02 - Storage Accounts (ADLS)19 min.
This is a tutorial on how to connect to an Azure Data Lake Storage account from a Databricks notebook using different methods. Manuel explains the advantages and disadvantages of using OAuth 2, SAS token, and access key, and shows the code and configuration needed for each option. He also demonstrates how to read a CSV file from a container and display it as a data frame in the notebook.
Module 03 - Connecting to Snowflake11 min.
In this video, Manuel shows you how to connect to a Snowflake database from a Databricks notebook and query data from a table. He explains the syntax and options for using the spark.read.format function and demonstrates how to create an options object to store the connection parameters. He also shows you how to use a query instead of a table name to filter the data.
Module 04 - Connecting to Web Pages24 min.
In this video, Manuel shows you how to connect to a web page and scrape data from it using Python and a library called Beautiful Soup. He explains the steps of installing the library, requesting the page content, parsing the HTML, finding the relevant tags and elements, and creating a Pandas data frame with the extracted data. He uses a Wikipedia page that lists the largest companies in the United States as an example and demonstrates how to get the table of data from the web page.
Module 05 - Azure Key Vault and Secrets27 min.
In this video, Manuel shows you how to use Azure Key Vault to store and retrieve sensitive information such as passwords, connection strings, and access keys for connecting to different data sources from Databricks notebooks. He explains the steps of creating a secret scope, a secret, and granting access to the Databricks service principal. He also demonstrates how to use the DB utils command to get the secret values and use them in the connection options for Azure SQL Database and Azure Data Lake Storage.
Manuel Quintana is a Senior Consultant, Trainer, and Content Author at Pragmatic Works with more than 12 years of experience helping organizations modernize their data and analytics platforms. He specializes in Microsoft Fabric and works extensively across its end-to-end workloads, including data engineering, data warehousing, data integration, real-time analytics, and business intelligence. Manuel helps organizations design and implement scalable data solutions that transform raw data into actionable insights while enabling self-service analytics and AI-driven decision making.
In addition to his consulting work, Manuel has spent over a decade developing and delivering technical training for thousands of professionals worldwide. His expertise spans the Microsoft data platform, including T-SQL, SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), SQL Server Reporting Services (SSRS), Power BI, Power Apps, Power Automate, Azure data services, and Microsoft Fabric. Known for his practical, hands-on teaching style, Manuel specializes in helping teams adopt modern analytics architectures and get measurable business value from their Microsoft investments.
Manuel is also an accomplished author, having written books and training content focused on Power BI and the Microsoft Power Platform. Beyond client engagements, he actively contributes to the Microsoft data community through conference presentations, user groups, webinars, and technical events, where he frequently shares expertise on Microsoft Fabric, Power BI, modern data architecture, and analytics best practices. When he's not consulting, teaching, or creating content, Manuel enjoys spending time with his wife, Lindsey, and their two children, Parker and Sebastian.
