Real-time Structured Streaming in Azure Databricks

Do you want to learn real-time Structured Streaming in Azure Databricks? In this recent webinar with Principal Consultant, Brian Steele, you’ll learn all about Structured Streaming, the main model for handling streaming datasets in Azure Databricks.

You’ll want to check out this presentation if you want to learn more about Databricks and what it can do for your organization, or if any of these points pertain to you:

• You have a large volumes of data and you’re processing that in a batch format
• You want to get real-time insights from your data
• You have a strong knowledge of your data, but limited knowledge of Azure Databricks or other Spark systems.

When using a batch processing architecture, we extract data out of various source systems and use Azure Data Factory to do batch processing. This process takes hours to run and typically is done once a day, so although we’d get good information upon completion, it could take almost 24 hours to get the information you need.

In Databricks Structured Streaming, you can stream your source information directly into Event Hubs and use Databricks Structure Streaming to get real-time processing, allowing you to bring impactful insights to users in almost real-time.

This webinar delves into why you should use Azure Databricks and the advantages of Structured Streaming. Brian will review each of the three parts that make up Structure Streaming: source parameters, transformations, and output parameters. Other areas covered (and demoed) are:

• Joint operations and stream-static joins
• Stream-stream joins
• Watermark vs time constraint
• Foreach Batch
• Going into production

So, if you’d like to learn about how to handle streaming datasets in Azure Databricks and get near real-time insights, this webinar is for you. You can also view the presenter’s slides by clicking here.

Need help implementing Databricks or any Azure product or service in your organization? Our team of experts is here to help. Our Remote Services offerings allow you to stay on track during these uncertain times. Click the link below or contact us to learn more.