Databricks is Fast – Databricks is fast because it’s built on Spark, which is an in-memory solution with Hadoop and this makes it a much faster operation to work with. A key note here is that the Databricks team built Spark and are one of the key contributors to that open source project.
As a result, they’re able to optimize that platform for us. That speed allows us to take advantage of Hadoop at scale and do some great work there.
Databrick is Easy – What makes it easy? First, let’s look at cluster management. Typically, one of the hardest things in a Hadoop environment is setting up the cluster. You can do this in the cloud or on premise, but in either case, you must make decisions about how to build and put together the cluster and no matter which distribution you use or how much help you have, there are issues around that.
With Azure Databricks we can build the cluster automatically and have the power of Azure behind that is basically creating the cluster on the fly as you need it. It also scales up as you need it, and for those doing a lot of testing work in that platform, it will also shut down automatically, giving you cost savings in Azure.
You may ask, is it easy to work with? Absolutely! Azure Databricks supports several typically used languages. Whatever you love to use, like Python, R, Scala or whatever your language of choice, they are all supported natively in the platform.
So, you can create a notebook native in one of those languages, but you can also switch back and forth, relying on the strings you may have from cross-skills within the platform. This makes it incredible easy to use.
They also include things like workspaces that help organize our work. We can put notebooks into workspaces, and we can take advantage of a well-organized solution, making it easier for us overall to use and manage Azure Databricks.
Look at what you want to do in Spark or for Hadoop and remember that Databricks makes this an easy platform for you to use and is surely worth trying out.
Bottom line is Azure Databricks allows you to have a fast and easy solution to work with. As we focus on Databricks this quarter at Pragmatic Works, I’ll continue to post about areas where it may be compelling to use.
If you have questions or want to learn more about Azure Databricks or Azure in general, click the link below or contact us. Our Azure experts are here to answer all your questions and help you in any way we can, no matter where you are on your cloud journey.