“Azure Databricks is designed in collaboration with Databricks whose founders started the Spark research project at UC Berkeley, which later became Apache Spark,” explains Rohan Kumar, corporate vice president, Azure Data. “Our goal with Azure Databricks is to help customers accelerate innovation and simplify the process of building Big Data & AI solutions by combining the best of Databricks and Azure.” According to Kumar, Microsoft developed Databricks with three main design principles. Firstly, it had to enhance user productivity when creating big data application. Second, it has to let customers scale globally without limits, and thirdly it had to incorporate enterprise compliance and security.
Democratizing AI
The service offers a one-click setup and an interactive workspace while integrating with services like SQL Data Warehouse, Azure AD, and Cosmos DB. The idea is to empower companies to think big with AI, fulfilling Microsoft’s vision to democratize artificial intelligence. To get there, it worked with popular open source Apache Spark project Databricks. The pair worked together over a two-year period, refining and launching beta versions. “The discussions initially started like any other third-party partnership, but then we realized that if we really, really want to make these companies successful with big data and AI, this really has to be integrated with all the other services,” said Databricks CEO Ali Ghodsi to VentureBeat. After importing data from Azure services, customers can output it to applications like Power BI to provide rich visual insights that are easily understandable. In turn, Microsoft expects this to drive innovation and unlock hidden data. You can learn more about Azure Databricks and its pricing on the official site.
