Data Chess Game: Databricks, MongoDB, and Snowflake Move Business Part 2
To further reinforce our commitment to providing cutting-edge data technology coverage, VentureBeat is delighted to welcome Andrew Brust and Tony Baer as regular contributors. Monitor their posts in the data pipeline.
This is the second in a two-part series. Lily part 1 dissecting how Databricks and Snowflake approach head-to-head competition.
As we noted yesterday, June was quite a month by post-lockdown standards, as back-to-back, MongoDB, Snowflake, and Databricks each held their annual events in quick succession. Historically, each of these providers may have crossed paths in the same businesses, but usually with different constituencies. So they didn’t face each other directly.
Despite recent declines in financial markets, each of these companies is considered one of the fastest growing growth players in cloud data platforms, with valuations (private or market) in the tens of billions of dollars. dollars. While Databricks is still private, MongoDB and Snowflake have their IPOs well behind them.
They each position themselves as default destination platforms for the company. Databricks and Snowflake are on each other’s competitive radars at this point and yesterday we gave our thoughts on the chess game they play. In this article, we take a look at what each player needs to do to attract the whole company. While there are differences in target markets, especially with MongoDB, there is a common thread for all three: to grow further, they are going to have to expand beyond their comfort zones.
So what are these comfort zones? Databricks and Snowflake came from different parts of the analytics world, while MongoDB focused on operational use cases. Historically, they each addressed different audiences. Databricks for data engineers and data scientists, Snowflake for business and data analysts, and MongoDB for application developers.
But recent tri-vendor initiatives are starting to break down those silos. Let’s start with the deployment. Of the three, MongoDB is the only one with an on-premises presence (the other two are pure cloud players), but barely five years after the start of its cloud database service Atlas, the company’s revenues are now primarily cloud-based. While MongoDB will probably never be a pure cloud game, the cloud is clearly driving its future.
Next come operations. With Snowflake adding a lightweight transaction processing engine and MongoDB taking early steps to start addressing analytics beyond visualization, we were asked a few weeks ago if they were on a collision course . Our opinion ? In the short term, they’re still in separate universes, but in the long term, never say never.
As for analytics, we noted yesterday that Databricks and Snowflake are talking more about developing each other.
Nonetheless, while MongoDB remains the most loquacious in sticking to its knitting as an operational database, below the surface it’s taking the first steps to coming to terms with the relational database folks and dipping its toes in the analysis.
Let’s take a look at the messages from each of the summits over the past month. MongoDB was about to double the number of developers. In CTO Mark Porter’s keynote, he spoke about the growing volume of new apps coming out over the next few years and, with it, the need for rapid approaches that allow developers to overcome barriers to implementation. application production. At Snowflake, it was about strengthening the “data cloud” as a destination by expanding its reach, both in transaction processing and machine learning. And for Databricks, it was benchmarks, governance capabilities and lineage showing the data lakehouse is ready for prime time and capitalizing on its open source strategy.
The starting points of each player relativize their ambitions. MongoDB’s official mission is to enable businesses to operate like “software companies”. This reflects the fact that the MongoDB constituency has traditionally been made up of software developers, and they must be able to be productive if their organizations are to operate at the speed of a software business. A recurring message from this strategy is that traditional databases have proven to be obstacles, due to the rigid nature of the relational schema and the inability to scale them.
For Snowflake, it’s about targeting business and data analysts who rely on data warehouses with a cloud-native reinvention tackling the hurdles of ease-of-use, scaling, and of data sharing.
And for Databricks, it’s about harnessing the breadth and scale of the data lake with a nut-soup development and runtime environment powered by Apache Spark, Photon, and Delta Lake.
The next steps
This is where getting out of the comfort zone becomes critical. Let’s look at each provider individually.
For MongoDB, it’s not just the application developers, but the database folks as well, as we pointed out in our article last month. For MongoDB to become the default operational data platform for new applications, it must go beyond being a development company to too become a data company.
MongoDB has taken some first steps in this direction, such as improving its security game and writing a genuine SQL query engine. The business needs to make deeper cultural shifts, like moving away from the denigrating SQL message and outdated database practices. MongoDB responds that relational database developers should also pivot, or at least accept that the document model does not mean moving away from the skills and disciplines they have developed. The MongoDB platform supports schema validation. But schema tends to be variable in most MongoDB implementations, so we’d like to see more focused efforts in the future to develop data lineage capabilities that could keep up with schema evolution.
Either way, our message to MongoDB remains: don’t alienate a key group (SQL database developers) that you will need to expand your company’s footprint. We would like to see more positive contacts in the future.
For Snowflake, it’s about convincing data scientists that Snowpark should be an efficient execution environment for their models. The company has a new partnership with Anaconda, which maintains Python libraries, to optimize them for running in Snowpark. But the skeptics remain; for example, H2O.ai claims it’s more efficient to bite the bullet and run machine learning models in their clusters that can handle multiple processes, then send the results back to Snowflake.
Since the introduction of Snowpark a few years ago, Snowflake has improved its ability to optimally scale resources for user-defined functions (UDFs) written in languages such as Java or Python.
Of course, the recent Unistore announcement puts operational analytics in Snowflake’s sights. However, we don’t see this as a massive land grab for a new constituency, as the company isn’t taking on the world’s SQL, Oracles, or MongoDB servers.
For Databricks, it’s about making the data lakehouse more user-friendly for businesses and database analysts. These people work with data modeling and BI tools, not laptops; there must be another entry path providing a view that makes Delta Lake look like a data warehouse.
And business analysts expect consistent performance for interactive queries and batch reports. TPC-DS benchmarks are designed around analysis/decision support workloads, but like EPA’s gas mileage ratings, your results will vary. Significantly, the next step for Photon is to reduce latencies under more typical query conditions, as well as expand table and file format support beyond Delta Lake and Parquet, respectively.
Bringing it all together for a winning strategy
The common thread is that, coming from different starting points, each provider must connect to new constituencies. The key will not just be the technology, but the culture and structuring of the core business. Go-to-market, field and support teams need to be recruited who can talk to different stakeholders. Debates about purity must go.
Can MongoDB speak to relational database people as well as developers? Will Snowflake speak the language of data scientists, and can Databricks cultivate the BI community? These are not talking points you’ll see on a press release.
VentureBeat’s mission is to be a digital public square for technical decision makers to learn about transformative enterprise technology and conduct transactions. Learn more about membership.