What is Data Mesh?

New terms, concepts, and buzzwords creep into our data terminology every day.

While staying on top of industry trends and innovative concepts is essential, so is having the ability to identify when these new terms aren’t necessarily depicting something genuinely new.

Data Mesh Architecture: A Definition

This might be the case with Data Mesh, which has been hailed as a new approach, based on a modern, distributed architecture for analytical data management. The term was first defined by data architect Zhamak Dehghani, a ThoughtWorks consultant.

Dehghani defines Data Mesh as a type of data platform architecture that “embraces the ubiquity of data in the enterprise by leveraging a domain-oriented, self-serve design.” 

This decentralized approach to data enables end users and stakeholders across a business to access and query data where it lives, without having to export it to a data warehouse or data lake first. What this aims to achieve is to avoid the bottlenecks of a centralized data team that acts as the gatekeeper for all insights. Instead, Data Mesh focuses on serving data as a product, so ownership of data and insights is distributed across the organization. 

Thinking of “Data as a Product”

Barr Moses, CEO of Data Observability platform MonteCarlo, explained in a post that unlike traditional monolithic data infrastructures that handle the consumption, storage, transformation, and output of data in one central data lake, a Data Mesh supports distributed, domain-specific data consumers and views “data-as-a-product,” with each domain handling their own data pipelines.

The tissue (mesh? fabric?) connecting these domains and their associated data assets is a universal interoperability layer that applies the same syntax and data standards.

Is Data Mesh the “Next Big Thing?” 

Aviv Noy, Rivery’s CTO, shared his view on the subject of Data Mesh on insideBIGDATA. He explains how the concept of Data Mesh isn’t necessarily new:

“Large data-oriented enterprises have had to figure out how to decentralize and manage access to data across their organizations. However, thanks to the cloud, thousands of smaller companies and startups can access and benefit from enterprise-grade data tools, systems and platforms – and quickly realized that a central BI or data team can become a bottleneck if analysts and engineers across the business can’t access the data they need, when they need it, right away.”

The upside of having a Data Mesh approach means empowering people across the business with access to the data they need. However, Noy explains that “ultimately, Data Mesh isn’t a product. Data is the product.”  Similarly to Barr Moses, he explains that the key to operationalizing this approach is through a platform that enables the creation of this connecting tissue/mesh.

Without a platform that helps connect all the dots and manage the entire operation, the idea of Data Mesh can’t be executed. Similar to how DevOps revolutionized the way teams manage continuous delivery and build lifecycles, a DataOps solutions will be at the core of embracing a Data Mesh or Data Fabric approach across an organization moving forward.