Data mesh architecture is a decentralized approach to data management that enables domain teams to perform cross-domain data analysis on their own. It was first defined by Zhamak Dehghani in 2019 and is based on four fundamental principles that bundle well-known concepts:
-
Domain ownership principle: This principle mandates the domain teams to take responsibility for their data. Analytical data should be composed around domains, similar to the team boundaries aligning with the system’s bounded context. Following the domain-driven distributed architecture, analytical and operational data ownership is moved to the domain teams, away from the central data team.
-
Data as a product principle: This principle projects a product thinking philosophy onto analytical data. This principle means that there are consumers for the data beyond the domain. The domain team is responsible for satisfying the needs of other domains by providing high-quality data. Basically, domain data should be treated as any other public API.
-
Self-serve data infrastructure platform: The idea behind this principle is to adopt platform thinking to data infrastructure. A dedicated data platform team provides domain-agnostic functionality, tools, and systems to build, execute, and maintain interoperable data products for all domains. With its platform, the data platform team enables domain teams to seamlessly consume and create data products.
-
Federated governance principle: This principle achieves interoperability of all data products through standardization, which is promoted through the whole data mesh by the governance group. The main goal of federated governance is to ensure that data is discoverable, accessible, and understandable across the organization.
Data mesh architecture is different from traditional data architectures in that it allows greater autonomy and flexibility for data owners, facilitating greater data experimentation and innovation while lessening the burden on data teams to field the needs of every data consumer through a single pipeline. It also promotes stronger governance practices as it helps enforce data standards for domain-agnostic data and access controls for sensitive data. The benefits of data mesh architecture include democratic data processing, data democratization, and improved data access, security, and scalability.
