Streaming Data Mesh: A Model for Optimizing Real-Time Data Services

Data meshes can help organizations decentralize data and improve scalability and maintainability. This book provides an overview of data mesh patterns for streaming and real-time data services, covering topics such as defining a data domain, building data products, and implementing data governance.

Format: Paperback / softback
Length: 200 pages
Publication date: 25 May 2023
Publisher: O'Reilly Media

As data volumes continue to grow at an unprecedented rate, the challenges associated with managing and maintaining data lakes and warehouses have become increasingly complex and costly. Traditional approaches to data management are no longer sufficient to meet the demands of modern businesses, which require faster and more efficient ways to analyze and extract valuable insights from their data.

One solution to these challenges is the adoption of data meshes, a distributed architecture that enables organizations to decentralize their data and distribute ownership of it back to the engineers who produced it. Data meshes provide a flexible and scalable way to manage and process data, allowing for real-time processing and analysis and enabling businesses to make data-driven decisions more quickly and effectively.

In this book, we will provide a comprehensive overview of data mesh patterns for streaming and real-time data services. We will examine the vast differences between streaming and batch data meshes and explore how they can be implemented to meet the specific needs of your organization. We will cover topics such as defining a data domain, building a good data product, and implementing data governance to ensure that your data mesh works seamlessly.

By the end of this book, you will have the skills and knowledge necessary to design a streaming data mesh using Kafka, identify a domain, build your first data product using self-service tools, apply data governance to the data products you create, and implement self-services that support decentralized data management.

Whether you are a data engineer, architect, data product owner, or someone in DevOps or MLOps roles, this book will provide you with the tools and strategies you need to succeed in the rapidly evolving world of data management. So, let's get started and explore the power of data meshes!

Dimension: 232 x 178 (mm)
ISBN-13: 9781098130725