Data Pipelines Pocket Reference: Moving and Processing Data for Analytics

Data pipelines are essential for data analytics, moving data from diverse sources and transforming it to provide context. This pocket reference defines data pipelines and explains how they work in today's modern data stack, covering batch versus streaming data ingestion, build versus buy, and common considerations for pipeline maintenance, testing, and alerting.

Format: Paperback / softback
Length: 200 pages
Publication date: 23 February 2021
Publisher: O'Reilly Media, Inc, USA

Data pipelines are the cornerstone of success in data analytics, as they enable the movement of data from diverse sources and its transformation into valuable insights. The key difference between having data and extracting meaningful value lies in the ability to transform raw data into contextual information. This concise pocket reference provides a comprehensive definition of data pipelines and elucidates their role within the modern data stack.

Within the realm of data pipelines, there are several key considerations and decision points that data professionals must navigate. One crucial aspect is the choice between batch versus streaming data ingestion, which determines the rate at which data is processed. Another important consideration is whether to build or buy pipeline infrastructure, considering factors such as cost, scalability, and flexibility.

This book delves into the most common decisions made by data professionals, covering foundational concepts that apply to open-source frameworks, commercial products, and homegrown solutions. It offers a comprehensive guide to building data pipelines, covering topics such as data ingestion, processing, storage, and visualization. Readers will gain a deep understanding of the tools and products used by data engineers to construct pipelines, enabling them to support analytics and reporting requirements effectively.

Furthermore, the book addresses the critical aspects of pipeline maintenance, testing, and alerting, ensuring that data pipelines remain robust and reliable throughout their lifecycle. By leveraging the power of data pipelines, organizations can unlock the full potential of their data and make informed decisions based on accurate and timely insights. Whether you are a data analyst, engineer, or business leader, this book provides valuable insights and practical guidance for building and maintaining efficient data pipelines.

Weight: 226g
Dimension: 177 x 107 x 19 (mm)
ISBN-13: 9781492087830