Delta Lake: Up and Running: Modern Data Lakehouse Architectures with Delta Lake

Delta Lake is an open-source lakehouse framework that provides a robust way to manage and analyze big data. This book teaches data engineers, data scientists, and data analysts how to use Delta Lake and its features to build data pipelines and applications, ensuring high-quality data and efficient insights.

Format: Paperback / softback
Length: 250 pages
Publication date: 31 October 2023
Publisher: O'Reilly Media

With the rise of big data and artificial intelligence (AI), organizations have the opportunity to rapidly generate data products. However, the success of their analytics and machine learning models relies heavily on the quality of the data they rely on. Delta Lake, an open-source format, provides a robust lakehouse framework that surpasses platforms like Amazon S3, Azure Data Lake Storage (ADLS), and Google Cloud Storage (GCS). This practical book aims to guide data engineers, data scientists, and data analysts in getting Delta Lake and its features up and running efficiently.

The ultimate objective of building data pipelines and applications is to extract valuable insights from the vast amounts of data available. This book will help you understand how your choice of storage solution impacts the robustness and performance of your data pipeline, from raw data to actionable insights. You will learn how to:

Utilize modern data management and engineering techniques to optimize data processing and storage.

Gain a deep understanding of how ACID transactions ensure reliability and consistency in data lakes, even at scale.

Execute streaming and batch jobs concurrently against your data lake, maximizing resource utilization.

Perform update, delete, and merge operations on your data lake with ease.

Employ time travel capabilities to roll back and examine previous data versions, aiding in data analysis and troubleshooting.

Build a streaming data quality pipeline that follows the medallion architecture, ensuring data integrity and accuracy throughout the pipeline.

By leveraging Delta Lake's open-source format and its powerful features, organizations can unlock the full potential of their data and drive innovation in their industries. Whether you are a data engineer, data scientist, or data analyst, this book will provide you with the knowledge and tools you need to succeed in the era of big data and AI.

Dimension: 233 x 178 (mm)
ISBN-13: 9781098139728