Data Pipelines with Apache Airflow

Data Pipelines with Apache Airflow
PDF, ePUB

Book Description
Data Pipelines with Apache Airflow teaches you the ins-and-outs of the Directed Acyclic Graphs (DAGs) that power Airflow, and how to write your own DAGs to meet the needs of your projects. With complete coverage of both foundational and lesser-known features, when you’re done you’ll be set to start using Airflow for seamless data pipeline development and management.

Pipelines can be challenging to manage, especially when your data has to flow through a collection of application components, servers, and cloud services. Airflow lets you schedule, restart, and backfill pipelines, and its easy-to-use UI and workflows with Python scripting has users praising its incredible flexibility. Data Pipelines with Apache Airflow takes you through best practices for creating pipelines for multiple tasks, including data lakes, cloud deployments, and data science.

Data Pipelines with Apache Airflow teaches you the ins-and-outs of the Directed Acyclic Graphs (DAGs) that power Airflow, and how to write your own DAGs to meet the needs of your projects. With complete coverage of both foundational and lesser-known features, when you’re done you’ll be set to start using Airflow for seamless data pipeline development and management.

Content

PART 1 - GETTING STARTED
1. Meet Apache Airflow
2. Anatomy of an Airflow DAG
3. Scheduling in Airflow
4. Templating tasks using the Airflow context
5. Defining dependencies between tasks

PART 2 - BEYOND THE BASICS
6. Triggering workflows
7. Communicating with external systems
8. Building custom components
9. Testing
10. Running tasks in containers

PART 3 - AIRFLOW IN PRACTICE
11. Best practices
12. Operating Airflow in production
13. Securing Airflow
14. Project: Finding the fastest way to get around NYC

PART 4 - IN THE CLOUDS
15. Airflow in the clouds
16. Airflow on AWS
17. Airflow on Azure
18. Airflow in GCP

Download Data Pipelines with Apache Airflow PDF or ePUB format free


Free sample

Download in .PDF format



Download in .ePUB format


Add comments
Прокомментировать
Введите код с картинки:*
Кликните на изображение чтобы обновить код, если он неразборчив
Copyright © 2019