In this talk we will show you how to design, build, deploy and manage data pipelines using Apache Airflow running on Microsoft Azure.
Agenda
- First we will go through a some Apache Airflow fundamentals, what it does, how it does and what its best suited for.
- Next we will look at setting up a Airflow local development environment where you will learn how to design, build and test a DAG.
- Then we'll look at deployment options for running a production Airflow instance on Azure (including using Astronomer's Astro service) and managing the deployment lifecycle with CI/CD integration to get fully orchestrated and managed pipelines running reliably in production.
Негізгі бет Orchestrating Data Pipelines with Apache Airflow on Azure
Пікірлер: 3