This is Marc Lamberti's webinar about sensors: kzitem.info/news/bejne/mYBmzmGMpZWQZ2k (you mentioned at 11:30)
@dataecstasy
11 ай бұрын
what would be the github integration look like for if companies want to keep the dag code on github
@Astronomer
9 ай бұрын
There's a few different approaches, check out this link that goes through various ways you can set up CICD to use github to store your DAG code! docs.astronomer.io/astro/ci-cd-templates/github-actions
@jugsma6676
2 жыл бұрын
If possible, Could you share with us the repo (github) link of Dag's
@waterhill
10 ай бұрын
Can Airflow be used to orchestrate a spark streaming YARN job that pulls data from Kafka and writes to HDFS?.. the idea is if the spark streaming job queues and it can be monitored/alerted/detected and restarted automatically by Airflow?
@Astronomer
10 ай бұрын
Oh definitely they can! Check out this link for the different options you have for managing Spark via Airflow, you'll probably want to use a Spark hook registry.astronomer.io/providers/apache-airflow-providers-apache-spark/versions/4.1.5
Пікірлер: 8