Top Apache Airflow frequently asked interview questions.
- What is Apache Airflow?
- What are the features of Apache Airflow?
- How does Apache Airflow acts as Solution?
- What are the basic concepts in Apache Airflow?
- What are some Airflow dependencies?
- What are some of the integrations in Airflow?
- What is Command line in Airflow?
- How do we create a new DAG?
- How can we restart Airflow webserver?
- How can we add logs to Airflow Logs?
What is Apache Airflow?Apache Airflow helps in authoring, scheduling, and monitoring all Data Pipelines. It began in 2014 with the Umbrella at Airbnb and is an open-source that is achieved by DAG(Directed Acyclic Graphs).
What are the features of Apache Airflow?Features of Apache Airflow are:
- Airflow helps us to schedule all of the jobs and its historical status.
- Airflow helps us to view Directed Acyclic Graphs and its relation dependencies.
- Airflow helps in supporting executions by using Web UI and CRUD Operations on DAG.
How does Apache Airflow acts as Solution?Airflow solves problems like:
Failures - Airflow helps in retrying if any failure happens.
Monitoring - Airflow helps in checking if the status fails or success.
Dependency - There are 2 types of dependencies:
Data Dependencies - helps in upstreaming of data.
Execution Dependencies - helps in deploying new changes.
Scalability - helps in centralizing the scheduler.
Deployment - helps in deploying changes easily.
Processing Historic Data - helps in backfilling historical data.
What are the basic concepts in Apache Airflow?
Airflow consists of 4 concepts:
- DAG - acts as a description of the order used for work.
- Operator - acts as a Template for carrying out work.
- Task - acts as a parameterized instance.
- Task Instance - acts as a task which is assigned to a DAG.
What are some Airflow dependencies?Some Dependencies in airflow are as follows: