Airflow: Decorators for a Clean Data Pipeline

How to abstract away complexity in your data pipeline using Airflow decorators

Imagine the scenario where you have to run multiple daily jobs to extract data from a data lake, preprocess them, and store the cleaned datasets to a dedicated database. It would be extremely tedious if we have to run the pipeline everyday, constantly checking for possible errors. This is where Airflow comes in handy: it provides you with all the tools to build and monitor multiple data pipelines, automatically.