Pycon APAC 2019 - Yohei Onishi - Building Analytics Workflow using Airflow and Spark

Yohei had built and operates a data analytics system for global retail logistics operations using Airflow and Spark since the end of last year. In this session, He will talk about how you can build a scalable analytics workflow system based on Airflow (Python) and write extensible job using Python. GCP has provided fully managed Airflow service called Cloud Composer. So he will explain how you can easily build Airflow cluster compared to building your own Airflow cluster on the on-premise server or AWS EC2.

Yohei Onishi is a Data Engineer who works for a Japanese retail company. He is currently working on an analytics data pipeline using Airflow, Spark, and Hive