How to build AWS Glue ETL with Python shell | Data pipeline | Read data from S3 and load Redshift

In this video, we will develop AWS Glue ETL script using Python shell. We can now use Python scripts in AWS Glue to run small to medium-sized ETL (extract, transform, and load) workflow. Previously, AWS Glue jobs were limited to Apache Spark environment.
Python shell jobs in AWS Glue support scripts that are compatible with Python 2 and 3 and come pre-loaded with libraries such as the Boto3, Numpy, SciPy, pandas, and others. We can also, install other libraries via .whl file.

Link to medium article on the topic: blog.devgenius.io/develop-aws-glue-etl-pipeline-wi…

Link to code, GitHub: github.com/hnawaz007/pythondataanalysis/tree/main/…

Subscribe to our channel:
youtube.com/c/HaqNawaz

---------------------------------------------
Follow me on social media!

GitHub: github.com/hnawaz007
Instagram: www.instagram.com/bi_insights_inc
LinkedIn: www.linkedin.com/in/haq-nawaz/

---------------------------------------------

#Python #ETL #AWS

Topics covered in this video:
0:00 - Introduction ETL with Python shell
0:53 - Pre-Requisites
1:30 - Create Python .whl file
2:35 - Python ETL script
4:15 - Upload scripts to AWS
5:11 - AWS Glue ETL Job
6:33 - AWS Redshift table
6:49 - Execute Glue ETL Job
7:17 - Review Data & logs