snowflake , Pyspark, Lambda, Kinesis, Distributed Systems, Agile, Python, RDS, Sql, Sdlc, Git, Iam, Unix Shell, Spark, Data Integration, Etl, Airflow, CI CD, data ingestion, Step Functions, Dms, cloud optimization, MWAA, large-scale datasets, Glue