Azure Data Engineer

Key Responsibilities

Design and develop robust ELT pipelines using Azure Data Factory

Build and optimize data workflows and transformations in Azure Databricks using PySpark

Write efficient Python scripts for data manipulation automation and integration tasks

Collaborate with data scientists analysts and business stakeholders to understand data requirements

Ensure data quality integrity and security across all data processes

Monitor and troubleshoot data pipelines and workflows

Implement best practices for data engineering and cloud architecture

Required Skills Qualifications

5-7 years of experience in data engineering or related roles

Strong proficiency in Azure Data Factory and Azure Databricks

Solid experience with Python scripting for data processing and automation

Handson experience with PySpark SQL and Delta Lake

Familiarity with Azure Synapse Azure Blob Storage and Azure Key Vault is a plus

Understanding of CICD pipelines and version control eg Git

Excellent problem solving and communication skills