Data Engineer

Job Description: Senior / Lead Azure Data Engineer (6+ Years)

Role Overview

We are looking for a highly skilled Senior Azure Data Engineer with 6 to 10 years of hands-on IT experience. You will design, develop, and maintain robust, scalable, and secure cloud-native data architectures on Microsoft Azure. You will collaborate with data architects, software engineers, and business stakeholders to transform raw data into high-quality analytical systems.

Key Responsibilities

• Data Pipeline Development: Design, build, and optimize complex ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks to ingest structured and unstructured data.
• Data Modeling & Architecture: Implement scalable data warehouse and data lake solutions using Azure Synapse Analytics, ADLS Gen2, and Snowflake (or similar modern storage systems).
• Performance Tuning: Monitor, troubleshoot, and fine-tune Spark jobs, SQL queries, and data pipelines for maximum efficiency and cost optimization.
• Cloud Migration: Lead end-to-end data migration projects from on-premises databases (SQL Server, Oracle) or legacy Hadoop environments to the Azure cloud ecosystem.
• Data Governance & Security: Implement data governance, security best practices, access controls, and data masking using Azure Key Vault, IAM, and Purview.
• CI/CD & DevOps: Set up CI/CD deployment pipelines using Azure DevOps or GitHub Actions for automated deployment of data workflows.
• Leadership & Collaboration: Mentor junior and mid-level data engineers, conduct code reviews, and drive architectural discussions.

Required Technical Skills

• Core Experience: 6+ years in data engineering, database development, or business intelligence.
• Cloud Platform: Deep expertise in Microsoft Azure services: ADF, Azure Databricks, Azure Synapse Analytics, ADLS Gen2, Azure SQL Database, and Cosmos DB.
• Programming Languages: Advanced proficiency in Python or PySpark and robust SQL scripting.
• Big Data / Processing: Apache Spark, Delta Lake architecture, and distributed data processing.
• Data Modeling: Strong experience in dimensional modeling (Star/Snowflake schema, Fact and Dimension table design).
• Version Control & CI/CD: Git, Azure DevOps, or Jenkins.

Preferred / Good-to-Have Skills

• Certifications: Microsoft Certified: Azure Data Engineer Associate (DP-203) or Databricks Certified Data Engineer Professional.
• Additional Platforms: Exposure to Snowflake or ClickHouse.
• Generative AI / Copilot: Familiarity with AI-assisted coding tools or integrating LLM pipelines into data platforms.