Data Engineer

Key Responsibilities: Design and implement scalable data pipelines using Apache Spark, Delta Lake, and Databricks Workflows. Build and maintain ETL/ELT processes on Databricks Lakehouse Platform to support BI and advanced analytics using Delta Live Tables. Optimize performance of data pipelines and queries using Spark tuning and caching strategies. Implement and maintain CI/CD pipelines for data engineering workflows using tools like Databricks Repos, Git, and Azure DevOps. Secure and govern data using Unity Catalog, data masking, and role-based access controls. Integrate with upstream/downstream systems such as AWS S3, Snowflake, or Power BI. Good Experience on Databricks Asset Bundles. Develop documentation and promote best practices in the use of Databricks notebooks and workflows.

Required Qualifications:Mandatory 5 to 8 years of experience in data engineering, analytics, or platform engineering roles. Proven hands-on experience with Databricks, Spark, Delta Lake. Proficiency in Python and SQL. Strong understanding of distributed computing and cloud data architecture and data warehousing. Familiarity with DevOps practices and tools for CI/CD and infrastructure as code.

Nice-to-have Knowledge of Lakehouse architecture, Unity Catalog, and Databricks SQL. Experience with data governance, metadata management, and security controls. Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional).