Data Engineer

Job Role :

Role Overview:

We are seeking a senior Data Engineer with extensive hands-on experience building enterprise data platforms on Databricks, alongside proven experience implementing Generative AI solutions using Databricks Genie. In this role, you will architect, scale, and maintain end-to-end data pipelines, enforce enterprise data governance using Unity Catalog, and deliver production-grade GenAI-powered data workflows for business stakeholders.

Key Responsibilities

Data Engineering & Architecture

• Architect, build, and optimize complex, scalable ETL/ELT data pipelines using PySpark, Databricks Workflows, Delta Lake, and Delta Live Tables (DLT).
• Implement robust data modeling, warehousing, and lakehouse patterns to support both analytical and operational use cases.
• Optimize Databricks clusters, query performance, and computational costs across high-volume enterprise datasets.

GenAI & Enterprise Delivery

• Design, implement, and deploy production-grade Databricks Genie enterprise data applications.
• Enable natural-language query interfaces over enterprise data lakes, integrating Genie with existing data assets, metadata, and business logic.
• Ensure seamless delivery of GenAI workloads, from prototype to production deployment, adhering to enterprise SLA, security, and accuracy standards.

Data Governance & Security

• Implement and manage fine-grained data access controls, row/column-level security, and audit policies using Unity Catalog.
• Maintain centralized data discovery, lineage tracking, and metadata management across multi-workspace environments.

Core Requirements

Experience & Education

• Data Engineering: 7+ years of experience in data engineering, data platform development, and big data architecture.
• Databricks Expertise: Strong proficiency in Delta Lake, PySpark/Scala, Delta Live Tables, Databricks Workflows, and performance tuning.
• GenAI Experience: 1–2 years of hands-on experience deploying Generative AI solutions, specifically delivering enterprise projects using Databricks Genie.
• Governance: Deep experience implementing enterprise-grade data security and governance policies via Unity Catalog.
• Languages & Tools: Proficiency in Python, SQL, Spark SQL, Git, and CI/CD deployment pipelines for data platforms.