Search by job, company or skills

Lead data Engineer

Lead data Engineer

infojini inc
10-12 Years
Not Disclosed
Early Applicant
  • Posted 10 days ago
  • Be among the first 10 applicants

Job Description

Role: Lead Data Engineer

Mode of work : Hybrid – Mandatory

Location: Bangalore

Timing: 3PM to 12AM

Data Engineer Lead

Skill Set:

10+ years in Data Engineering.

5+ years of hands-on experience in Azure Databricks/PySpark.

Experience with Microsoft Fabric.

Strong SQL optimization expertise.

Experience leading teams, architecture discussions, and enterprise data transformation programs.

Exposure to DevOps, governance, security, and cloud cost optimization.

Hands-on coding is mandatory for this role. The Senior Data Engineering Lead must actively contribute to development, write and optimize production-grade code, troubleshoot complex technical issues, and lead by example rather than focusing solely on team management or architecture.

  1. Data Engineering Leadership: Lead the design, development, and delivery of enterprise-scale data engineering solutions, mentoring data engineers and driving technical excellence.
  2. Advanced Databricks Development: Strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, Delta Lake, and Medallion Architecture.
  3. ETL/ELT Design and Implementation: Design, develop, and optimize scalable ETL/ELT pipelines, incremental data ingestion, CDC, and batch/streaming data processing.
  4. SQL Development and Optimization: Expert-level SQL skills, including complex queries, stored procedures, execution plan analysis, indexing, partitioning, and query performance tuning.
  5. Azure Data Platform Expertise: Strong experience with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Databricks, and related Azure data services.
  6. Microsoft Fabric: Good hands-on experience with Microsoft Fabric, including OneLake, Lakehouse, Warehouse, Data Factory, and Spark-based data engineering capabilities.
  7. Data Modeling and Architecture: Expertise in data warehousing, dimensional modeling, star/snowflake schemas, lakehouse architecture, and data quality management.
  8. Performance and Cost Optimization: Identify and resolve data processing bottlenecks, optimize Spark workloads, and improve pipeline performance, reliability, and cloud cost efficiency.
  9. Technical Design and Team Mentoring: Lead technical design discussions, architecture reviews, code reviews, engineering best practices, and knowledge-sharing initiatives.
  10. Enterprise Integration and DevOps: Experience delivering enterprise-scale data integration and analytics solutions, implementing CI/CD, automated testing, monitoring, governance, security, and production support processes.

More Info

Job Type:
Industry:
Employment Type:

Key Skills

SQL optimization

Azure Data Lake Storage Gen2

Medallion Architecture

Delta Lake

Microsoft Fabric

About Company

Similar Jobs

10-12 yrs
Bengaluru, India
Skills:
Data Security, Data Modelling, Sql, Nosql, Distributed Systems, Data Privacy, Data Lake, modern data engineering tools and platforms, data pipelines, Big Data architecture, data-centric applications, large-scale data pipelines, data visualisation techniques, Pyspark programming language, distributed storage platforms, data schemas, distributed processing platforms
7-12 yrs
Bengaluru, India
Skills:
Data Modeling, Redshift, Sql, ELT, Jenkins, Python, AWS, Etl, Airflow, SageMaker, GitHub Actions, dbt, Dagster, CI/CD
8-10 yrs
Bengaluru, India
Skills:
S3, Cloudformation, Redshift, Sql, Spark, Python, AWS, AWS CDK, Lake Formation, Infrastructure as code, Glue, CI/CD, Athena
4-10 yrs
Bengaluru, India
Skills:
snowflake , BigQuery, Scala, Pyspark, Apache Spark, Kafka, Sql, Apache Airflow, Git, Kinesis, Docker, Databricks, Python, CI CD, Prefect
8-10 yrs
Bengaluru, India
Skills:
Testing Frameworks, Python, Data Manipulation, Version Control, Performance Tuning, Workspace Management, Sql, Git, Spark, Databricks, Advanced Apache Spark Programming, Lakehouse Architecture, Role-Based Access Control, Databricks Notebooks, Delta Live Tables, DB Asset Bundles, modular design, CTEs, Job Orchestration, Cluster Configuration, Unity Catalog, CI CD pipelines, Delta Lake, Advanced SQL window functions, MLflow Integration, error handling