Search by job, company or skills

Lead Data Engineer

Lead Data Engineer

Randomtrees
8-13 Years
Not Disclosed
Quick Apply
  • Posted a month ago
  • Over 100 applicants have applied

Job Description

Key Responsibilities:

Data Pipeline Development & Optimization

  • Design, develop, and optimize data pipelines using Azure and Databricks.
  • Build and manage scalable ELT/ETL workflows with Databricks, Delta Lake, ADF, and other Azure services.
  • Optimize Spark, SQL, and Python code for performance and scalability.

CI/CD & Environment Management

  • Hands-on work with Databricks Asset Bundles (DABs) for CI/CD and environment management.
  • Continuously adopt and apply new Databricks features to accelerate delivery.
  • Apply software engineering best practices (TDD, CI/CD, modular design, code reviews) to ensure robust and maintainable pipelines.

Team Mentorship & Collaboration

  • Mentor team members to improve technical capability and overall productivity.
  • Deliver high-quality solutions in a fast-paced environment with quick turnaround expectations.

More Info

Job Type:
Industry:
Function:
Employment Type:

Key Skills

About Company

RandomTrees is a global provider of Data and AI services. Led by distinguished Industry experts with a passion to innovate with data. We help reinvent your enterprise with cutting edge Generative AI backed by our niche expertise to handle any type of data. We bring a unique set of data & and AI accelerators that help jumpstart & accelerate AI journey in the enterprise. Our differentiators are backed by the leading enterprise cloud partners and clients span across life sciences, Automotive and other manufacturing Industries Our portfolio comprises of: (1) Cloud data engineering services & Accelerators (2) Gen AI Products, Accelerators & services.

Similar Jobs

7-9 yrs
Chennai, India
Skills:
snowflake , Data Lineage, Metadata Management, Data Modelling, Automated Testing, Informatica, ELT, Data Quality, Git, Databricks, Data Warehousing, Talend, Etl, Airflow, Data Lakes, Luigi, Relational Databases, Deployment Practices, dependency management, Azkaban, Data Lakehouse, NoSQL Databases, Distributed Data Processing
6-10 yrs
Chennai, India
Skills:
snowflake , Data Architecture, Data Modeling, Linux Scripting, Adobe Analytics, Sql, Devops, Data Quality, Powerbi, Azure Data Lake, Databricks, Python, DBT framework, Azure Data Engineering
7-9 yrs
Chennai, India
Skills:
snowflake , Sql, Databricks, Data Warehousing, Cloud, Devops, Jira, Tableau, Power Bi, Etl, Gcp, Git, ELT, Azure Data Factory, Data Visualization, Datadog, AWS, Kubernetes, Python, Servicenow, Azure, Docker, Redshift, Monitoring, Azure Monitor, dbt, Looker, Synapse, CI CD pipelines, Big Query, Ticketing Workflow Tools, Log Analytics, Freshdesk, Azure SQL Server
7-15 yrs
Coimbatore, Chennai, Bengaluru
Skills:
Gcp, Apache Beam, Java, BigQuery, Sql, Data Modeling, Cloud Storage, Dataproc, Kafka, Git, Agile, Python, Spark, Data Quality, Data Governance, Cloud Security, Google Dataflow, ETL/ELT, Batch Processing, Streaming Data Pipelines, Pub/Sub, Cloud Composer, CI/CD, Performance Optimization
8-13 yrs
Chennai
Skills:
Azure, Databricks, Azure Sql, Python, Sql, Spark