T
AWS Databrics ,ADF, Pyspark , Python , Terraform
T
- Posted 8 hours ago
- Be among the first 10 applicants
Job Description
Hiring: AWS Databricks Data Engineer
We are looking for an experienced AWS Databricks Data Engineer with strong expertise in Databricks, Azure Data Factory (ADF), PySpark, Python, Terraform, and AWS Cloud.
If you have experience building scalable cloud-based data pipelines and are looking for your next opportunity, we'd love to connect!
Job Details
- Role: AWS Databricks Data Engineer
- Experience: 6–9 Years
- Locations: Indore / Pune / Hyderabad
- Notice Period: Immediate Joiners / 30 / 60 / 90 Days
- Key Skills: AWS, Databricks, ADF, PySpark, Python, Terraform
Key Responsibilities
- Design, develop, and maintain scalable data engineering solutions using Databricks.
- Develop high-performance data processing pipelines using PySpark and Python.
- Build and manage ETL/ELT workflows using Azure Data Factory (ADF).
- Work with AWS cloud services for data storage, processing, integration, and deployment.
- Design and implement efficient data ingestion and transformation frameworks.
- Develop and maintain cloud infrastructure using Terraform / Infrastructure as Code (IaC).
- Optimize Databricks jobs, Spark workloads, and data pipelines for performance and scalability.
- Implement data quality, monitoring, logging, and error-handling mechanisms.
- Troubleshoot production issues related to data pipelines and distributed processing.
- Collaborate with architects, analysts, DevOps teams, and business stakeholders to deliver reliable data solutions.
- Follow CI/CD, version control, security, and cloud engineering best practices.
Required Skills
- 6–9 years of experience in Data Engineering / Cloud Data Engineering.
- Strong hands-on experience with Databricks.
- Strong development experience in PySpark and Python.
- Hands-on experience with Azure Data Factory (ADF).
- Good experience working with AWS Cloud.
- Hands-on knowledge of Terraform and Infrastructure as Code.
- Strong understanding of ETL/ELT, data pipelines, data transformation, and distributed data processing.
- Experience working with large datasets and optimizing Spark-based workloads.
- Good understanding of SQL, data modeling, and cloud data architectures.
- Strong analytical, troubleshooting, and communication skills.
More Info
Key Skills
ETL/ELT
Infrastructure as Code (IaC)
Azure Data Factory (ADF)
About Company
Skills:
Docker, FastAPI, Datadog, Python, LangChain, Vector Databases, Uvicorn
Skills:
Django, Numpy, Pandas, Flask, Data Structures And Algorithms, Version Control, Python, Testing, Software Development Best Practices, Scikit-learn
Skills:
RDS, Pyspark, AWS Glue, Dynamodb, Emr, Sql, Lambda, Cloudwatch, Sqs, Iam, Sns, Python, AWS, Lake Formation, CloudTrail
Skills:
snowflake , Data Modeling, Automated Testing, Version Control, Sql, Release Management, ELT, Performance Tuning, Python, Etl, Auditing, Data Quality Checks, Orchestration and Scheduling, dbt, CI/CD, Role-based Access, Monitoring
Skills:
Unit Testing, Hadoop, Pyspark, Apache Spark, User Acceptance Testing, Hive, System Testing, Shell scripting, Python Programming, Data historical load and overall Framework concepts, AWS ecosystem, Spark related performance tuning, Git repository, CDC operations, UNIX operating system concepts, Google Cloud BigQuery, Jenkins or equivalent CICD tool, Agile delivery model, Cloudera Hortonworks Data Platform, AWS S3 Filesystem operations