Position: Senior Data Engineer
Location: Bangsar South, Malaysia
Contract Duration: 12 months
Role Purpose
We are looking for a Senior Data Engineer to design, build, and maintain scalable data pipelines, ensuring data quality, reliability, and accessibility for analytics and reporting needs.
Key Responsibilities
Data Pipeline Development
- Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines.
- Develop, implement, and maintain ETL processes to collect, clean, and store data from various sources.
- Optimize and enhance existing data pipelines for better performance, scalability, and reliability.
Data Quality & Operations
- Ensure data quality and integrity through validation, monitoring, and error handling.
- Troubleshoot and resolve data-related issues in a timely manner.
Collaboration & Support
- Partner with data scientists and analysts to provide necessary data sets and tools for advanced analytics and reporting.
- Implement data security and privacy best practices to protect sensitive information.
Continuous Improvement
- Stay updated with the latest trends and technologies in data engineering.
- Contribute to the continuous improvement of data infrastructure.
Technical Focus
- Must have hands-on experience with Databricks for building and optimizing data pipelines and performing data engineering tasks.
Requirements
Education
- Bachelor's degree in Computer Science, Information Technology, or related field (or equivalent work experience).
Experience
- 5+ years of experience in data engineering or a related role.
Technical Skills
- Proficiency in Python, Java, or Scala.
- Strong experience with data warehouse technologies (AWS Redshift, Google BigQuery, Snowflake).
- Hands-on experience with ETL tools and frameworks (Apache Spark, Apache NiFi).
- Experience with Qlik.
- Hands-on experience with Databricks.
- Knowledge of database systems (SQL and NoSQL) and data modeling.
- Familiarity with data integration and orchestration tools (Apache Airflow).
- Experience with version control systems (Git).
Soft Skills
- Understanding of data security and compliance standards.
- Excellent problem-solving and communication skills.
- Ability to work independently and collaboratively in a fast-paced environment.
- Strong attention to detail and commitment to delivering high-quality work.
