Position Name: Data Engineer
Job Summary:
Responsible for the design, development and support the implementation and maintenance of cloud data platform pipelines and data models both globally and across different markets/countries.
Job Description:
- Design, develop, document and implement end-to-end data pipelines and data integration processes, both batch and real-time. This includes data analysis, data profiling, data cleansing, data lineage, data mapping, data transformation, developing ETL / ELT jobs and workflows, and deployment of data solutions.
- Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness, and to optimize and fine-tune ETL / ELT processes.
- Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of ETL / ELT processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards.
- Ensure development adhere to guidelines and governance policies on data and business intelligence platform.
- Collaborate with IT team members, SMEs, vendors and internal business stakeholders, to understand data needs, gather requirements and implement data solutions to deliver business goals.
- BAU support for any data issues and change requests, document all investigations, findings, recommendations and resolutions.
Job Requirement:
- Hands-on experience on Azure Big Data Solution such as Azure Spark pool, Data Factory, Databricks, Gen2, Synapse, Power BI.
- Experience with various of ETL/ELT frameworks, data warehousing concepts, data management framework and data lifecycle processes.
- Experienced in handling and processing different types of data (structured, semi-structured and unstructured).
- Strong knowledge in various database technologies (RDBMS, NoSQL and columnar).
- Understanding of ETL programming languages like Python, Scala, Java, Spark, Shell and PLSQL.
- Ability to communicate and present technical information in a clear and unambiguous manner.
- Strong ability to work independently and cooperate with diverse teams in a multiple stakeholder's environment.
- Strong sense of work ownership, high affinity with anything data and a desire for constant improvements.
- Experience with SAP data sources preferred.
Education
- Bachelor's degree in Computing/Information Technology or equivalent.
Work Experience
- Up to three years experience in similar role.
