JOB SUMMARY
The data engineer is to assist in the development of data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. To work on optimal data delivery architecture that is consistent throughout ongoing projects for the organization.
KEY DUTIES AND RESPONSIBILITIES
- Construct, test and maintain architectures such as databases and large-scale processing systems of data
- Build processes supporting data transformation, data structures, metadata, dependency, and workload management using SQL and Azure Big Data technologies
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
- Create and maintain optimal data pipeline architecture and analytics infrastructure that enables almost every other function within the data scopes
- Identify, design, and implement internal process improvements such as automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability etc.
- Build data lake and data tools for analytics and data science team members that assists them in building and optimizing our solution into an innovative industry leader
- Work with stakeholders to identify the business requirements and expected outcome especially on improving efficiency, productivity and monetization using various data science methodology
- Wok with vendors/consultants on various analytics projects and apply the best practice for internal usage
- Collaborate with different units in the Data Analytics Centre and other departments to support data collection, integration and retention requirements
PERSON SPECIFICATION
Minimum Qualifications
- At least a bachelor's degree in any of the following disciplines but not limited to Mathematics, Computer Science, Statistics, Economics, Actuarial, Data Science
- 2 5 years working experience of relevant database architecting, preparation, design and modelling
- Advanced working SQL knowledge and experience working with relational databases
- Data Warehouse modelling (star schema, data vault)
- Experience in Power BI, SSAS and DAX languages
- Experience with SQL / Python / SAS / Tableau / R / SPSS / Adobe or other Machine Learning programming languages
- Experience with Hadoop / Hybrid / Cloud and other data storage tools
- Involved in the development of data marts, data lakes and databases
- Proficient in Machine Learning and Artificial Intelligence
Knowledge, Skills and Abilities
- Knowledge in Azure Cloud Environment (setup and configuration), Azure Data Factory and DevOps will be added advantage
- Knowledge of database architecture, data modelling, data warehousing, ETL methodologies and computer engineering
- Knowledge of various technological decision making for the business's future data, analysis, and reporting needs
- Knowledge of business analytics understanding data from various systems for deeper comprehension of data structure based on subjects.
- Have involved in the development of stakeholder-ready deliverables and presenting them with confidence and influential
- Preferably to have attended Big Data Analytics short-term courses or have obtained certification in Data Science
- Self-driven with strong analytical and creative in problem-solving ability
- Able to complete multiple tasks within the deadline
- Maintaining and balancing various requests from stakeholders and assess the objectivity of the requests for prioritization
