Job Summary
We are seeking a Lead / Senior Data Engineer to design, build, and optimize our core data infrastructure. In this role, you will bridge the gap between raw data and production-ready environments, ensuring our analytics systems are scalable, reliable, and high-performing. You will collaborate closely with Data Scientists to support advanced modeling and lead data engineering initiatives.
Key Responsibilities
- Data Pipelines: Build and orchestrate automated ETL/ELT pipelines for batch and real-time streaming data.
- Architecture: Design and maintain scalable data lake and data warehouse structures.
- Data Modeling Support: Partner with Data Scientists to optimize data pipelines, feature stores, and datasets required for statistical modeling.
- Performance: Tune and optimize complex SQL queries and data workflows for large-scale datasets.
- Leadership: Establish technical best practices, ensure data quality, and mentor junior engineers.
Requirements
- Experience: Minimum 4+ years in Data Engineering, Big Data, or Data Architecture.
- Education: Degree in Computer Science, Software Engineering, or a related quantitative field.
- Technical Proficiency:
- Languages & Core Libraries: Mastery of Python (including advanced Pandas and NumPy for data manipulation) and expert-level SQL.
- Data Modeling & ML Support: Strong familiarity with preparing data for statistical modeling and predictive frameworks (e.g., XGBoost, Regression, Decision Trees).
- Infrastructure & Orchestration: Experience with modern data stack tools (e.g., Spark, Airflow, dbt) and cloud data platforms (AWS, GCP, or Azure).
Benefits & Work Environment
- Flexible Hybrid Arrangement: Balance of remote work autonomy and in-office collaboration.
- Professional Growth: Lead high-impact architectural decisions alongside a talented AI and analytics team.
- Competitive Compensation: Monthly salary up to RM 15,000, commensurate with experience and technical assessment.
