If you are looking to excel and make a difference, take a closer look at us…
Overview:
The Senior Data Engineer and Developer is responsible for designing, coding, and implementing new business application sources to be extracted, transformed and loaded to the data lake.
Functional Responsibilities:
1. System Development Lifecycle & Engineering End-to-End Delivery
- Accountable for all phases of the development process, including analysis, design,
construction, testing, and implementation.
2. Technical Architecture
- Provides a sound understanding of Big Data application development concepts/principles,
alongside a strong knowledge of concepts and principles in other technology areas.
3. Advanced Problem Solving & Data Analysis
- Root Cause Analysis: Solves and works through complex problems and projects via in-depth evaluation of business processes, system processes, and industry standards performs root cause analyses.
- Data Exploration: Utilizes ad-hoc techniques to perform on-the-fly analysis of data.
- Strategic Evaluation: Provides evaluative judgment based on the analysis of factual information in complicated and unique situations.
4. Quality Assurance & Support
- Post-Implementation Review and Support: Engages in post-implementation analysis of business usage to ensure successful system design and functionality.
- Quality Ownership: Directly impacts the business by ensuring the quality of work provided by self and teammates.
5. Stakeholder Management & Collaboration
- Technical Consulting: Consults with users, clients, and other technology groups on issues and recommends advanced programming solutions.
- Cross-Functional Impact: Impacts own team and closely related work teams. Exhibits sound and comprehensive communication and diplomacy skills to exchange complex information.
6. Leadership & Soft Skills
- Influence & Negotiation: Strong leadership, interpersonal, influence, negotiation, and written/verbal communication skills required.
Technical Responsibilities:
1. Data Engineering & ETL/ELT Pipelines
- End-to-End Pipeline Design: Designing and building complete ETL processes, moving and transforming data for all warehouse layers from source to downstream reporting.
- ETL Tools: Hands-on experience with ETL tools like Informatica and building SSIS packages.
- Real Time Data Ingestion: Must have experience with Real-Time Data Ingestion using modern streaming/analytics technologies (e.g., Kafka, Flink, Hudi, Hive, Clickhouse).
- Operating Systems & Scripting: Hands-on experience with Linux open-source software platforms, alongside Shell Scripting for automation.
2. Database Design & Advanced SQL
- Advanced Querying: Writes advanced SQL, including query tuning and optimization.
- Schema & Architecture: Core expertise in Database Designing for data warehousing.
- Data and Storage Optimization: Implementing data compression techniques for space- saving and storage efficiency.
3. Data Quality and Healthcheck
- Data Governance: Implementing automated data quality checks to ensure data integrity.
- Deployment Management: Managing deployment and debugging processes within a production Data Lake environment during implementation phase.
4. Methodology & Frameworks
- Experience working within Agile and waterfall development model
Experience We Are Looking For:
- Minimum of 5-8 years of experience in Data Engineering ETL, SQL and Big data preferably within the Financial Services industry.
- At least 3-5 years in a leadership or supervisory capacity.
- Proven track record of delivering complex, large-scale data projects (e.g., system migrations, regulatory reporting).
For more job opportunities, please go to HLB Careers:
