Job Description: Pentaho Data Integration (PDI) Developer
A results-driven Pentaho Data Integration (PDI/Kettle) Developer with over 5 years of hands-on experience in designing, developing, and deploying end-to-end ETL pipelines across Data Warehouses and Data Lake environments.
Highly proficient in integrating data from diverse source systems such as Oracle, SQL Server, PostgreSQL, MySQL, APIs, and cloud-based sources. Skilled in building scalable ETL workflows, implementing data quality checks, designing dimensional models, and optimizing transformation performance for enterprise data platforms.
Experienced in working closely with business teams, data modelers, architects, and BI teams to translate business requirements into robust technical ETL specifications. Strong expertise in environment migration, supporting ETL deployments across Dev, SIT, UAT, and Production using repositories.
Key Skills & Expertise
- Pentaho Data Integration (PDI/Kettle): Transformation design, job orchestration, workflow branching, parameterization, repository migration, error handling & logging.
- ETL Development: End-to-end data pipeline development covering staging EDW data marts, including incremental loads.
- Data Integration: Experience extracting data from RDBMS, flat files, APIs, cloud sources, and loading into DWH/Data Lake environments.
- Database Skills: Strong SQL experience with Oracle, SQL Server, PostgreSQL, MySQL, Vertica.
- Cloud & Big Data: Exposure to AWS, S3/Data Lake storage, Parquet outputs, and distributed data processing.
- Data Quality & Governance: Data profiling, validations, cleansing transformations, SCD Type 1 & 2 logic, audit & reconciliation frameworks.
- Performance Optimization: Tuning transformations, SQL optimization, lookup caching, memory management, and parallel execution.
Roles & Responsibilities
- Collaborate with business users, BI teams, and data architects to understand reporting and analytical requirements and translate them into ETL specifications.
- Design and develop PDI transformations and jobs to ingest, cleanse, validate, and integrate data from multiple source systems into the enterprise Data Warehouse.
- Implement business rules, data quality checks, error handling, and exception management within ETL PDI workflows.
- Develop and maintain dimensional models (fact tables, dimensions, surrogate keys, SCD logic) to support reporting and dashboards.
- Optimize existing ETL workflows for performance, scalability, reliability, and proper resource utilization.
- Manage scheduling, monitoring, and alerting for daily, weekly, and monthly ETL jobs using Kitchen/Pan and OS schedulers.
- Troubleshoot job failures, conduct root cause analysis, and provide permanent f ixes to ensure uninterrupted data operations.
- Work with DBAs, cloud teams, and infrastructure engineers for performance tuning, environment setup, data modeling, and query optimization.
- Maintain technical documentation including mapping sheets, ETL logic, SDS/SRS-style specifications, runbooks, and deployment guides.
- Support UAT cycles, production releases, and post-deployment validation to ensure high-quality deliverables.
Role Requirements:
- Minimum 3 years and above of experience in Data Integration, Extract, Transform, Load (ETL), and Data Warehousing
- Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field
- Experience in Data Modeling and strong Analytical Skills
- Familiarity with Pentaho Data Integration (PDI) tools is highly preferred
- Ability to design efficient workflows and pipelines for large-scale data processing
- Knowledge of relational databases and open-source tools is an advantage
- Strong problem-solving, critical thinking, and organizational skills
