Data Engineer / Data Lake
Responsibilities:
· Develop Extract/ Load /Transform (ELT) solution using Talend Data Studio for data ingestion / integration and DQ rules for cleansing and other data processing needs
· Development experience in Talend programming (Java) and Talend Components
· Experience in Hadoop file format development (Parquet, AVRO, and ORC) and Hive/Impala ingestion.
· Experience in handling Sensitive and PII data
· Integrate data for batch, real-time and near real time processes
· Perform data analysis, quality, and cleansing activities
· Good knowledge of Cloudera Platform, Big Data, HDFS, Hive, Impala, Kafka
· Perform unit testing, integration testing and provide support to users in UAT phase
- BAU Support: Monitor ELT job operation, recovery/troubleshooting, optimization, and performance tuning
- Prepare ELT technical specification documentation
- Troubleshoot to handle Talend & Cloudera/Hadoop technical issues
- Experience in Talend Administration Console and Cloudera Manager
- Should have worked on SCD types (Slow changing dimensions), Change Data Capture (CDC) and Operational Data Source (ODS).
· At least 3-4 years of Development & implementation experience of Talend on Cloudera Data Platform
· At least 3 years experience in full system development life cycle
· At least 1 year database programming experience on Hive/Impala
· Should be an independent contributor and willing to help team members when required.
· Have working experience in Cloudera Manager / Hue / Python / Unix is an added advantage
· Good written and interpersonal communication skills
· Ability to work in team
· Familiar with project management methodologies like waterfall and Agile and ability to work on critical delivery timelines.
· Possess outstanding professionalism with pleasant attitude and personality
· Demonstrate analytical and problem-solving skills
· Relevant university degree
JOB TYPE
Industry
Function
Roles
Skills
Education
Kamlax Global Technologies is a leading IT services & business solution provider delivering cutting edge technology solutions to enterprises across the world. we add value to organizations through a synergy of skills, technology insight, innovation, products and services that orchestrate our customer's business to perfection. Our expertise bridges the gap between the businesses and IT and offers detailed, process-driven solutions enabling our customers to enhance productivity and achieve better ROI. Our endeavor is to ensure customer satisfaction and to add value to their overall business objectives. Recognizing the importance of quality in meeting client needs plus hiring and supporting the best team possible to assure we achieve these quality objectives. Quality and good customer solutions via good staffing is our primary business mission.
Similar Jobs
People Also Considered
Career Advice to Find Better
-
Job Search Strategy
-
Interview Tips
-
Resume & Cover Letters
-
Salary Negotiation