About Horizontal: Established since 2003 in the US, Horizontal solves complex challenges across two distinct businesses: Horizontal Digital and Horizontal Talent. We are consistently recognized for being a top workplace and one of the fastest growing private companies. Horizontal Talent specializes in staffing for IT, Digital & Creative and Business & Strategy markets. We have global offices in US, UAE, India, Malaysia and Australia.
About The Role
We are seeking an experienced Senior Data Engineer to design, build, and maintain modern, scalable data platforms and pipelines. In this role, you will partner with tech leads, solution architects, and clients to deliver high-performance, reliable, and cost-efficient data solutions that power analytics, business intelligence, and AI initiatives. You will also play a key role in mentoring junior engineers and driving engineering best practices across the team.
Key Responsibilities
- Design, build, and optimize ETL/ELT pipelines using GCP services such as Dataflow, Dataproc, Composer (Airflow), Dataform, and Cloud Functions.
- Develop both batch and streaming pipelines to handle structured, semi-structured, and unstructured data.
- Implement scalable data models in BigQuery to support analytics, BI reporting, and machine learning workloads.
- Apply data governance frameworks, quality checks, and metadata management practices to ensure trusted and compliant data.
- Optimize queries, pipeline performance, and storage costs while ensuring platform scalability, reliability, and fault tolerance.
- Monitor, troubleshoot, and tune pipelines for continuous improvement.
- Lead data migration projects from on-premises systems (e.g., Oracle, MicroStrategy) to modern cloud environments.
- Integrate data from multiple sources, including APIs, databases, and event streams, into unified data platforms.
- Work closely with analysts, BI developers, and business teams to enable self-service analytics and faster decision-making.
- Mentor and coach junior engineers, set coding standards, review designs, and share best practices.
Requirements
- 8+ years of experience in data engineering.
- Minimum 2 years of hands-on experience with Google Cloud Platform (GCP).
- Proven experience building and scaling cloud-native data platforms and pipelines.
- Strong proficiency in SQL and Python.
- Hands-on experience with GCP tools (BigQuery, Dataflow, Dataproc, Composer, Dataform).
- Experience with workflow orchestration tools (Airflow/Composer).
- Familiarity with CI/CD practices for data pipelines.
- Strong understanding of modern data modeling (relational, dimensional, schema-on-read).
Nice to Have
- Exposure to Azure Data Stack (Synapse, Data Factory, Databricks, Logic Apps).
- Knowledge of data governance frameworks, metadata management, and data security best practices.
Soft Skills
- Strong problem-solving and analytical thinking.
- Good communication skills and ability to work with cross-functional teams.
- Ability to balance technical depth with business impact.
- Leadership and mentoring capability.
