Lead Site Reliability Engineer

About Us

At YTL AI Labs, we build sovereign AI models that perform on par with the world's best—while staying grounded in local needs, values, and context. Our flagship model, Ilmu, is designed to be culturally aware, contextually intelligent, and fluent in Bahasa Melayu, delivering cutting-edge solutions that empower Malaysian businesses with intelligence that truly understands the market and the people they serve.

As pioneers of sovereign AI, we believe every nation should have the power to shape its own intelligence—guided by its people, priorities, and principles.

Role Summary

The Lead Site Reliability Engineer leads the reliability, platform security, scalability, and operational integrity of key services on the YTL AI Labs platform. Working within the Platform & Infrastructure function, the role leads the delivery of the systems, tooling, and practices that allow product and solutions teams to run AI workloads with confidence.

This is a hands-on engineering role with technical leadership responsibilities. The Lead SRE puts platform reliability standards into practice across a set of critical services, leads incident response within those areas, and leads a small group of engineers spanning site reliability and platform security in delivering the improvements that keep production healthy and secure. The role is also the engineering owner of the platform security workstream. It applies the platform-wide standards set by the Principal SRE, and provides day-to-day technical direction to the engineers in the group. The role operates at the boundary between infrastructure, software engineering, and product delivery, and works closely with engineers and engineering leads across the organisation.

What you'll do

• Lead a small group of engineers across site reliability and platform security, setting priorities, providing day-to-day technical direction, and mentoring mid-level and senior engineers on reliability, security, and operational discipline
• Lead incident response for services within scope, acting as a point of escalation for the on-call team, facilitating blameless post-incident reviews, and helping shape a sustainable on-call rotation
• Lead the platform security workstream, owning platform controls including vulnerability management and patching, container and Kubernetes hardening, network policy, and security monitoring, while the Security team owns policy, risk assessment, and audit
• Implement and refine identity, access, and secrets management patterns that are secure by default, and coordinate the response to platform security incidents
• Own the reliability of a defined set of critical services, implementing and maintaining SLOs, error budgets, and alerting, and seeing reliability risks through before they manifest as customer-facing incidents
• Drive improvements to observability infrastructure across metrics, logging, distributed tracing, and synthetic monitoring, and help others use it effectively
• Design and deliver components of multi-tenant, sovereign-capable deployment topologies, ensuring they meet applicable regulatory and data-handling obligations
• Implement and improve infrastructure-as-code, CI/CD pipelines, and deployment patterns, with automated, auditable delivery and rollback capability
• Build, and guide the development of, tooling and automation that reduce mean time to detection and mean time to recovery
• Contribute capacity, utilisation, and cost data to platform capacity planning, and provide input to the infrastructure roadmap owned by the Principal SRE
• Run reliability reviews for significant platform changes, partner with product and solutions teams during design, and advocate for reliability as a first-class engineering concern

What we're looking for

• Seven or more years in software engineering, infrastructure engineering, or site reliability, including at least two years in a senior SRE or technical lead capacity operating production systems at meaningful scale
• Experience leading a small group of engineers, including setting priorities, mentoring, reviewing work, and providing day-to-day technical direction
• A proven track record of improving the reliability and operability of distributed systems, including implementing SLOs, building observability, and leading incident response
• Hands-on experience with platform security practices, such as cloud and Kubernetes hardening, vulnerability management, identity and access controls, secrets management, and security monitoring
• Strong working knowledge of cloud infrastructure, Kubernetes, container orchestration, networking, identity, and security fundamentals
• Proficiency in Python, Go, or equivalent, and experience building internal tooling and automation
• Practical understanding of modern AI platform operations, including inference infrastructure and LLM reliability and cost challenges
• Ability to communicate operational risk clearly to engineering and non-engineering audiences, and to lead technical discussions within and across teams
• A track record of driving reliability issues to root cause and delivering lasting fixes rather than repeated workarounds

Nice to have

• Experience operating sovereign or on-premises AI infrastructure for government or regulated industry clients in Malaysia or the wider region.
• Familiarity with data residency, network segmentation, and compliance controls relevant to public sector deployments.
• Experience with ML infrastructure tooling including model registries, inference servers (TensorRT-LLM, vLLM, or equivalent), and GPU cluster management.
• Exposure to multi-tenant SaaS platform operations, including per-tenant isolation and quota management.
• Contributions to open-source reliability or infrastructure tooling.
• Security certifications such as CKS, a cloud provider security specialty, or equivalent.
• A degree in computer science, software engineering, or a related discipline, or equivalent practical experience.