When a database fails at 3am, or a SaaS provider we depend on goes dark, somebody has to already know what happens next. That's not a document sitting in a drive nobody's opened since the last audit. It's a tested, current, engineering-grade plan — and a person who can run the room when it matters.
We're not hiring someone to keep a DR binder up to date. We're hiring someone to own reliability as an engineering discipline: designing the failure tests, running the incidents, and turning every near-miss into a system that's harder to break next time.
Why This Matters
Trading for Anyone, Anywhere, Anytime means our platform doesn't get to have a bad day. Millions of traders, real money, live markets, around the clock — an outage isn't an inconvenience, it's a trader who couldn't close a position. Regional outages, database failures, and SaaS disruptions aren't hypotheticals; they're a matter of when, not if.
Most companies treat disaster recovery as a compliance exercise: a runbook written once, tested rarely, and trusted blindly. We're building the opposite — DR and incident management as connected disciplines that make the platform measurably more resilient, with real failover tests, real RTO/RPO targets tied to business risk, and AI tooling that helps us model failures and correlate signals faster than a human staring at five dashboards at once.
Why Deriv
- We run live DR failover tests, not tabletop exercises that stay theoretical
- Our incident command spans engineering, infrastructure, SRE, Compliance, and Risk — in real time, not in a post-mortem doc three weeks later
- We're building toward AI-assisted failure modelling, incident signal correlation, and automated runbooks — not replacing judgment, but giving you better information faster when every minute matters
We share what we learn. Deriv is where we write about what we're building, what breaks, and what we figure out the hard way.
What You'll Do
Own disaster recovery as an engineering discipline
- Design, schedule, and run DR exercises for business-critical applications — database failures, regional outages, SaaS provider disruptions, live failover tests
- Challenge business impact analyses directly with system and SaaS owners until RTO/RPO targets reflect actual business risk, not last year's assumptions
- Keep DR strategies, runbooks, and readiness dashboards matched to what's actually running in production — not what the architecture diagram from two years ago says
Command incidents when it counts
- Lead critical and major incidents from declaration to resolution, coordinating engineering, infrastructure, SRE, Compliance, and Risk in real time
- Make the fast calls under pressure: assess blast radius, set priority, pull in the right responders, and be the single source of truth while everyone else is asking what's happening
- Brief the right level of detail to the right audience — on-call engineers need different information than the C-suite, and giving everyone the same update either drowns engineers in noise or leaves leadership guessing
Turn incidents into architecture, not just tickets
- Lead post-incident reviews that find the systemic weakness, not just the symptom
- Hold teams accountable for closing findings from DR exercises, incidents, and audits — a finding that never closes is just a risk you've agreed to keep taking
- Challenge new systems before they reach production if they don't meet recovery requirements
Use AI as a working tool, not a buzzword
- Model failure scenarios and simulate downstream impact before they happen for real
- Correlate incident signals and surface patterns across past incidents instead of relying on institutional memory
- Automate runbooks and improve DR/incident reporting so the next 3am page has better information waiting for it
Who You Are
You've run live incidents, not just read about them
- 7+ years across disaster recovery, business continuity, incident management, infrastructure resilience, SRE, or similar. You've commanded a live P1 or P2 and know the difference between calm authority and just staying quiet.
You know cloud infrastructure well enough to ask the right question fast
- Practical knowledge of AWS, GCP, Azure, or similar — compute, storage, databases, networking, and where each of them actually breaks. You've run live DR failover tests, not just written the plan for one.
You think in frameworks but talk in outcomes
- Practitioner-level understanding of ITIL service management, business continuity, problem management, and service design — but you translate that into decisions people can act on, not slideware.
You communicate clearly when everyone else is panicking
- Strong written and verbal communication under pressure. You can brief a war room and an executive in the same hour without either one leaving confused.
You treat resilience posture as a fact, not a feeling
- You give leadership honest, data-backed visibility into where recovery capability actually stands against business risk — including when the answer isn't good.
You're fluent with AI tools, not just aware of them
- Comfortable using AI for scenario modelling, incident triage support, pattern detection, and reporting — you know it accelerates the work, not that it replaces the judgment call.
You close the loop
- You hold teams accountable for fixing what DR exercises, post-incident reviews, and audits find — including when it's not your team's mess to clean up.
Bonus Points
- Direct experience interfacing with Compliance and Risk on regulatory classifications, audit readiness, and evidence management
- Experience building real-time DR readiness dashboards or leadership reporting on resilience posture
- Experience moving continuity processes from manual checklists toward automated or self-healing infrastructure
The Honest Reality
This is demanding work. You'll be the person paged when a database fails and the person expected to have already tested for it. You'll challenge system owners on assumptions they've held for years, and sometimes they won't like it. Post-incident reviews will surface things that make people uncomfortable, including decisions you made in the moment with incomplete information.
But you'll build the muscle that keeps a platform serving millions of traders standing when things go wrong — and you'll have the authority, in the moment that matters, to make the calls that matter.
If you want DR to be a document nobody reads, this isn't it. If you want to own reliability as an engineering discipline, it might be.
