Incident Response in the Age of AI (Incident Fest)

Incident Fest is our fun, free virtual festival. More importantly, it’s a place for incident responders to share their stories and learn from others.

This year, we were delighted to welcome a variety of exceptionally talented speakers to the stage to discuss the evolving (and not always harmonious) AI/incident response relationship. Note that these are a few highlights, so please go and watch the full talk recordings after!

The Theme

‍

Where AI Offers Wins

All three speakers were upfront that AI is already pulling real weight in incident response:

• Stu Rimell (Uptime Labs) noted that LLM tooling "is starting to show promise in addressing the low-hanging fruit of incident response" i.e. auto-summarising chat threads, intervention suggestions, remedial PRs.
• On code itself: "AI is a tremendous tool for understanding code"
• On freeing up attention: "the idea of being able to offload some or most of that cognitive load is legitimately exciting", letting responders focus on the genuinely hard & novel parts of an incident instead of the rote ones.
• J. Paul Reed (Chime) cited research showing an upside when an AI's diagnostic suggestions were correct - the humans using it "performed 53 to 67% better than when they worked without AI assistance".
• Sylvain Kalache (Rootly) pointed to Meta's agentic mutation-testing tool (thousands of synthetic bugs generated, 73% accepted by engineers as valid tests) calling it "a massive win that would have been impossible to do manually or extremely resource-intensive"
• His summary: "AI-assisted coding is not going anywhere. It's here to stay" - the goal isn't resistance; it's building the muscle to use it well.

In other words, AI offers a variety of exciting, innovative ways to make engineers’ lives easier. The question then becomes ‘how do we enable AI safely in the short and long-term?’, which is  the question the festival aims to unpack carefully.

‍

The Leftover Principle (Stu Rimell, Uptime Labs)

Stu opened with a story: he’d just landed in Seattle for SREcon and his rideshare app was convinced he was standing on the street outside the terminal when he was actually three floors up in the parking garage. GPS is a solved problem - until the moment it isn’t, and you’re back to reading signs and asking strangers for directions.

Stu’s story, he said, is exactly what it feels like every time automation reaches the edge of what it can do.

The Leftover Principle describes the tasks left over once automation has done all it can, or was designed to do.

• Leftover tasks tend to be either too trivial to bother automating or too rare, complex and novel to automate at all.
• Incidents fall squarely into that second (gnarlier!) category.

Historical grounding

The concept traces back to Alphonse Chapanis: the ‘godfather of human factors,’ who redesigned the B-52 cockpit after pilots kept retracting the landing gear instead of the flaps. Then, it passed to David Woods and Erik Hollnagel, who pushed back on ‘automate everything; thinking. The canonical reference is Lisanne Bainbridge’s 1983 paper Ironies of Automation - required reading, Stu notes. Its key paradox: the more you automate, the more important the human role becomes, not less.

At London’s OOPS community meetup, Stu heard two approaches emerging: auto-diagnosis and copilot mode. LLM tooling shows promise on the low-hanging fruit; complex scenarios remain human territory.

‍

The Four Dragons

• Harder leftovers: what’s left is rarer and more novel by definition - that’s why it wasn’t automated already
• Skill atrophy: less practice erodes skills; Bainbridge warned systems would end up ‘riding on skills which later generations of operators cannot be expected to have’
• Situational context loss: arriving only at the leftover point is ‘like coming into an argument halfway through’
• Accountability gap: humans remain accountable for incidents even as their expertise to actually exercise that accountability erodes. The risk is ending up in a job that's "very boring but very responsible", with no real opportunity to build or maintain the incident response skills that responsibility demands.

Stu backed this with numbers: GitHub’s weekly commits jumped from 19 million to 275 million, and the 2026 Faros AI Engineering Report found incidents per PR up almost 250%.

“The dream of being able to sleep through on-call, as one AI SRE vendor advertises, is for some a pleasant one, but for some it’s a horrific nightmare.”

Ultimately, ‘augment, don’t replace’ is a decent rule of thumb. Copilot beats that fantasy; lean on Klein et al.'s 2004 team-player challenges; and above all, practice: game days, tabletops, chaos engineering, the way pilots keep training despite autopilot. It’s the gap Uptime Labs is built to close.

‍

Vibe Firefightin’: When AI Has Entered the Incident Bridge (J. Paul Reed, Chime)

J. Paul Reed, who holds a master’s in human factors and system safety, covered ironies of automation and AI, joint cognitive systems, and tips for the incident bridge.

“We were doing agentic stuff without AI long before AI came along.”

Ironies of automation and AI

Manual skills deteriorate when unused; automation forces a speed-versus-correctness trade-off, so we spot-check for acceptability rather than correctness. Tracing an AI’s reasoning can be flatly impossible.

Forty years after Bainbridge, researcher Mica Endsley extended this to AI: the more capable AI seems, the worse we get at compensating for its shortcomings (see: fatal Tesla Autopilot disengagements). Plus, he more naturally it talks, the harder it is to judge if it’s lying.

“It’s become sort of… the ‘you’re absolutely right’ joke…

Joint cognitive systems

Incidents are worked by people, data, automation and AI, all acting as ‘agents.’ Coordination depends on autonomy, authority, directed attention and interpredictability: AI still falls short on the last two. As Dave Woods puts it: “Technologists often mistake connectivity… for coordination.”

Tips for the bridge

Tell your incident commander when you’re using AI; post your interpretation, not AI slop; engage the AI rather than letting it broadcast unsolicited answers; and ask for explanations, not recommendations. In Woods’ study of nurses using AI diagnostics, correct AI boosted performance 53-67%, but misleading AI made it 96-120% worse than no AI at all.

‍

More Code, More Incidents? Staying Reliable When AI Writes the Code (Sylvain Kalache, Rootly)

Sylvain, who leads Rootly AI Labs, framed the talk around a formula: incident rate = C × P

(C represents the volume of changes; P the odds of introducing a failure)

C is climbing fast

Coding-assistant-heavy developers ship 10x more code, PRs have doubled in size and Rootly’s data shows incidents per customer tripled since 2023. Subsequently, Amazon and even Anthropic have had rough reliability stretches tied to this. Plus, there’s less help available, since  git blame might now lead to a shrug:

“Hey, sorry, mate, I didn’t really write this piece of code. I just prompted it. You are on your own.”

P is climbing too

CodeRabbit’s research found AI-generated code ships with meaningfully more bugs. Favourites: AI tests that validate already-broken logic rather than real intent; a Vercel agent that hallucinated a repo ID and deployed the wrong codebase; and slopsquatting, where attackers pre-register package names that LLMs are likely to hallucinate. His point: every AI screwup has a human equivalent, just ~10x faster - blameless culture should extend to LLMs too.

Keeping P low

The fundamentals (deploys, observability, resilience) matter more than ever. Rootly risk-triages PRs by blast radius and revertibility, and requires every PR to document its why (human-only) and a revert plan. Intercom ships 180x/day behind flags that can be killed in under a minute, and Meta’s agentic mutation testing generated thousands of synthetic bugs, 73% of which were accepted as valid tests.

Sylvain summed up the tension:

“Your manager is like, ‘chop, chop, chop - you need to do more with less because you have AI by your side.’”

Like pilots training for an engine failure they’ll likely never face, teams need to keep practising incident response as AI takes over more of it (which is why Rootly partnered with Uptime Labs on Rootly Academy).

Sylvain’s closing line: C is out of your control. P is your job.

‍

For the Full Experience, Visit the Festival!

Instead of overpriced beers and dubious headliners, come and explore Incident Fest in the comfort of your own home. As well as recordings of these excellent talks, there’s a poll booth, an incident challenge with prizes and more!