What Makes Difficult Incidents So Difficult?

I’m using the term “difficult incident” carefully here — because, sure, that begs the question: what makes an incident easy or hard?

For this post, lets agree that a difficult one usually involves multiple teams, unclear ownership, and no obvious explanation.

When Conversations Stall

If you’ve been on support duty, you’ve probably seen incident chats that look something like this:

Engineer 1: “Anyone from Networks online? Can’t connect ClientData service to its database.”

Network engineer: “I need source and destination IP addresses and port numbers.”

Engineer 1: “We only have service and dependency names in the config. I don’t know the IPs.”

Now we’re stuck waiting — the conversation drags, and precious time is lost.

When Conversations Flow

Compare that to this version:

Engineer 1: “Anyone from Networks online? ClientData service on HOSTXXX can’t connect to its DB at 192.168.x.x:1521. Also seeing SYN_SENT in netstat.”

Network engineer: “Thanks. SYN_SENT usually points to a firewall issue.”

This second exchange moves quickly and gets closer to resolution faster.

It’s All About Mental Models

The difference? The application engineer understands a bit about networks, and the network engineer understands enough about applications to engage meaningfully. Their mental models overlap — and that overlap is key.

But as we introduce more abstraction to make it easier to ship and run code, this shared understanding becomes rarer. You don’t have to know how something works… until you’re knee-deep in an incident.

What Happens When We Have Different Understandings of The Same Thing?

There’s actually a term for this: "Fundamental common ground breakdown."

It’s what happens when people working the same incident make conflicting assumptions about what’s been said — and it often leads to delays or full-blown disasters.

A Personal Example

I was new in a job at a financial trading firm when I spotted high memory usage and rising connections on a load balancer. I suggested a rolling restart of instances to avoid trouble. Platform engineer agreed, and we kicked it off.

Except... it led to a full outage.

Turns out, the load balancer took a while to mark instances as “up.” I assumed “rolling restart” meant a fast, safe handoff between instances. They didn’t clarify. Boom. Outage.

That’s a mental model mismatch — and it cost us.

How to Build Overlap on Purpose

Expanding your knowledge beyond your immediate area — even just a little — can prevent incidents or help resolve them faster. Think application developers understanding databases. Database engineers understanding network and operating systems.

We can leave that overlap to chance… or we can intentionally build it.

Rotating engineers across teams or running incident drills are great ways to build those shared models. While we can't ensure identical mental models, we can foster overlap among team members' mental models.