Why "We Didn't Know" Is the Most Expensive Sentence in Retail IT

I've sat in enough post-incident reviews now to recognise the moment it comes.

Not shouted. Not defensive. Usually just quiet. Someone says it almost apologetically: "we didn't know." And the room moves on, because what else is there to say?

I used to think that sentence was about a gap in effort. It isn't. I've spent years around IT and operations teams running distributed retail estates, and the people in those rooms are almost always working harder than they should have to.

What they don't have is visibility. And I think we, as an industry, have been far too comfortable treating that as a minor detail rather than the actual problem.

---

This isn't a dig at anyone

I want to say that clearly, because it would be easy to read a piece like this as having a go at IT teams. It's the opposite.

Every operations leader I've spoken to is doing their best with an estate that's grown more complicated every year, more sites, more systems, more vendors, while the way they find out something's wrong has barely changed. A store calls. Someone raises it. Somebody starts digging. That's not a people problem. People can't fix visibility they were never given.

Three things I keep hearing, in different words

I don't have a single customer story I can point to here, and I don't think I need one, because I hear some version of the same three things constantly, from people who've never spoken to each other.

They find out too late. Something's been going wrong for a while before anyone in IT knows about it. A till slowing down. A connection dropping in and out overnight. None of it trips an alarm, because most systems are built to tell you something has failed, not that it's starting to.

They don't know what it's costing them. Is this one till in one store, or is it quietly happening in twenty? Nobody can say, in the moment, whether this is a five-minute problem or a five-figure one. So people end up prioritising by instinct, or by whoever's shouting loudest, which isn't really prioritising at all.

It takes too long to work out what's actually wrong. A huge amount of time in this industry goes into figuring out the problem before anyone even starts fixing it. Multiply that by however many incidents a year, across however many sites, and it's an enormous, mostly invisible cost that nobody's ever really added up.

---

None of this is one company's fault

I don't think this is a story about any specific organisation getting it wrong. It's what happens when a retail estate grows the way they all grow, one system bolted on at a time, each one solving its own problem, none of them designed to talk to the others or to whoever's trying to run the whole thing.

Nobody sat down and decided to build it that way. It happens gradually, over years, until one day you're running hundreds of stores with less real-time visibility than you'd probably want if you were starting from scratch.

What actually needs to change

The fix isn't another alert. Most estates already have plenty of those. What changes things is knowing, in real time and across every site, what's actually going wrong, about to go wrong and what it's costing you, before someone has to find out the hard way.

Once that's visible, prioritisation stops being a guess. Nobody's choosing between problems on instinct or on volume of complaints. The five-figure issue in three stores gets attention before the five-minute issue in one does, because for the first time, someone can actually tell the difference.

It changes what engineers are sent to do, too. Instead of being sent out to work out what's wrong, they get sent to fix it, because someone already knows. And somewhere down the line, "we didn't know" stops being the honest answer in that post-incident review. Not because anyone worked harder. Because the gap that made it true in the first place isn't there anymore.

That's the part I keep coming back to. Not that any of us are bad at our jobs. Just that a lot of us are doing a genuinely good job with a lot less visibility than what we're running actually deserves.

If this sounds familiar, I'd like to hear how you're handling it today.

Ross Palmer

Founder & CEO, OpSiteAI