Every business function is redesigning for AI, except software engineering

The local optimum trap in AI-assisted engineering

When AI entered customer service, nobody said “just give the agents ChatGPT and carry on.” When it reached finance, nobody pretended the existing approval chains would hold. Leadership understood, almost immediately: the process has to be redesigned around the capability. You don’t get transformation by adding a tool to an unchanged system.

In software engineering, we’re making exactly that mistake. And we’re making it because the tools are too good to force the question.

The exemption nobody decided to grant #

Claude Code, Copilot, Cursor, Codex. These are genuinely excellent tools. They produce real code. They pass real tests. They accelerate experienced engineers and lower the barrier for junior ones. I use them daily. I’m not here to tell you they don’t work.

They work. That’s the problem.

In every other function, the AI is obviously not good enough to just drop into the existing process. The gap between what the tool can do and what the process requires is visible, immediate, and uncomfortable. That discomfort is productive. It forces the redesign conversation.

In engineering, the tools are good enough to carry the individual task. An engineer with Copilot writes code faster, explores solutions faster, iterates faster. And that felt experience of individual productivity creates a specific cognitive trap: it suppresses the organisational question that every other function is being forced to confront.

Should we redesign the process around AI, or just give everyone better tools?

Every other domain is being pushed through the uncomfortable redesign because their tools aren’t good enough to create the illusion. Engineering leadership is being protected from that necessary discomfort by the quality of their tools.

This is not a theoretical argument. If you’re in a regulated industry, the governance separation your AI tools are making invisible is one your regulator already expects you to maintain. DORA, the EBA’s Model Risk Management Guidelines, DNB’s supervisory expectations: all require that governance bodies retain decision responsibility regardless of the tools used. The tools haven’t changed that obligation. They’ve made it easier to forget.

The local optimum trap #

There’s a name for this pattern. You’ve found a configuration that produces visible, real output. It measurably works. And precisely because it works, it prevents you from discovering the configuration that would work at a fundamentally different level.

That’s a local optimum. And local optima have a nasty property: the longer you stay, the harder it is to leave.

Every month of “Copilot is working great” is a month where you’re not building the organisational capability that would actually compound. The gap widens invisibly, because the tools keep producing visible output.

The tool debt accumulates invisibly because the tool keeps producing visible output.

Why engineering needs this redesign more, not less #

Your engineering organisation isn’t slow because your people code slowly. It’s slow because every change requires establishing that you know enough to make it safely. Understanding what depends on what, what breaks if this moves, who needs to sign off. That’s the actual bottleneck. It was the bottleneck before AI tools, and it’s the bottleneck now.

AI coding tools make the coding part faster. They don’t touch the bottleneck. They fill the pipe before the clog.

And here’s what makes engineering uniquely vulnerable: the testability of each individual change is precisely what prevents leadership from asking the systemic question. Every change passes its tests. Every change is reversible. Because that feels safe, nobody builds the organisational picture of how the estate is evolving as a whole. The safety of the parts becomes the complacency about the whole.

Which means the more successful your AI tool adoption, the more pressure builds at the point you haven’t redesigned. You’re accelerating into your own constraint.

Every other function figured this out because their tools couldn’t mask it. In engineering, the tools mask it beautifully, until deployment frequency tells the real story.

Where the gap shows up #

If you’re an engineering leader, here’s how to check whether you’re in the trap.

PR velocity vs. release frequency. Google’s 2025 DORA report found that AI coding assistants produced a 98% increase in merged pull requests, while organisational delivery metrics stayed flat. If your own numbers look similar, you’ve found the gap. The tools filled the pipe. The pipe is still clogged where it was before: establishing that a change is safe to ship.

That’s not a tooling problem. That’s a process design problem. No amount of better autocomplete will fix it.

Code quality under AI assistance. GitClear’s 2025 research documented a fourfold increase in code duplication since AI tool adoption. 40% of developers report that AI has increased technical debt. For the first time in measured history, developers are pasting code more often than they are refactoring or reusing it. The tools are producing more code. Whether it’s better code is a question the current tool-level approach cannot answer, because the tool grades its own output.

The senior engineer test. When a senior engineer leaves, how much of your AI-assisted capability leaves with them? Their prompts, their context-loading habits, their instinct for what to check. If the answer is “most of it,” then your engineering knowledge lives in your people, not in your organisation. One scales with headcount. The other scales against it.

What “redesign” actually looks like #

Every other function that’s getting real value from AI has done some version of the same three moves. These are not technical moves. They are organisational ones.

First: separate who does the work from who decides it’s good enough, and commit to the criteria before the decision. In customer service, the AI handles the conversation, but quality evaluation is systematic, not left to the individual agent. In finance, AI processes the transaction, but approval authority is structural, and the criteria for approval exist before the transaction arrives. In engineering today, the AI tool writes the code and runs the checks and can change the checks if they fail. The person who does the work is the same person who decides it’s good enough. And the criteria for “good enough” are often implicit, reconstructed after the fact to justify the decision already made. That’s not governance. That’s rationalisation. The redesign means deciding what “safe to ship” means before the change is made, naming who decides, and making that decision replayable.

Second: make knowledge organisational, not individual. When a customer service team redesigns for AI, they build shared playbooks, response templates, escalation rules. The AI operates within a structure that captures what the organisation knows. In engineering today, every AI session starts from scratch. The context window is the engineer’s private workspace. Nothing compounds. Nothing transfers. The redesign means building shared, verified context about your systems, what depends on what, what the rules are, what’s been checked, so the AI operates within what the organisation knows rather than what one engineer remembers.

Third: make the governed path the easy path. This is the one most organisations get wrong. If doing the right thing is harder than improvising, people will improvise. In the redesigned process, an engineer doing a standard change should find it easier to use the governed, AI-assisted path than to do it ad hoc. The organisation makes the safe path the fast path. Through design, not enforcement.

Why this requires a platform, not better tools #

Here is the uncomfortable truth the article has been building toward: these three moves do not happen at the tool level.

Copilot, Claude Code, Cursor. By design, authority, context, and evaluation all live inside the engineer’s environment. The engineer decides the scope. The engineer loads the context. The engineer evaluates the output. The tool inherits whatever the individual brings. That is the right design for exploration and discovery. It is the wrong design for governed execution at scale.

The three moves above require a different abstraction layer: a platform where task scope, context boundaries, evaluation criteria, and decision authority are defined by the organisation, not by the individual engineer. The AI operates within that platform. The platform defines what the task is, what context it needs, and what “done” looks like. The underlying model is a component, not the architecture.

Your current tools are your discovery layer, and they should stay. They are how engineers explore, prototype, and find patterns worth industrialising. But the governed execution path lives at the platform level. The tool can serve that platform. It cannot be that platform.

This is the investment most engineering organisations have not made, because tool-level results feel sufficient. They are sufficient for individual productivity. They are insufficient for organisational capability.

What this looks like in practise #

Take a concrete example. An engineer proposes a change to a shared data model used in regulatory reporting. Today, the AI tool reads the repo, generates the migration, runs the tests. The engineer reviews, approves, ships. All the context about why this change is safe, what downstream services depend on this model, what compliance surfaces it touches, who was consulted, lives in that engineer’s head. The decision to ship was made and justified in the same motion.

Now imagine the redesigned version. Before that change is made, the organisation has already answered: what criteria make a data model change safe to ship? Who decides? The criteria are written down, not as a 40-page governance document, but as three concrete questions this class of change must answer. The engineer makes the change with AI assistance, but within a platform-defined scope that provides the relevant context and enforces the relevant criteria. The decision to ship is made against pre-committed criteria, by a named decider, and the decision is logged in a way that can be replayed.

Both paths are fast. The difference is that the second path produces organisational knowledge. The next engineer who touches this model inherits everything the system learned. The first path produces a closed context window and a Slack thread nobody will find.

What this costs you #

Let me be direct about what’s at stake.

Every month your AI coding tools work well enough to postpone the redesign question is a month where your engineering knowledge accumulates in people instead of in your organisation. People leave. Context windows close. Nothing compounds.

Meanwhile, every other function in your organisation is being pushed, by the productive discomfort of imperfect tools, to build AI capability into the process itself. They’re building organisational muscle. Engineering is building individual skill.

In two years, your customer service operation will have AI-native processes that work regardless of who’s on the team. Your finance function will have AI-augmented controls that improve with every cycle. And your engineering organisation will have a collection of very productive individuals whose capability walks out the door every evening and has to be rebuilt every morning.

You’re investing in making your engineers faster at the part that was never the bottleneck. And every month that investment feels like it’s working is a month you’re not making the one that would actually compound.

That’s the local optimum trap. It doesn’t feel like a trap. It feels like progress.

Here’s the test. Can you name who decided your last significant AI-assisted change was safe to ship? Can you state the criteria they used, criteria that existed before the change was made? Could you replay that decision with the same criteria and reach the same outcome?

If not, your governance is invisible. Your regulator already expects it to be visible. And every AI tool you adopt makes the cost of that invisibility higher.

A caveat worth naming: this solves the visibility problem, not the full governance problem. You will likely discover mid-implementation that authority structures don’t exist at the granularity required. That discovery is a feature, not a failure. It surfaces exactly what needs redesigning. The first decision class you make repeatable will expose governance gaps you didn’t know you had. That exposure is the beginning of the real work.

The exit is a platform design decision, not a procurement decision. And the longer you wait, the more it costs to leave.