Spending judgement where it matters: how we structure AI-native delivery

Jean-François Moy

10 min read Original article ↗

A few weeks ago I wrote about cognitive debt: the slow loss of judgement that comes from letting a model think in your place. The natural question that follows: how do you use these tools seriously, at full speed, without paying that price?

Here is my answer, after a year of building a production fintech this way. It isn't using the tools less. It isn't trying harder to stay alert, because attention will fade when deadlines loom closer. It's structure: a delivery pipeline where human judgement focuses on the thinking and decisions that deserve it. Everything else is enforced by gates that don't depend on anyone's attention surviving a deadline.

AI-native does not mean AI-autonomous. It means the delivery system is designed from first principles around what agents do well and what only humans can do, the way cloud-native never meant nobody runs infrastructure.

The division of labour is strict, and worth stating up front. Humans own the product thinking, the thin breakdown of work, the architecture and the design: every ticket carries those decisions before an agent sees it. Agents own the implementation inside those constraints, and the gates verify it. Every line an agent writes is read by its engineer before anyone else sees it: the change itself, not an agent's summary of it. Nothing in this system asks a model what to build, or how the system should be shaped.

The scarce resource has changed

For my whole career, engineering capacity was the constraint. Roadmaps were shaped around what the teams could carry. With agentic tooling, that constraint has clearly shifted: a small team can now produce what used to take a department.

What hasn't moved is the supply of judgement. Someone still has to say: we are solving the wrong problem; this technical design will not scale and hurt us in a year; this edge case is the one that matters.

I think that is now the real job of an engineering leader. It is not adopting AI; everyone has. It is about deciding where human attention goes, and building the system that encourages it and protects it.

The initial lag can fool you

The dangerous thing about uncontrolled usage is that it feels great. The tools are impressive, output pours out, velocity charts point up. Nothing seems wrong.

The bill comes later, and it compounds quietly: vicious bugs in code nobody truly read; confident implementations of acceptance criteria that were invalid, or misunderstood, because nobody challenged them before building; a team that no longer knows its own codebase, because everything was delegated and stamped. By the time the symptoms show, you don't have a bug. You have an archaeology dig, run by people who have lost the fluency to dig.

Cognitive debt and technical debt end up collapsing into the same debt. The cost is simply deferred, invisible as it grows, and due all at once. "It's working fine so far" is not evidence. This initial lag is the trap.

What the structure looks like

The pipeline our team runs replicates the roles of a delivery team, sequenced so work flows from ticket to implementation to released change, with a quality gate at each handoff. Every engineer runs it the same way, each owning the judgement calls on their own tickets. They are ultimately accountable for the outcome.

It starts before any code is written. Tickets are authored as user stories that state the customer problem being solved and how we intend to solve it, with acceptance criteria covering the edge cases and the non-functional requirements. A good ticket is a good ticket, whatever the audience; agents didn't change what clarity looks like, they raised the price of skipping it. Each ticket is then challenged: the system's first job is to argue with the scope, highlight misalignment with the existing system, surface the gaps, and openly ask the contentious questions. In the meantime, the engineer shapes the ticket and the technical design. Anything diagnostic follows a data-first rule: metrics, logs and data records before hypotheses.

The implementation starts with a test-first approach against the acceptance criteria: the behaviour is pinned before a single line of production code exists. Once the implementation is complete, a suite of reviews runs in parallel, each an independent lens: one checks the change against the ticket acceptance criteria, one against the project's architecture and conventions, one for API-contract stability, one for security and the supply chain, and finally, a simplification pass whose only job is to remove complexity that doesn't pay rent. None of this depends on someone remembering to run it.

The final brick is the live validation: an agent runs the change against a real environment, driving the actual UI, checking database side effects, and checking logs. It walks the happy path, but also pokes at the unhappy ones. The acceptance criteria are proven on a running system, beyond code testing.

Before any of it is shared, the engineer reads the whole change and iterates on it. The rule is old and simple: don't put in front of others what you haven't read yourself. The gates argue with the code; deciding what to do about their findings is human work, and this reading loop is where the engineer makes the change their own.

Then the engineer demos the finished feature themselves, recording a video walkthrough in which every acceptance criterion is exercised by hand. The demo is more than a check: it is a feedback loop, for the engineer, who feels the rough edges the moment they drive the product as a user would, and for everyone watching, who can challenge the result and improve it. Live validation is an agent's evidence; the demo is the engineer standing behind the user's experience, and it is never delegated.

By that time, a rich PR is opened and merge-gated on clean re-review by a separate agent, green CI and explicit human sign-off. That sign-off means what it has always meant: accountability. The engineer read and shaped every line before the PR existed; signing off says this change is mine, its behaviour and its consequences included. What the pipeline removes from review is the grind, not the ownership.

Our "kickoff" command carries a ticket from scope-challenge to merged PR. During that time, human attention is spent at exactly two points: shaping the work at product and engineering level before it starts, and the contentious calls the gates escalate along the way. The tiny implementation details that used to eat whole review cycles never reach a person.

Why this keeps judgement sharp

Cognitive debt grows when you fully delegate your judgement and the model does your thinking. Your role is to stamp its output. Our methodology reverses that twice.

First, it concentrates the thinking that stays human. Nobody reviews boilerplate or formatting, so the judgement each engineer does exercise, on scope, design and the genuine hard calls, gets full attention. The skill you keep is the skill you use.

Second, the gates are structural, not optional. Speed and safety stop competing the moment quality checks are steps the next stage depends on, rather than qualities someone must remember to practise. A pipeline that cannot merge without independent review, passing tests and live validation doesn't get lazier as the people get faster. This rigour pays off.

There's a compounding layer too. The system remembers the team's conventions; lessons are taught and documented once. Preferences become guardrails instead of tribal knowledge. Every mistake and inefficiency becomes a rule, exactly once.

These gates do not slow us down: they spend machine time, not human attention, running in parallel while we shape the next piece of work. Working this way, three engineers took a new product from concept to public launch in six weeks, on a platform that processes millions of financial events a month, in software where a wrong number in front of a customer is unacceptable. I don't believe that pace, at that bar, is reachable for a team this size any other way.

What it costs

None of this is free. The harness took real time to build and tune; the gates encode our conventions and, more importantly, our architecture, and encoding them was work. That last point matters: the codebase has a purposeful shape, decided by people, and the pipeline defends it on every change. A model without that constraint will happily grow you a system with no shape at all. The whole system lives or dies on well-formed acceptance criteria: feed it vague or ambiguous scope and it will confidently build the wrong thing. That's why the scope-challenge stage exists, and why it's the one stage I would never automate away. The live-validation environment is infrastructure that needs an owner, like any other.

There is a limit that belongs in every engineer's mental model. These models remain probabilistic: the likelihood of the next token being the correct one rests on how well the problem is represented in the training data and in the context you have laid out. Well-trodden problems implement beautifully. A complex problem with little public precedent will often come back subtly wrong, however sound the design that framed it. Part of the judgement this system asks of engineers is anticipating that boundary: knowing what problem they are actually solving, how much context they have really provided, and when the honest call is to write the implementation themselves with the agents demoted to review and test. Delegation is a decision made per problem, not a default.

And one problem I won't pretend to have solved: this model assumes seasoned judgement at the gates. How a junior grows that judgement inside a system that shields them from the grind that used to build it is, I think, the hardest problem in this new AI-era engineering leadership. This is a problem we need to collectively tackle: the future of our discipline and our companies depends on this new generation of software engineers developing their judgement.

A message to engineering leaders and engineers out there

Adoption of agentic programming is not a question any more; the tools are everywhere, and their impact is undeniable. The divide that matters is between teams that use them under control and teams quietly accumulating the lag-hidden debt above while their dashboards look great. The second group may not feel wrong yet. That's the point.

For leaders, the path that works takes more design effort than either enthusiasm or caution. Decide which decisions deserve human judgement in your context and domain. Build the structure that routes those decisions to people. Make every other quality property a gate. The result is the combination everyone assumes is a trade-off: faster delivery, less debt, and engineers whose judgement sharpens with use, because it's the only thing they spend all day exercising.

For engineers, the gates are just skills: plain files that encode what your team knows (the domain, the conventions, the architecture, the strategy behind the codebase). Anyone can write one. Champion the framework, own it, improve it, add the rule you would have needed yesterday. It isn't your manager's system; it's the team's shared memory, and everyone can contribute to it.

One boundary worth stating plainly: everything here is about building things right. Whether you're building the right things is a different discipline, and no pipeline answers it for you.

As the tools keep improving, the structure decides whether they make your team more capable or just more prolific.