The Pull Request is not the product

There is a particular kind of conversation that happens inside large software companies during performance-review season. Engineers begin collecting evidence of what they have done over the previous six or twelve months. They count projects, pull requests, design documents, incidents resolved, releases completed, systems migrated, and sometimes even the number of teams they have “influenced”. Managers assemble this information into review packets, calibration documents, promotion cases, and talent discussions.

None of this is inherently unreasonable. Engineering work is difficult to observe, especially when hundreds or thousands of people are contributing to a product at the same time. A company needs some way to understand who is doing good work, where teams are struggling, and whether an expensive engineering organisation is producing enough to justify its cost.

The trouble begins when evidence of activity quietly becomes evidence of value.

A pull request is easy to count. A line of code is easy to attribute. A ticket has an owner, a status, and a completion date. A product outcome is much harder to assign to one person. Customer retention may improve because of engineering, design, pricing, marketing, sales, support, or a change in the market. Reliability may improve because of one architectural decision made months earlier. A project may appear successful because it shipped on time, even though customers barely use it. Another project may look slow and uneventful while preventing an outage, removing years of accumulated complexity, or saving several teams from building the wrong thing.

When organisations struggle with that ambiguity, they often retreat towards what is visible. Activity becomes a substitute for impact because activity leaves a convenient trail.

This is especially common in large, respected product companies. These organisations are rarely naïve about engineering. They employ experienced technical leaders, maintain sophisticated career ladders, and speak extensively about customer obsession, ownership, quality, and long-term thinking. Yet the systems used to assess engineers can still drift towards measurable proxies. A senior engineer is expected to demonstrate a larger “footprint”. A staff engineer is expected to show influence across several teams. Managers are asked to provide concrete proof of delivery. Promotion committees want evidence that can survive comparison across departments.

The result is a strange contradiction. Everyone agrees that lines of code are a poor measure of productivity, but engineers still feel pressure to show how much they produced. Everyone agrees that pull-request counts do not capture impact, but contribution graphs and delivery records still appear in performance conversations. Everyone says that technical judgment matters, yet the easiest judgment to defend in a calibration meeting is often the one supported by a long list of visible outputs.

The system does not need to explicitly reward pull-request volume for people to understand the signal. Employees are remarkably good at identifying what an organisation actually values, regardless of what its leadership principles say.

When visibility starts shaping the work

Once people suspect that visible output matters, behaviour changes in small and rational ways.

An engineer who could solve a problem by modifying an existing component may decide to create a new service because the larger project demonstrates broader ownership. Someone may divide a change into several pull requests, partly for reviewability but also because the work appears more substantial. A senior engineer may join multiple initiatives to establish organisational reach, even when concentrating on one difficult problem would have produced a better result. A team may continue building a platform because stopping it would leave fewer accomplishments to report at the end of the half.

None of these choices needs to be dishonest. People are responding to incentives while continuing to believe they are doing good work. Most organisational dysfunction is not produced by villains gaming the system. It is produced by capable people adapting sensibly to what the system recognises.

This is why engineering organisations can become extremely active without becoming meaningfully more effective. There are more projects, more internal platforms, more migrations, more abstractions, more meetings, more roadmap items, and more progress updates. The organisation can point to an enormous amount of completed work, yet customers may still struggle with the same core workflows. Engineers may still wait weeks for simple changes. Reliability may remain fragile. The codebase may become harder to understand even as delivery dashboards remain healthy.

The most dangerous part is that activity produces its own evidence. A complex system creates maintenance work. Maintenance work creates more tickets. More tickets create more completed work. The organisation then uses that work as proof that the teams responsible for the complexity are highly productive.

A software company can therefore become trapped in a cycle where it rewards people for solving problems that its own earlier solutions created.

This does not mean that a large number of pull requests is suspicious or that substantial code contributions have no value. Some products genuinely require a great deal of implementation. A major security programme, platform migration, regulatory change, or new product area may involve thousands of changes and years of coordinated effort. The mistake is not counting activity. The mistake is assuming that more activity automatically means more contribution.

Output is evidence that effort was converted into artefacts. It is not proof that those artefacts were worth creating.

Why lines of code remain attractive

Lines of code are an obviously flawed metric, which is precisely why most companies would deny using them directly. Very few serious engineering leaders would openly rank developers by how much code they write.

Yet the underlying idea survives in more respectable forms. Engineers are praised for “high throughput”, “consistent delivery”, “sustained execution”, or a “strong volume of contributions”. Teams are encouraged to increase velocity. Senior employees are asked to show a growing scope of ownership. The language is more sophisticated, but the intuition remains similar: more visible production suggests greater value.

Software does not behave like a physical product, however. Code is not inventory. It is closer to a long-term obligation.

Every line added to a production system may need to be reviewed, tested, secured, monitored, documented, upgraded, debugged, and eventually removed. It increases the amount of knowledge future engineers must carry. It creates more places for assumptions to hide and more interactions that can fail. Even well-written code has a cost after it is shipped.

This makes software unusual because addition and removal can both represent progress. A team may create value by building a new capability, but it may also create value by deleting an obsolete one. An engineer may solve a problem by introducing a new abstraction, or by discovering that the abstraction is unnecessary. A migration may be valuable because it moves the company to a better architecture, or wasteful because the expected benefits never justify the years of disruption.

The direction of the code change tells us almost nothing by itself.

Consider an engineer who spends two months removing a layer of custom infrastructure that was once believed to be strategically important. The work reduces deployment failures, shortens onboarding, cuts infrastructure costs, and allows teams to use a standard internal platform. The final change deletes more code than it adds.

Now consider another engineer who builds a technically impressive framework with thousands of lines of code, extensive configuration, detailed documentation, and adoption across two early teams. A year later, most teams still avoid it because the framework is difficult to operate and solves a problem they do not consider urgent.

The second project creates more visible engineering output. The first creates more organisational value.

The uncomfortable lesson is that a company cannot infer the importance of work from its size. Large projects can be essential or unnecessary. Small changes can be trivial or transformative. Deletions can be reckless or brilliant. The meaning comes from the problem, the result, and the cost of getting there.

Value is the change that remains after the work is finished

Engineering value is often discussed as though it were self-evident, but companies regularly use the word without defining it. Teams say that they are delivering value when they ship features, complete migrations, improve test coverage, or adopt a new platform. Those activities may create value, but they are not value in themselves.

Value is the improvement that exists because the work happened.

For customer-facing work, that improvement may be a higher task-completion rate, fewer abandoned sessions, faster response times, lower error rates, improved accessibility, greater trust, or a capability that customers are willing to pay for. For platform work, it may be shorter lead times, fewer deployment failures, reduced operational load, lower infrastructure cost, or faster onboarding for engineers. For security and reliability work, value often appears as a reduction in expected loss rather than an increase in visible usage.

The definition needs to be broad enough to include revenue, cost, risk, customer experience, organisational leverage, and learning. It also needs to be strict enough to distinguish an outcome from an artefact.

A new service is an artefact. Lower latency is an outcome.

A design system is an artefact. Faster product development with more consistent interfaces is an outcome.

A migration is an artefact. Reduced operational cost, improved reliability, or the removal of an important constraint is an outcome.

A machine-learning capability is an artefact. Better decisions, improved conversion, reduced manual effort, or new customer behaviour is an outcome.

This distinction sounds simple, but it changes how work is discussed. Instead of asking how much was delivered, leaders begin asking what became better, for whom, for how long, and at what cost.

Calculating value without pretending it is perfectly measurable

Engineering value will never be reducible to one universal formula. Some benefits are financial, some are operational, and some are strategic. A company that forces every project into an exact monetary estimate will usually end up creating fictional precision.

That does not mean measurement is impossible. It means the calculation should make assumptions visible rather than hiding them behind a score.

For customer-facing improvements, a useful starting point is to estimate reach, the size of the improvement, expected adoption, and how long the benefit will last.

Imagine that a product change affects 40,000 active users each month and reduces the time required to complete a critical task by two minutes. Research suggests that roughly 60 per cent of those users will encounter and benefit from the change. Over a year, the estimated customer time saved would be:

40,000 users × 2 minutes × 12 months × 60 per cent

That produces 576,000 minutes, or 9,600 customer hours saved each year.

The calculation does not tell us the full value. The saved time may increase task completion, reduce frustration, improve retention, or reduce support requests. Those effects require further measurement. What the estimate does provide is a clear chain of reasoning between the engineering work and the expected customer benefit.

Platform work can be evaluated in a similar way. Suppose an internal deployment improvement saves ten minutes for each production release. If the company performs 3,000 releases per month, the change saves 500 engineering hours every month. At an estimated fully loaded engineering cost of £70 per hour, that is approximately £35,000 of recovered engineering capacity each month.

Even that figure should be treated carefully. Saved time does not always turn directly into money. The company may not reduce headcount or spending. The real benefit may be that engineers can deliver more useful work, respond faster to incidents, or experience less operational frustration. The financial estimate is one way to express scale, not a claim that the organisation has received £35,000 in cash.

Risk-reduction work requires a different calculation. The value comes from reducing the probability or impact of a damaging event.

Suppose a fragile dependency has an estimated 15 per cent chance of causing a major incident during the next year. The total impact of such an incident, including lost transactions, service credits, recovery work, and customer damage, is estimated at £800,000. The expected annual loss is therefore £120,000.

If an engineering project reduces the probability of that event from 15 per cent to 3 per cent, the expected annual loss falls to £24,000. The estimated annual risk reduction is £96,000.

The numbers will always involve judgment. The probability may be debated. The cost of failure may be uncertain. That debate is useful because it forces the organisation to articulate why the work matters and how serious the risk actually is.

Learning should also be included. A prototype that proves customers do not want a proposed product may look unsuccessful on a delivery dashboard. If it prevents the company from investing £2 million in the wrong direction, however, the learning has significant value. The experiment did not create a lasting feature, but it improved the company’s allocation of capital and attention.

Any serious value assessment must also subtract cost. Development effort is only the beginning. The calculation should include infrastructure, licensing, support, maintenance, security, coordination, migration, and cognitive complexity. It should also include opportunity cost, because a useful project can still be a poor choice if it prevents a much more important one from being pursued.

A practical expression of engineering value might therefore look like this:

Expected value = customer or business improvement + risk reduced + organisational leverage + learning gained − total lifecycle cost − opportunity cost

This is not a formula to be applied mechanically to individual engineers. It is a way to evaluate work with more honesty than a count of outputs provides.

The individual performance problem

At this point, an obvious difficulty appears. Product outcomes are rarely controlled by a single engineer.

A feature may fail because the original customer research was weak. Adoption may depend on sales, marketing, documentation, pricing, or product design. A platform may be technically excellent but poorly introduced to the teams expected to use it. A migration may be strategically necessary even if it creates no visible customer improvement. Market conditions may change after a project has started.

It would therefore be unfair to evaluate engineers only on whether a project produced the expected business outcome.

The answer is not to return to pull-request counts. It is to separate the quality of an individual’s contribution from the eventual result of the initiative.

An engineer’s contribution can be assessed through the decisions they made within the context they controlled. Did they understand the problem before designing the solution? Did they identify important risks? Did they consider simpler alternatives? Did they challenge assumptions when the evidence was weak? Did they avoid unnecessary complexity? Did they collaborate effectively with product, design, operations, and other engineers? Did they adjust when new information emerged? Did they leave the system easier or harder to operate?

The outcome still matters because engineering does not exist merely to demonstrate good process. However, it should be interpreted alongside the quality of the decisions that produced it.

An engineer who executes a weak strategy perfectly should not receive unlimited credit because the project shipped. An engineer who conducts a disciplined experiment and disproves an important assumption should not be punished because the original idea failed. A senior engineer who prevents an unnecessary programme from starting may create more value than someone who leads a large programme to completion.

This is where performance evaluation becomes difficult, because it requires judgment rather than counting.

Large companies often try to reduce subjectivity by demanding more evidence. That instinct is understandable, especially when compensation and promotion decisions affect thousands of employees. The danger is that the evidence becomes dominated by artefacts that are easy to attach to a review document. The engineer who produces many visible objects has a stronger case than the engineer whose value came through restraint, diagnosis, simplification, mentoring, or preventing bad decisions.

A fair system should not eliminate evidence. It should broaden what qualifies as evidence.

A reduction in incident frequency is evidence. A decision record showing that a team avoided a costly approach is evidence. Adoption by other teams is evidence. Customer behaviour is evidence. Reduced operational load is evidence. Faster onboarding is evidence. A simpler architecture is evidence. The improved effectiveness of engineers who received mentorship is evidence.

A pull request can support that story, but it should not become the story.

Output metrics belong to the system, not the ranking of people

Pull-request volume, review time, deployment frequency, change-failure rate, ticket throughput, and code churn can all be useful. They reveal friction in the engineering system.

A team with unusually long review times may have too few experienced reviewers. A product with high code churn may have unstable requirements. A team that deploys infrequently may be constrained by fragile release processes. Very large pull requests may indicate work that is difficult to review safely. Low activity may reveal dependency bottlenecks, unclear ownership, or an architectural problem.

These signals should lead to questions.

They become harmful when they are used as answers.

If one engineer merges more pull requests than another, the organisation has learned that one engineer merged more pull requests. It has not learned who solved the more important problem, made the better decision, reduced more risk, or created more durable value.

Once individuals believe that a system metric influences promotions or compensation, they will naturally adapt their behaviour. Pull requests may become smaller than necessary. Engineers may seek work with visible ownership. Teams may favour projects that produce recognisable artefacts. Quiet but valuable work becomes less attractive because it is harder to defend.

The metric then stops describing the system and begins changing it.

This is why engineering productivity metrics are safest when applied to teams, workflows, and constraints rather than individual rankings. Their purpose should be to improve how work moves through the organisation, not to produce a league table of developers.

The work that does not leave a large trail

Some of the most valuable engineering decisions produce surprisingly little visible output.

A senior engineer realises that an existing service can support a new requirement, avoiding the need for another platform. An architect persuades several teams to abandon a migration whose benefits no longer justify its cost. A developer finds that a production problem can be solved through a configuration change rather than a six-week rewrite. A platform team removes a custom tool and adopts a well-supported standard. An engineer spends time helping another team make a better decision, even though the resulting code is committed by someone else.

These contributions are easy to undervalue because they reduce the amount of work the company gets to count.

A culture that over-indexes on visible production will gradually reward construction over restraint. New systems look ambitious. Simplification can look incremental. Launching a platform creates a clear ownership story. Preventing one from being built does not. Leading a large migration demonstrates scope. Demonstrating that the migration is unnecessary may appear less impressive, even when it saves the company millions.

This creates a structural bias towards more software, more programmes, more coordination, and more maintenance.

Over time, the company becomes slower because every previous achievement becomes part of the environment that future engineers must understand. Teams spend increasing amounts of time navigating systems that were once introduced as productivity improvements. New platforms are created to manage the complexity of old platforms. Migrations become permanent features of organisational life.

The company does not lack talented engineers. It lacks a reliable way to reward the removal of unnecessary work.

A better question for engineering leaders

The purpose of engineering is not to produce the largest possible amount of software. It is to improve the product and the organisation through the responsible use of technology.

That improvement may require a substantial new system. It may also require a small change, a deletion, a migration, an experiment, a difficult investigation, or a decision to stop.

Leaders reviewing a project or an engineer’s contribution should therefore ask what changed because the work happened. Did customers complete something more easily? Did the company earn or retain more revenue? Did a meaningful risk become less likely? Did another team become faster or more reliable? Did the organisation learn something that changed its direction? Did the system become easier to maintain? Was the outcome worth the full cost, including the complexity left behind?

These questions will not produce perfectly comparable numbers. They require context and judgment, which makes them less convenient than counting pull requests.

Convenience, however, is not the same as accuracy.

Software engineering is a profession built around reasoning under uncertainty. Companies trust engineers to make decisions involving architecture, security, reliability, cost, and customer experience. It would be strange to believe that the quality of this work can be understood primarily through the number of artefacts it produces.

Pull requests matter. Code matters. Delivery matters. None of them carries meaning in isolation.

The real measure is what became better, how much better it became, how confidently that improvement can be attributed to the work, and what the organisation had to spend or inherit in return.

Everything else is evidence waiting for context.