It’s not hard to find someone who hates bureaucracy, indeed it actually represents the vast majority of people. They aren’t wrong to be mad either, the AMA cites it as one of the major causes of physician burnout, and the Harvard Business Review finds that the US economy alone loses approximately $3 trillion (or 17%) of its GDP due to excessive bureaucracy. Yet the alternative is anarchy, and we know where that path leads. For the complex systems we rely on to work, we need shared standards and repeatable processes.
Given that we can’t eliminate it, its important that we make it effective, and to do that we need to diagnose where the machine breaks down.
To understand bureaucracy we have to look at its history, and perhaps the most important figure is German sociologist Max Weber, who at the time was watching the world undergo a transformation from feudal loyalties and hereditary privileges to one that he called “rationalization”, the systematic organization of human life around calculable rules and technical efficiency.
Given that Weber was coming from a world in which bloodlines largely dictated human life, it was understandable he was excited about the prospect of a more rational world, indeed he saw bureaucracy not as a source of stagnation but as a crowning administrative breakthrough, so much so he framed it as the purest expression of rational-legal authority. According to him, this process was as important as the industrial revolution, since a formal bureaucracy was supposed to be technically superior to any previous form of administration in speed, precision and predictability.
In his ideal model, the process was designed to eliminate human vagaries through five core principles:
Jurisdictional hierarchy: Clearly defined authority where lower offices are directly supervised by higher ones.
Documentation: Continuous archival record keeping to preserve institutional memory and ensure decisions are governed by precedent.
Credentialism: Officials selected and promoted based on evidence-based technical competence rather than patronage or social status.
Specialization: A division of labor where each role executes a clearly defined mandate.
Radical Impersonality: Operations carried out without anger or passion, treating every case according to uniform rules regardless of who is asking.
These core principles all function under the idea that ground-level data goes in, experts codify the logic to turn that data into information and action, and the most rational, objective outcome would emerge. In practice, I think we can all agree that no modern bureaucracy functions that way, whether it be public or private sector.
Policy is rarely made with an open-minded review of empirical data. Instead they are often kneejerk reactions to political scandals or public panics. When leadership is pressured to “do something”, new rules are drafted based on whatever priors, ideological biases, or risk-averse instincts the decision-makers already hold (or what they imagine their superiors to hold).
When data is presented, it’s often after the decision has been made as a reverse-engineered artifact, which only serves as defensive camouflage. This isn’t speculation either, Sociologists John Meyer and Brian Rowan demonstrated that organisations adopt formal rubrics, oversight committees and audit frameworks not because they improve operational output but as “myth and ceremony”. A lot of these procedures are decoupled from the day-to-day minutiae, and as such they cease to be engines of rationalization and instead just exist to project legitimacy and manufacture the appearance of scientific rigor while shielding leadership from blame. Once this happens, the process transforms from rationalization into something more sinister, the use of power over others. This quote from The Road to Serfdom hangs heavy in my mind:
Michael Crozier in The Bureaucratic Phenomenon observed that bureaucratic hierarchy is actually a contest over discretion. Organisations and administrators expand their institutional leverage by standardizing, monitoring and constraining the behaviour of other groups, while guarding their own autonomy. Power in these organisations doesn’t necessarily take the form of overt tyranny, it exists by having the structural authority to force others to absorb procedural friction. This makes the consequences of bureaucracy asymmetric in the following ways:
Administrators face much lower costs than the subjects: A compliance officer can draft a memo requiring two new approval layers and a weekly tracking sheet in 20 minutes.
The subject absorbs all the friction: The hundreds of people subject to that memo lose thousands of cumulative hours navigating the newly manufactured bottleneck.
In his book The Utopia of Rules, David Graeber argues that administrative overhead acts as a mechanism of coercive subordination. Forcing workers to spend hours filling out forms and begging for sign-offs takes away their autonomy, reinforcing who sits at the top of the food chain.
TPS Reports in Officespace
Given enough time, this dynamic produces what Robert Merton calls “goal displacement”. This is where the rule book has overridden the original function of the organization and becomes the primary mission. In a truly rational system, discovery of a rule harming operations would lead to a swift update, either a modification or removal of the rule. However when rules are monuments to institutional authority to enforce obedience, questioning them is not welcomed as feedback, it is penalized as insubordination. Workers quickly realise that their survival has little to do with their stated output and everything to do with their compliance.
It isn’t hard to see why people would develop such a disdain for these processes when they devolve this way.
An unfortunate side effect that most policy makers tend to not realise is the assumption that bureaucracy scales linearly. For example a naive administrator would think if a project with five people requires two hours of administrative overhead per week, doubling the team to ten people will require four hours, and adding one extra compliance reviewer will introduce a modest delay. This is far from the truth, to understand this we need to look at how these processes become non-linear.
In 1937 management theorist Vytautas Andrius Graicunas showed that organizational complexity doesn’t scale with head count, but with the network of interpersonal relationships this creates. While a manager might naively think they are just adding one or two people to a group, this is actually exploding the communications combinatorially. This is also echo’d in the book The Mythical Man-Month where the author formulated Brooks’s Law:
adding manpower to a late software project makes it later.
In general this overhead actually scales approximately as:
Where n represents the number of people in the organisation. A team of four has six interpersonal channels to navigate, a team of twelve has 66, a team of 30 has 435.
When everyone is required to be kept in the loop, the energy required to coordinate, schedule meetings, and achieve consensus quickly outstrips the energy required to complete the task.
Political scientist George Tsebelis developed the idea of Veto Player Theory in 2002, which proves that as you increase the number of actors whose consent is required before an action is taken, the winset of possible changes shrinks dramatically. This is buttressed by analyzing why government is slow, in what Ezra Klein calls a Vetocracy.
Bureaucracies often evolve to become dense lattices of veto players, a single new initiative must run a gauntlet of independent checkpoints such as legal, IT, comms, finance, HR and risk. If an initiative requires approval from these six different departments, and each department has a 90% probability of approving these requests, the compounded probability of this passing is not 90%, it drops all the way down to 53% (0.96). If you double the review gates to twelve, this falls to just 28%.
If each veto actor has an unpriced veto, then you end up with a system that is mathematically attracted towards the status quo.
In 1961 John Kingman made a paper called “The single server queue in heavy traffic” and proved that as resource utilization approaches 100%, the queue does not increase gradually, it approaches infinite.
If we look at how organisations operate today, compliance reviewers are almost always running at near total capacity. If someone submits a routine document for a sign-off, the actual time to review and sign the document might be 15 minutes, while the time it sits idle in the queue may often be two weeks or more. Lets say this procedure requires another four or five sequential reviews, you end up with this delay on each step, so a task that takes only two hours of work actually ends up taking three months of calendar time.
Ironically, from the inside everyone seems exhaustively busy, yet from a work perspective almost nothing moves.
High value work requires sustained blocks of deep concentration. This is somewhat unfortunate, because bureaucracy treats human attention as an infinite, perfectly fungible commodity that can be sliced into 15 minute increments. When an organisation becomes focused on its own compliance, it introduces a large amount of low grade interruptions, and while the formal cost for each interruption is trivial, maybe five or ten minutes, the actual cognitive cost is catastrophic.
The driver of this cognitive destruction is what Sophie Leroy calls “Attention Residue”. When a worker transitions from Task A (high value cognition) to Task B (bureaucratic compliance), the brain does not make a clean, instantaneous context switch. A portion of cognitive bandwidth remains stuck processing Task B, even after returning to Task A. In their paper “The effect of regulatory focus on attention residue and performance during interruptions” Sophie Leroy and Adam Schmidt argue that when people switch between tasks before one is fully resolved, their executive function, memory retrieval, and analytical performance on subsequent work degrades significantly.
In addition to that tax, it takes a while to even begin to switch back to Task A. In a 2008 paper entitled “The Cost of Interrupted Work: More Speed and Stress” Gloria Mark et al found that after being interrupted it takes a worker an average of 23 minutes and 15 seconds to return to their original focus, so when a worker is interrupted three or four times a day by compliance, they are forced to spend hours merely rebuilding their train of thought. These researchers also found that to compensate for these interruptions, workers attempted to work faster, leading to higher levels of mental exhaustion. Where a bureaucrat sees a quick check in, a worker experiences a fractured day where sustained work becomes impossible.
Human memory has biological capacity limits, in cognitive load theory mental effort is divided into intrinsic loads (the mental effort required to solve a problem) and extraneous loads (the mental effort demanded by the environment). Bureaucracy floods the brain with extraneous load, it acts as a background hum which crowds out the mental space available for genuine problem solving. The cost to this in organisational terms comes down to degraded cognition.
Despite all the downsides I have mentioned, the biggest cost of bureaucracy is never the money or time it wastes, these are verifiable and easy to audit. The true damage lies in what it never allowed to appear in the counterfactual sense. These systems are optimised to manage and measure what is observable, however some things are not as easily measurable, such as people’s ideas that they may have let go of simply due to the demotivation demanded by bureaucracy.
In an old essay by French economist Frederic Bastiat “That Which is Seen, and That Which is Not Seen” he argues that bad economic policy stems from focusing on the immediate and visible effects while ignoring unseen consequences. In bureaucracy, this can be simulated as a regulatory agency blocking a project or adding severe friction through years of mandatory review. The effect is visible, safety, order and the status quo are preserved, indeed in a world where no action is taken, and no downside-risk eventuates from the lack of action, then the process appears to have worked. But what it destroyed was the alternative universe it wiped out, such as the clean energy grid that was never built, the high speed train that never came to exist, or the software product that was never developed, these are all upside-risks. Unfortunately counterfactuals like this leave no paper trail, and therefore allows the bureaucrat to claim credit for the hypothetical risks they averted, while bearing zero accountability for the future it destroyed.
To find the cost this counterfactual destruction has on society, we can look towards economist Sam Peltzman’s paper “An Evaluation of Consumer Protection Legislation: The 1962 Drug Amendments” where he takes a look at the FDA’s 1962 Kefauver-Harris Amendments in the wake of the Thalidomide crisis. These amendments mandated that pharmaceutical companies prove not only safety, but exhaustive empirical efficacy through extensive clinical trials before bringing a drug to market. Visibly this was a triumph, ineffective drugs and high profile toxicity failures were kept off the shelves for consumers. But Peltzman analysed the counterfactual ledger, the exponential increase in clinical costs and multi-year regulatory review times triggered a collapse in the development of new drugs, this was known as “the drug lag”. Due to this, thousands of patients quietly suffered and died from treatable conditions while life-saving drugs were stalled in process or abandoned entirely due to the capital needed to survive the multi-year regulatory gauntlet. The victims of these delays were more numerous than the victims of potentially bad drugs, but because their deaths existed in the counterfactual, the institution bore zero culpability.
It does get worse, as the chilling effect of bureaucracy on these projects is even harder to measure. When the procedural barrier to entry is too high, and it often is for private industry on purpose (see regulatory capture), the cost of the bureaucracy acts as a regressive filter that weeds out unconventional, high-variance ideas in favour of safe incrementalism. The institution never needs to reject these ideas, they just never show up, the gravity of the procedure kills them at conception.
In the long run an organisation has to be able to adapt to the new regimes it finds itself in, whether political or in the market, and to do that it has to capture asymmetric upside by taking smart, non-linear bets and harvesting the payoffs of high-variance ideas. It is then unfortunate that bureaucracy stifles this, and is fundamentally at odds with the long-term viability of the organisations it serves while being good for the short-term viability.
To illustrate this, we can look at Daniel Kahneman and Dan Lovallo’s paper “Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking” which demonstrates organisations punish errors of commission while ignoring errors of omission. An error of commission is visible, an administrator might approve a novel software architecture, and if the initiative stumbles the failure is visible, auditable, and tied to their name. While an error of omission is invisible, an administrator might reject the initiative due to an abundance of caution, demand more rounds of feasibility reviews allowing the breakthrough to quietly die. The organisation suffers, but there is no auditable trail leading back to the administrator, in fact they may be rewarded for their diligence in protecting the organisation from downside-risk. If the penalty for taking action and stumbling is high, but the penalty for taking no action is zero, the only rational choice for a bureaucrat is total inertia.
As Gerd Gigerenzer reveals in his book “Risk Savvy: How to Make Good Decisions” this leads to defensive decision making. Managers routinely pick the safe option because it is the easiest to defend when someone audits it, leading to the old adage:
nobody ever got fired for buying IBM
This attitude represents a breakdown in the principal-agent relationship between the organisation and the managers within it where the ownership of the resource is separated from the control of the resource, and when those interests diverge, the agent (in this case the managers or administrators) will maximise their own personal utility rather than the welfare of the principal. The individuals making these procedures rarely share in the upside or downside of the operational consequences of these procedures. To quote OfficeSpace again “That’ll only make someone work just hard enough to not get fired”:
The famous mathematician Nassim Nicholas Taleb calls this asymmetry a lack of Skin in the Game. In healthy systems decision makers are exposed to personal downside when their judgements fail and meaningful upside when their initiatives succeed. Bureaucratic systems break this relationship by providing the bureaucrat no downside exposure to stagnation, and no upside capture from breakthrough success. What they essentially hold is a free put option, imposing the costs of delays and missed opportunities onto the workers while insulating themselves from failure, and they are incentivised to take it. Because most transformative breakthroughs happen in the fat right tail of the distribution, the one that they suppress, an organisation becomes functionally fragile with what seems to be stability on the surface yet structurally incapable of seizing the future.
Despite all we have said against bureaucracy in this article so far, it does have its place, and we would be nowhere without it, indeed the civilisation you enjoy right now could not exist without it, which is why this line of inquiry is so exciting. Although we practice bureaucracy, we do it in such a blind fashion, that to really understand its failure points means we can make it so much more effective and unlock so much more out of our economies.
We can come close to the Weberian ideal by making some small structural changes, and accepting new technologies to greatly reduce the innate friction it imposes on organisations. While there are many counterweights, lets discuss four common ones that have been deployed with success in the past.
Organisations suffer from a “ratchet effect” known as status quo bias, where adding a rule carries zero cost, while repealing one takes immense effort.
To make this more Weberian, we need to consistently demonstrate the effectiveness of this rule, and the only way to do that is with data and studies. These rules ought to be justified periodically so the people practicing them can have faith that they are still a good use of their time. The burden of this proof should fall on the regulator, and without proper study this rule should automatically dissolve. This shifts some cost of maintenance back onto the regulator.
In incredibly burdensome regimes, it may even be appropriate to do a one-in two-out approach. This enforces a cost to the regulator that prevents unchecked expansion. In 2015 the paper “Cutting Red Tape in Canada: A Regulatory Reform Model for the United States?” the author shows that the Canadian province of British Columbia tried this approach and went from economic growth 1.9 percent below the Canadian average to 1.1 percent above after the changes went into effect.
If you are labouring under bureaucracy, it must be nice to know that you can transparently see the data backed studies informing the process, and have the knowledge that there are costs involved with its process that are paid by the people imposing it, not just you, the one following it.
Often the apparatus making the bureaucratic processes are removed from the people enforcing it, imagine a committee making rules and a customer service representative enforcing them. This removes the ability for the organisation to make optimal ground level decisions and respond to unforeseen circumstances. In these cases the correct architecture is a subsidiary one, where authority and resources can be delegated to the perimeter where the information is the freshest.
A good example of this is the Ritz-Carlton model where frontline staff are given the unilateral authority to spend up to $2,000 to resolve an issue for a guest without managerial sign-off.
In Veto Player Theory (discussed earlier), it was shown that multiple veto checkpoints have a probability of innovation that trends towards zero, and this is largely due to unpriced vetoes.
To remove this, you must put a price on those vetoes, the first is one of labour, where you cannot allow vagaries around risks to block a proposal. German law has a concept known as a constructive vote of no confidence, where an objection is only valid if the dissenter formally proposes, sponsors and resources an alternative path that achieves the goals in a similar time frame. In this way you make a veto costly to enforce, non-blocking and constructive.
Secondly, you can apply the costs associated with halting a project to the department responsible for the blocking, and make sure it ends up as part of an audit and not as an invisible cost.
To eliminate the principal-agent distortion created by bureaucracy, a number of techniques can be used:
Replace consensus seeking committees with owners who are empowered to execute over a domain and who bear accountability for its safety and compliance as well as retain upside if it works.
Any administrator wishing to impose a new process upon a system needs to run a pilot on it and get honest anonymous feedback from those it affects as part of an implementation process.
Auditing the counterfactuals is a must to learn of the true cost of these systems, and the administrators must be responsible for the counterfactual losses they incur.
All these fixes have talked about imposing costs to maintain accountability in bureaucratic systems, however there is a new technology on the block which could go a long way towards reducing the costs these systems operate under, even fundamentally.
We previously discussed how queue theory can cause large buildups within organisations due to the resource asymmetry between bureaucratic bottlenecks and the people using them. If AI is used to do these reviews, and can maintain a level that is comparable to human output, then you reduce the queue time from weeks to the compute time the AI needs to solve the review problem, which should be far less and is horizontally scalable.
We also discussed the attention tax, and AI may be able to complete a lot of these bureaucratic tasks on a workers behalf, enabling them to continue their focus unbroken saving huge amounts of time between switching tasks, and getting a better quality output.
Finally if a lot of bureaucracy is down to having the correct information, and we use it to perform work (as a lot of software engineers do), then perhaps we can greatly reduce the bureaucracy while providing the same amount of information the administrators may need to create these rules in the first place. In fact this may allow a solution to Hayek’s famous knowledge problem without standardisation, and standardisation is why a lot of these processes exist in the first place.
Ultimately, if we can even make a small dent in the problem that is bureaucracy, if it currently sucks 17% out of our GDP, then we could realise huge amounts of productivity and savings, which is definitely something to be happy about!