I’ve spent most of the last decade unable to answer the simplest question anyone asks: where’s home? I’ve lived in multiple countries, with a passport situation that makes border guards frown. Now I’m moving again, to The Hague, and it was house-hunting that finally made me see it. Every flat I scrolled, in a city I barely know, I already recognised: the same refinished parquet, matte-black taps, rewilded pot plant on the same windowsill. Same with the café near every place I’ve lived, flat white and exposed brick and oat milk assumed. I’m at home everywhere because everywhere is converging on the same few templates, and I’m exactly who those templates were optimised for. The recommender’s algorithm’s ideal user, basically.
Here is what puzzled me. We have never had more individual data and naive intuition says all that granular personal data should fragment us, splinter the world into a billion niches. And yet the opposite is happening. The more the machines learn about us as individuals, the more alike our worlds look. That paradox, more personalisation producing more sameness, is what this piece is about, and by the end I want to convince you this is a political economy story that needs and has an easy intervention.
Imagine forty options, cuisines, genres, shop types, and a standard algorithm: predict what people want, show them more of it, watch what they pick, update. Then let the loop run.
The mode gets shown more, so it gets picked more, so the model grows more confident the mode is what people want, and the tails starve. Within a few dozen rounds, a catalogue that began almost perfectly even collapses onto one dominant option. But why does it collapse toward the mode rather than fan out across all that individual data?
Almost every recommender is trained to minimise a prediction error, get the rating wrong by as little as possible, or maximise the chance you click. But the problem is that under squared-error loss, the prediction that minimises your expected error is the conditional mean. So an algorithm that is uncertain about you, and it is always at least a little uncertain, hedges toward the average. The more uncertain it is, the harder it pulls you toward the crowd. This is why more data doesn’t save us. Personalisation under a standard loss function is regression to the collective mean with extra steps. Variance, the technical word for the stuff that makes you you, is expensive to predict. Basically, diversity is variance, and optimisers are built to minimise variance.
What makes this political economy (and hence why I decided to write about it) is that the system doesn’t have to be right, it only has to be listened to. A recommender that’s a mediocre predictor of what you’d love in a rich, diverse world becomes an excellent predictor of what you’ll click in the impoverished one it creates.
This is also where I should say plainly that I didn’t invent any of this. Economists would call it performativity, the way a model, once acted on, bends the world into agreement (MacKenzie’s markets that reshaped themselves to fit their own equations). Machine-learning researchers have started calling a version of it model collapse: train a system on its own outputs for long enough and the variance drains away until everything converges on the mode. Urbanists have their own word, placelessness, for what’s left when everywhere optimises toward the same template. What I’m going to show here though is that these are one mechanism seen at four different scales. Think of it as temperature zero for culture: always take the most likely next token, and watch the surprise drain out of the sequence. Let’s make it concrete.
As a first test, I took the Greater London Authority's map of 640 high streets, mapped 18,000 London food places onto it, tagged each by cuisine, and flagged the chains, meaning any name recurring three or more times across the city (Subway, Pret, PizzaExpress, the M&S Food-to-Gos). Then I asked a question you can only ask with the whole city at once: how similar is any one high street to any other? For every pair I compared their cuisine mix, scored 0 for nothing shared, 1 for identical for all sixty thousand pairings.
Pick two London high streets at random and, on average, they already share almost half their food profile. For a city like London that sells itself on the variety of its neighbourhoods, that's a lot of similarity. But the average is just a snapshot, what drives it is even more interesting.
So, I sorted streets into quartiles by chain share. The streets with the fewest chains are the most distinctive; the other three quartiles all look noticeably more like the rest of London. Chains seem to act as a convergence layer: you don’t need a high street to be dominated by them. A relatively small number of repeated names is enough to make otherwise different streets resemble one another more.
And that’s not surprising since the chains are running exactly the loss-minimising loop from the last section. Gail's (my favourite UK middle class bakery/coffee shop) uses an AI site-selection system that learns from its existing bakeries, combining trading data with demographics, competition and location to forecast where the next one should work. Pret has used hundreds of data points, from historic sales to footfall, to decide where it opens. Waitrose has long run its own version with geodemographics, GIS and regression (raising my favourite question of what came first: the Waitrose or the gentrification?) Each of these is a model asking "where does it look most like the places that already worked?", which is to say: find the mode, and put another one there. Congratulations: your neighbourhood has been classified as Gail's-compatible, and is starting to look a bit more like all the others. I ran the numbers on my own future street in The Hague. Luckily, Pret and it’s highly mediocre coffee hasn’t yet decided to move the Netherlands yet.
My previous pub-closures work gets at the same loop from the other side. I trained a model to predict which pubs were most likely to close and, in doing so, effectively produced a profile of the kind of pub most likely to survive. The counter-intuitive bit is that many of the things that make a pub unusual, local or hard to categorise can also make it look risky. The features that make a place difficult to predict are often the same ones we later describe as its character.
The high street flattens in physical space. I assumed culture would too, so I checked, and the answer turned out to depend entirely on where you point the loop. I started with lyrics: every year-end Billboard Hot 100 hit from 1950 to 2015, just over four thousand songs, measured for how the words changed.
Within a song, the flattening is unmistakable. Vocabulary richness, distinct words relative to length, has fallen by more than a quarter since the early 1960s, while the share of repeated lines has climbed by nearly a third. The modern hit says less and says it more often, because the hook that works gets repeated. Btw, in the data AI music hasn’t even entered the scene here yet…
But zoom out to the chart and the loop reverses. Hits resemble one another less than they used to, the vocabulary across the chart as a whole has widened, and the biggest study of cross-national streaming charts finds music diversity across countries rising since 2017, not falling. I fully expected to find monoculture here, but it handed me the key to the whole argument. Prediction flattened the one dimension something was optimising on, the hook, the singable mode, and left every other dimension free. Nothing was minimising a similarity loss across songs, so nothing pulled them toward each other. Something was very much minimising a boredom loss within songs, so they collapsed inward. The flattening is a choice about which dimension we point the loss function at. Which means the real question is never whether prediction flattens the world, but which dimension we’ve aimed it at, and who did the aiming.
And on people, we’ve aimed it straight at consumption. A recommender shows you the handful of things it predicts you’ll like, you engage because they were selected to be engaging, it retrains on that, but it only ever learns from what it already showed you. Everything it didn’t surface decays out of consideration, untested. So, we have more data points than ever, but they’re a censored sample.
MovieTweetings shows the endpoint: nearly a million public film ratings since 2013, across roughly 38,000 films. If everyone watched a bit of everything, the line would run along the diagonal. Instead it shoots almost straight up. The top one percent of films, about 380 titles, pull nearly forty percent of all attention; half of everything anyone watched sits in the top two percent; thirty-eight films, a tenth of one percent of the catalogue, account for more than an eighth of all activity on their own.
That’s what “more choice than ever” looks like from the inside: a vast catalogue you can technically reach, and a near-vertical wall of sameness almost everyone actually climbs. We call ending up at the top of that wall “knowing your taste”, but it’s just the menu that narrowed.
The same problem showed up in an experiment of my own, one that happens to also be a small research industry right now. Everyone from survey shops to LLM labs is trying to build synthetic respondents: "personas" meant to behave like a population before you spend money asking actual humans. It's a genuinely useful idea. It also fails in exactly the way this whole essay is about, which is why I couldn't resist poking at it.
The obvious way to do it is to give each person some attributes and let a model decide what they think. But real populations aren’t collections of independent variables. Age predicts education, education interacts with income, values cluster with attitudes, friends influence friends. In other words, people come in lumps.
So I tried progressively harder to put the lumps back in. I started with a deliberately dumb generator that rolls each characteristic independently, then added latent psychological traits, recognisable archetypes, and eventually social networks in which similar people cluster and influence one another. I did the same on the response side: independent sampling through correlated, conditional and causal models.
A problem arose: the fancy models produced more elaborate stories about why people differed, but the population-level answers stayed remarkably smooth, on many questions spread almost evenly, close to maximum statistical diversity. In one comparison the dumb independent model was the best at matching the target distribution.
And it’s the same theorem as the recommender, just standing further back. A synthetic population trained to get the averages right buys that average-rightness the only way any optimiser can, by suppressing variance. The marginal comes out perfect and the structure, the clusters, the correlations, the strange combinations, slowly dies. Which is a genuine problem, because a population can have perfectly respectable averages while representing nobody in particular. Fifty percent left and fifty percent right describes a country split into two furious camps and a country full of mild centrists equally well. And this is now (occasionally and by some) being sold, with a straight face, to campaigns and market researchers as a substitute for asking actual voters. (I may or may not have argued exactly this on a panel in London a few weeks ago.)
The obvious objection is: so what? If people like Gail’s, Spotify gives me music I enjoy (I’ll admit that I recently moved back to Spotify after getting way too frustrated with the algorithm of Apple Music, Apple, seriously, step this up), and Netflix saves me forty minutes of scrolling, maybe predictability is just convenience.
But variety is where new options come from. The strange Georgian-Japanese street-food fusion restaurant nobody can classify, the pub that doesn’t fit the template, the song that sounds unlike what came before, most will never matter. But some will. A world that keeps removing the unlikely also removes the things that could have become the next likely. Ecologists have known this for a long time already and even priced it: they call it the insurance value of biodiversity, the standing reserve of rare species that cost energy and contribute nothing, until the climate shifts and one of them turns out to be what survives. A monoculture is efficient right up until the weather changes.
And there’s a second cost, harder to see, and it’s the one that actually a bit scared of. Once prediction shapes the choices in front of us, we lose the ability to tell the difference between what people wanted and what the system made easy to want. If an algorithm feeds me Scandinavian crime dramas for ten years because I once watched two, my viewing history becomes airtight evidence that I love Scandinavian crime dramas. Admittedly, I have watched two Scandinavian crime dramas in my life. Netflix has since decided this is my entire personality. We risk mistaking a world shaped by prediction for a world shaped by preference, and the more these systems decide what survives, the more power moves from the people choosing among options to the systems deciding which options get a chance to be chosen.
Here is the good news, and it’s the most hopeful finding I have: the fix is known, it’s old, and it’s cheap, it just requires a small change. The whole emerging political economy problem is a version of the single most studied trade-off in computer science, exploration versus exploitation. Any system learning from feedback has to divide its effort between exploiting what already scores well and exploring options it hasn’t tried, in case they’re better. This is the multi-armed bandit problem (yes, I actually paid attention in my undergrad minor), and the maths is unambiguous: an agent that only exploits gets trapped on the first decent option it finds and never discovers the best one. Pure exploitation is a known failure mode.
So the entire fight is forcing exploration back in, and when I simulate it, the result is actually encouraging. The only thing I vary is how much exploration I force, and diversity goes from near-total collapse to fully preserved. But, and this is the finding regulators need, it’s a threshold. Two or five percent, roughly the level of a token “discovery” tab, does almost nothing. You have to clear a cliff, around a fifth of the entire system’s attention, before variety survives at all (just as in the nature).
Which means this cannot be left to voluntary good behaviour, because no engagement-maximising firm buys a fifth of its own attention out of principle, and the maths guarantees that whoever does spend it will look, in the short term, like they’re leaving money on the table. This is a textbook case for regulation (aka failed markets), and the encouraging thing for those of us watching from this side of the North Sea is that the scaffolding already exists on both sides of it. The EU’s Digital Services Act already forces very large platforms to offer at least one recommender feed not based on profiling, and to open their systems to vetted researchers, which is precisely the audit access you’d need to measure whether a feed clears the exploration threshold.
The UK, post-Brexit, is writing its own version and could go either way. The Online Safety Act and the new Digital Markets, Competition and Consumers Act hand Ofcom and the CMA real leverage over how the largest platforms rank and recommend, but so far the framing is almost entirely about harm and competition, not diversity. That’s a gap, and a chance to lead: Britain has form here, the New Economics Foundation’s “Clone Town Britain” work named high-street homogenisation two decades ago. Interoperability is the other lever the CMA in particular could pull, because the deepest discipline on a feed that has trapped you is the credible ability to leave it and take your data with you. And at the level of the high street, friction turns out to be a feature: if a handful of chains does that much homogenising work, then protecting the independents around them, business rates relief, planning powers over frontages is the easy fix.
At the level of the self, sure, force your own exploration, the book from the shop that might close, the restaurant with four reviews instead of four thousand. But I want to be honest that this is the weakest lever, and I’m suspicious of every version of this argument that ends there, because “just be a more adventurous consumer” is exactly the conclusion that lets the systems off the hook. The exploration has to be mandated at the level of the machine, because that’s where real change can happen.
Going back to the café, and the question I still can’t answer. I used to think my problem was having no home, too many countries, no single belonging. The truth is though that the world is building me a home everywhere, frictionless and familiar and mine, and I should be more frightened of how good it feels than I am. Belonging everywhere and belonging nowhere turn out to be the same condition, and a model that can manufacture the first guarantees the second. The home I can find in any city is basically just a prediction about me that came true. I’d rather, I think, be somewhere that surprises me, even if it means being, for once, a stranger.
Thank you for reading! I have some cool, new ideas I’m working on. If you want to keep my Substack alive, buy me a coffee and/or subscribe:)