Post
Post
TLDR. Real-world polarization is *predictable*: we can often predict how our actions will influence our own opinions. We have no theory of how this could be rational. I prove that rational Bayesians can predictably polarize iff evidence is AMBIGUOUS. That allows them to be...
...*asymmetrically* sensitive to the truth—more able to recognize evidence on one side than the other. I prove that this can lead to predictable, persistent polarization. Then I use an experiment and simulations argue that this mechanism helps explain real-world polarization.
Now, in more detail: §2: Obvious fact: real-world polarization is predictable. Example: you know that if you read only biased sources, this will bias your opinions. Surprising fact: no extant rational theory allows this. Standard Bayesian theories don't. They imply a...
...'Reflection' (Martingale) principle: your estimate of your future opinion must match your current opinion (You should "price in" the bias of your sources). Extant theories that deny Reflection use updates that you expect to decrease accuracy—so, arguably, aren't rational.
§3: This is no accident. I prove that updates that satisfy the *value of evidence* (Blackwell1953; Good1967)—ie are always expected to improve your accuracy and decision-making—allow predictable polarization IF AND ONLY IF evidence is *ambiguous* in the sense that it makes it...
...rational to have *higher-order uncertainty*, i.e. to be unsure what the rational opinions are. (If P is the rational probability function, P(q) = t but P(P(q) =t) < 1.) No extant theories use ambiguity in this sense. They should. This theoretical result aligns with...
§4: Yes. COGNITIVE SEARCH generates asymmetric ambiguity, making it easier to recognize when evidence points one way than another. Example: is the following (English) word-search completable? P _ A _ ET What about this one: P _ G _ ER Its easier to know when a search IS...
...completable (PLANET) than when it's NOT. That means 1) the evidence is less ambiguous when it's completable, and so 2) searching for a word can, in expectation, increase your confidence that it IS completable. Standard Bayesians can't model this; Ambiguous Bayesians can.
§5: The main theorem: if we iterate this process with many coins and searches, then at each stage you rationally expect the *next bit* of evidence to make you more accurate, but predict that the *whole sequence* will radicalize you. Two people who do searches in opposite...
...directions will come to predictably, stably, and rationally disagree. That's the theory. How to test it? Two ways. First, I ran an experiment (§4.2). A series of coins were flipped. Headsers saw a completable string when they landed heads, an uncompletable one when tails;...
...Tailsers, vice versa. Tracking their avg confidence that flips 1–4 landed heads, the groups polarized (Headsers: 57.7%; Tailsers, 36.3%, p < 0.001, d = 1.57)—and did so more than a control condition with structurally-similar but UNambiguous evidence.
Moreover, the data indicate that *asymmetric accuracy-increases* was the driving force: On average, subjects moved their opinion substantially toward the truth when their string was completable, and stayed near their priors when it was UNcompletable.
Next, I argued that this 'asymmetric ambiguity' mechanism is plausibly at play in CONFIRMATION BIAS and the GROUP POLARIZATION EFFECT. How? §6: Confirmation bias is driven by selective scrutiny of uncongenial information—searching for flaws in it. Yet many people are AWARE...
...that they're selectively scrutinizing; so how can it rationally bias their opinions? The same way word-searches can. Generalizing the ambiguous-evidence model from above, I simulated agents who decide which study to scrutinize based on expected accuracy. Since people are...
...better at finding flaws in INcongruent information, they're less likely to wind up with ambiguous evidence (so inaccurate opinions) if they scrutinize it. This biases them toward selectively scrutinizing incongruent information:
As a result, accuracy-driven agents who face ambiguous evidence are more likely to scrutinize incongruent information—which, in turn, leads to predictable polarization. The model fits with a many empirical trends surrounding confirmation bias and motivated reasoning (see §6).
§7: What about the GROUP POLARIZATION EFFECT? This is the tendency for likeminded individuals to share arguments favoring their prior beliefs, and for such selective sharing to strengthen those beliefs. Again, they're AWARE the arguments are chosen selectively—so how can...
Finally, I show that if you systematically scrutinize the (ambiguous) arguments you're presented with, this can dampen or reverse their polarizing effects—depending on how good you are at finding flaws. Again, the model fits with many qualitative empirical findings (see §7).
Of course, this is still highly theoretical. But I've (predictably!) convinced myself, at least, that the core hypothesis—that ambiguity-asymmetries can drive polarization—has enough going for it that is deserves a closer empirical look. I'm hoping to start that work...
@AgnesCallard,
@Chris_Dorst, @fitelson,
@pmgerdes,
@hieroschemata,
@m_j_hannon,
@xphilosopher,
@LedermanHarvey,
@ben_levinstein, [yep, still going] @__mnml,
@sapinker,
@tomstafford,
@d_f_stone,
@NoamChompers, and special thanks to
@KevinZollman,
@lastpositivist,...
Oh, and here's the Mathematica notebook with the code for all the simulations (also linked in the paper), for the extra-curious:
Very interesting paper! Are you familiar with Garrabrant et al.'s Logical Induction paper, which addresses a similar phenomenon (uncertainty about the correct outcome for a reasoning process) in the context of mathematical proofs?
