Text as a “Market for Lemons”

I would like to write a bit about text. That is: technical writing, legal briefs, or even an opinion piece such as this note. Such writings make up much of our society and form a “marketplace of ideas.”

Texts are now very cheap to produce using large language models (LLMs). Some simulated texts remain correct and useful, and some contain numerous subtle flaws and fabrications. In my opinion it remains expensive to reliably determine which text is which type, as LLMs are not as good at detection as fabrication.

Only the text’s author (if there even is one) has a chance of knowing the provenance of the material. A text author (or “seller”) knows a lot about how they wrote the work and what they directly confirmed versus what they are relying on sources for. A text reader (or “buyer”) can’t always easily judge the quality of novel texts. In fact previously useful signifiers such as elaborate formatting and typesetting are now uninformative. Proper editing is a difficult and specialized skill, and fact checking an untrusted text can be expensive.

Our marketplace of ideas now has a huge information asymmetry between sellers and buyers, strongly favoring the sellers.

Akerlof described markets with such asymmetries as being “markets for lemons.” “Lemon” being a United Stages slang for a bad or defective car that has numerous unfound problems. See: Akerlof, George A. “The Market for ‘Lemons’: Quality Uncertainty and the Market Mechanism” The Quarterly Journal of Economics, vol. 84, no. 3, 1970, pp. 488–500. JSTOR, https://doi.org/10.2307/1879431.

In our example buyers can only afford to pay the expected value of a text minus the cost to vet or correct the text. As more simulated texts flood the market, this expected value plummets. Bad sellers can easily insert many cheap LLM “lemon” simulated texts into the market. This gives us a variation of Gresham’s law as “bad text drives out good.” Finally price discovery fails for any price above zero, and the market collapses. This happens regardless if “price” is dollars, euros, exposure, visibility, marketing, good will, or even just praise.

The collapse is already visible in scientific conferences. Some amusing examples include:

• 9th Research Integrity Conference hit with AI-generated abstracts: link.
• Major AI conference flooded with peer reviews written fully by AI: link.
• Scientists reportedly hiding AI text prompts in academic papers to receive positive peer reviews: link.

Without external mechanisms (such as trust, shame, certifications, guarantees, refunds, or insurance) the marketplace of ideas collapses. I believe individual readers will have to become much more active in maintaining a list of who they trust work from, and become unfairly vigilant in ignoring sources without a known good reputation. This has its own downsides.

Categories: Opinion

Tagged as: ai LLM Market for Lemons