260914: Generation-Verification Asymmetry
Collin Lysford was the first person I talked to who stressed the asymmetry whereby LLMs collapse the cost of generating context, text, information—but the price of verification remains high. (And grows proportionally higher and higher.) So I will give him credit for the following speculations.
In a GPUWorld, this asymmetry becomes a driving information dynamic. Content creation is free; content curation (broadly construed, to include e.g. fact-checking) is expensive. "Human attention is the most important resource now," as Sam Shillace writes. (Ditto for legitimacy, trust, presence, and I guess land.) Curation winnows the wheat from chaff of generated content, so that readers' attention is maximally well-spent.
Tastemaker authorities, sorting through it all, should gain status. Reputation is critical & aggregates around skilled verifiers. Already, online info-overload has given us human curation like Guzey's Twitter Roundup newsletter, or the Yams music recommendation service. But obviously they have been outcompeted at-scale by algorithms. Everyone complains about algorithms, but they are good enough and cheap enough that the vast majority of online consumption is mediated by their sort.
At least for recommendations. Consecration still appears to be a human game. Yelp reviews may be volume-based, but Michelin Stars will probably always involve tastemakers. There are separate systems of popularity and prestige. And prestige is about conferring reputation.
Reputation involves making unverifiable claims expensive to the claimant. How? By imposing severe penalties whenever a claim doesn't hold up. Romantic partners can be discredited by a single discovered lie, with the reasoning that discovering lies is somewhat difficult, so that a single discovery implies the possibility for more systematic deceit. LLMs lack this authority—we would not make major medical decisions on their recommendation—but instead offer something like creativity, in the form of being able to turn up unreliable new leads. Similarly, we don't trust an artist the way we trust a neurosurgeon, because part of being a great artist involves missing sometimes, taking risks, experimenting. But at least a human artist is still staking his reputation with every decision he makes.
Claude says:
The forecast I'd actually stake something on: algorithmic curation degrades relative to staked human curation over the next several years, and not because the algorithms get worse. Ranking systems won a war fought under scarcity of supply. Production cost was itself doing enormous work as a quality prior — somebody spent a week on this, so it's probably not garbage. Remove production cost and you remove the prior, and every engagement-optimized ranker becomes saturable by unbounded cheap content optimized against it. Goodhart at a volume the mechanism was never designed for. The early signal is already visible in people appending "reddit" to searches, in the migration into closed group chats, in paid newsletters — all of which are moves toward provenance, which is the thing ranking can't supply and staking can.
Apprenticeships consist of junior team members performing cheap generation, under supervision of a trained senior role who can verify and provide feedback. If you get rid of junior roles, because they can largely be automated, you save money in the short-term (GPUs are cheaper than entry-level humans) but in twenty years, you haven't trained any new senior-level employees to perform the verification layer.
And compare this dynamic with Erik Hoel, "Culture Becomes A Dark Forest":
Al looks more like slash-and-burn agriculture, where you purposefully reduce the existing natural growth to ashes, grow a couple years of crops on it, and then move on to the next plot.