Survey & Range

290913: LLMs As Search Aggregators

LLMs as a search aggregator: your prompt is your query. What's special is that you get true semantic search, you're matching vector embeddings. See also LLMs as solving certain search-based coordination (matching) problems.

Traditional keyword search demands you already speak the indexed community's idiom. LLMs can exchange in and out of diverse denominations: they translate between idiolects.

Traditional keyword search matches you to specific, particular instances. LLMs turn up generic aggregates of many instances.

Traditional keyword search ranks particulars by popularity; LLMs' equivalent of PageRank is volume in training corpus.

Retrieval-Augmented Generation attempts to minimize the weaknesses of LLM search by bolting on a traditional keyword search, resulting in a Frankenstein hybrid. (As well as keeping them up to date with the world post-training.)

LLMs, by lessening coordination friction (i.e. Coase's transaction cost), weaken the stranglehold of the Firm over the modern world.

Perhaps, in a post-firm world, the household again becomes the organizing unit of economic life, as Jefferson dreamed in the Age of Farms.

#coordination #matching problem #retrieval-augmented generation #semantic search #the great unbundling #thomas jefferson