In a recent Note, I discussed the problem with LLM/LRM hallucinations. They are not only a generalizing error from training data but a "plausible sounding but false answer." That was a Note; let me expand it here.
From Generalization to Hallucination
"Generalization" is a must-know concept for all this machine learning meets neural networks meets LLM guff, because all ML systems infer by generalizing beyond their training data. Here's a simple example using a task known in information extraction/natural language processing as "Named Entity Recognition" (NER). We want training data pairs like:
dog NOUN
Fido PROPER_NOUN
walk VERB
quickly ADVERB
... and so on.
We feed these token-label pairs (in a .csv file or a structured dataset) to the training algorithm, and it learns a model—a high-dimensional function—that estimates the probability distribution over possible labels for any given token. This function can then be used to make predictions on previously unseen text. Generalization happens when the model guesses a label for a word it never saw in training. If it's accurate often enough, great—we say it generalizes well.
Model generalization has been studied extensively in statistical learning theory—think VC dimension, bias-variance tradeoff, PAC learning—and we’ve got decades of methods to test and benchmark how well a system is expected to do on unseen data drawn from the same distribution.
So, so good. But when we get to LLMs/LRMs we have a catch.
Keep reading with a 7-day free trial
Subscribe to Colligo to keep reading this post and get 7 days of free access to the full post archives.