nahiko mikasa · · 3 min
You stopped learning new words
and you didn't even notice
What does GPT mean?
Don't google it.
...
Generative Pre-trained Transformer.
You typed "GPT" a thousand times in the last 3 years.
You never asked yourself what it means.
Even worse, your brain tricked you into thinking you know what it means.
You are lying to yourself
This has a name: "good-enough" processing.
You read, you meet a new word, and your brain whispers "this must be something like X". It sounds plausible, so you keep going.
You don't decide to skip the word. There is no decision. You are being played, by yourself.
To be fair, stopping at every uncertainty would make reading a never-ending task. Your brain is optimizing so you can finish that chapter tonight.
But there's more.
Your knowledge decreases
When you meet an unknown word, the probability that you learn it is somewhere between 5 and 10%.
It gets worse.
Batia Laufer documented what she calls deceptive transparency: words that look decomposable get confidently mis-guessed, and you never notice.
You're not skipping the word. You're learning it wrong.
You tricked yourself into thinking you knew what GPT means.
(And a few lines ago, you tricked yourself into thinking you know what a transformer is. Didn't you?)
The gap keeps widening
Hu and Nation measured it in 2000: guessing from context only works when you already know about 98% of the words around it.
So the gap between the experts and us normies keeps widening (my agent tells me this is called the Matthew effect).
Now apply that to AI. Every week you see hundreds of new words.
Every word you mis-guess lowers your understanding of the next one.
You are either compounding up or, more probably, compounding down. And you don't even notice.
Audit yourself
Here are five words you've probably read this week.
For each one, what you probably think it means, next to what it means:
| Word | What you probably think it is | What it actually is |
|---|---|---|
| reasoning | a special mode in which the model works things out | The model talking to itself before it talks to you. |
| temperature | a creativity slider | How much randomness gets added when it picks the next word. Turn it to zero and it says the same thing forever. |
| quantized | compressed, like a zip file | The model's weights are stored with less precision (3.2935325235542... becomes 3.2935). You lose a little accuracy and get a lot of memory back. |
| distilled | compressed, like quantized? | A different model entirely, trained on the big one's answers, never on the original data. |
| attention | what the model is "focusing" on | Every word scoring how relevant every other word is to it. It's the T in GPT, and it gets its own post. |
If you scored 5/5, congrats.
Brains are still super cool
To be fair, your brain's guess is close enough most of the time.
But no one ever double checks. It just slides by.
You couldn't name the words you guessed even if you tried.
Because you are sure you know them.
That's what prompted this article.
It wasn't discipline. It never is.
I decided I will not let my brain stop me from learning anymore.
Next week I'll show you how.
See you soon,
nahiko