Stochastic Parrot

What does stochastic parrot mean?

"Stochastic" means driven by probability. A parrot can repeat sounds without understanding them.

Put the two together, and you get the idea behind stochastic parrot: a language model predicts likely next words and can produce remarkably fluent text without necessarily having the kind of understanding that the text seems to suggest.

People use the phrase to push back on claims that chatbots genuinely "understand" or "think." Whether the metaphor fairly describes modern language models is still debated.

How people use it

Every time it sounds insightful I remind myself it's a stochastic parrot. Then it says something clever and I stop being sure.

Example written for this dictionary.

You'll see the phrase used as a serious argument, a dismissive jab and, sometimes, an ironic badge. Skeptics use it to urge caution about reading too much into fluent language. Some AI researchers and enthusiasts argue that modern models do more than simply repeat patterns, while others use the phrase jokingly even when they don't fully agree with it.

Why it matters

The phrase points to one of the biggest questions in AI: what, if anything, counts as understanding in a language model?

Researchers disagree over how far prediction and pattern learning can produce abilities that resemble reasoning or understanding. The parrot metaphor gives that disagreement a memorable name.

Where it came from

The phrase comes from the 2021 paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell, credited in the paper as "Shmargaret Shmitchell."

It was presented at the FAccT conference. The paper itself covered much more than the metaphor, including the environmental costs of large models, biases in training data and the risks that come with increasing scale.

Source & further reading

Bender et al.: On the Dangers of Stochastic Parrots (FAccT 2021)