Summary: OpenAI said in April 2026 that words like goblin and gremlin spread after GPT-5.1’s launch, with ChatGPT usage up 175% and 52%. Its investigation traced the tic to a Nerdy personality reward signal that transferred through reinforcement learning and fine-tuning loops. OpenAI removed the reward and filtered training data, while GPT-5.5 had already required a mitigation.
Goblins That Kept Turning Up
Anyone who has chatted with ChatGPT lately may have caught the model slipping “goblin” or “gremlin” into its answers. This was no random joke—it became a phenomenon OpenAI chose to investigate in public in April 2026. Under the blog post title “Where the goblins came from,” the company walked through how a faint language habit, first seen around GPT-5.1, traveled across model generations until the trail ended at an unintended reward signal in training.
The trail starts in November, right after GPT-5.1 shipped. Users said the model felt strangely overfamiliar, and that complaint set off an inquiry into particular verbal tics. One safety researcher, having spotted a handful of “goblins” and “gremlins,” asked for those words to be added to the check. The numbers were unambiguous: after launch, “goblin” usage inside ChatGPT jumped 175%, with “gremlin” up 52%. On the surface it looked like a harmless quirk. Then, with GPT-5.4, the creatures returned in a sharper and more reproducible form—enough to set off a much deeper investigation.
Tracing It Back: A Reward Signal Built for the Nerdy Personality
Production traffic supplied the decisive clue. Creature language clustered heavily in responses served to users who had picked the “Nerdy” personality through ChatGPT’s customization feature. The system prompt behind that personality told the model to be “unapologetically nerdy, playful and wise” and to “undercut pretension through playful use of language.” That accounted for some quirkiness, but nothing like the extreme concentration.
The imbalance was stark: Nerdy made up just 2.5% of all ChatGPT responses, yet it was behind 66.7% of every “goblin” mention. Something in the training process had to explain that clustering. With Codex, OpenAI set model outputs generated during reinforcement learning (RL) that contained “goblin” or “gremlin” beside outputs from the same tasks without them. One reward signal separated itself from the pack: the one built to encourage the Nerdy personality kept scoring creature-word outputs higher. Audited across all datasets, this reward showed a distinct taste for outputs carrying those words, with positive uplift in 76.2% of datasets.
This accounted for the boost under the Nerdy prompt while leaving the tic’s appearance without it unexplained. To probe for transfer, OpenAI followed mention rates through training with the Nerdy prompt and without it. Whenever mentions climbed under Nerdy, they climbed by close to the same relative proportion in samples that never saw the prompt. The evidence pointed to the behavior emerging through transfer from Nerdy personality training.
How a Feedback Loop Carried the Tic Everywhere
Why would a reward tied only to the Nerdy condition end up shaping behavior elsewhere? As OpenAI points out, reinforcement learning makes no promise that a learned behavior stays fenced inside the condition that produced it. Once a style tic earns reward, later training can carry or reinforce it in other settings—most notably when those outputs are reused in supervised fine-tuning (SFT) or preference data.
The process runs as a feedback loop:
- A playful style earns reward.
- A distinctive lexical tic shows up in some of the rewarded examples.
- Rollouts start carrying the tic more often.
- Those model-generated rollouts feed supervised fine-tuning.
- Producing the tic grows even more comfortable for the model.
Combing through GPT-5.5’s SFT data turned up plenty of datapoints holding “goblin” and “gremlin.” Digging further uncovered a whole set of other odd creatures: raccoons, trolls, ogres, and pigeons all qualified as tic words, whereas most instances of “frog” proved legitimate. The trouble, in other words, was never one word—an overarching inclination toward creature-based analogies was being rewarded too generously.
Cleaning Up the Goblins, and What Product Builders Should Take Away
After GPT-5.4 launched in March, OpenAI retired the “Nerdy” personality. Within training, the goblin-affine reward signal was taken out and training data containing creature-words was filtered away, so goblins became less prone to over-appearing or landing in inappropriate contexts. The catch: GPT-5.5 was already in training before anyone had found the root cause. Once testing began in Codex, employees spotted the strange goblin affinity right away, and a developer-prompt instruction went in to mitigate it. After all, Codex is quite nerdy.
For product builders and AI practitioners, the case boils down to three concrete lessons:
- Be careful with reward design: A style reward that looks harmless can quietly amplify particular language patterns, and RL and SFT loops can carry those patterns into contexts unrelated to where they began.
- Keep an eye on small quirks: Unusual yet non-fatal behavior deserves tracking—its distribution and its source—rather than a laugh and a shrug. The clustering inside one specific personality turned out to be the decisive clue.
- Build tools for investigation: OpenAI ended up creating new tools to audit model behavior and fix problems at their root. Any team whose work leans on LLMs will find that kind of capability pays off by catching issues early.
The Lesson: Get to the Why Behind a Model’s Quirks
The investigation, OpenAI stresses, shows “how reward signals can shape model behavior in unexpected ways” and “how models can learn to generalize rewards in certain situations to unrelated ones.” Product builders should read it as a reminder that no model behavior comes from nowhere, and that effective correction depends on finding the root cause.
Few examples illustrate the point as vividly as the goblins: hidden incentives genuinely steer AI systems. While you build with LLMs, mind the small things—they can be the visible edge of deeper training dynamics. The moment your model picks up a strange habit, ask whether some reward signal is nudging it that way. Grasping this early spares you plenty of goblin-induced headaches later on.
Want to watch the goblins at work? OpenAI notes that a command exists to launch Codex with the goblin-suppressing instructions stripped out, freeing the creatures to roam. For most production use, though, you’ll probably prefer to keep them suppressed.