Scientists came up with a fake disease - and AI believed it was real

This case showed how easily a convincingly framed lie can enter the digital environment and begin to look like medical fact.

Details

Bixonimania appeared in 2024 as an experiment. Researchers published material online about a supposedly new eye disease. But everything was fictional: the disease itself, the authors, the institutions, the geography and the funding. Among the "sources" were knowingly absurd names like University of Fellowship of the Ring and Galactic Triad.

Despite these clues, large language models, including ChatGPT and Gemini, began to treat bixonimania as a real condition. As a result, the fictional disease began to look like a real medical problem - at least in the AI's responses.

Nature wrote about a similar case separately: bixonimania existed only in obviously fake academic texts, but AI systems still warned users about the fictional illness. After the publication of the article, two preprints about bixonimania were removed from the Preprints.org server.

The authors of the piece in The Conversation attribute this story not only to AI errors, but also to the human tendency to trust a confident pitch. To show this, they conducted an experiment in the style of the show The Traitors: four participants presented research, some of which was false, and the audience tried to identify the deception. In the end, listeners were more likely to be suspicious of the honest researchers, while the "cheaters" were considered more convincing.

Why it matters

The bixonimania story shows that medical misinformation can spread not only through social media or fraudulent websites. It can be amplified by AI tools too, if they fail to recognise fake sources and retell them in confident language.

The problem is that users often take chatbots' answers as already verified information. But AI can sound convincing even when it relies on non-existent research or incorrect data.

This is especially dangerous in the topic of health. If a person searches for an explanation of symptoms and receives a confident answer about a fictitious disease, they may become frightened, self-medicate, or make the wrong decision.

Background

The authors emphasise: misinformation has always existed, but what has changed is the speed of spread, the tools to create it and how believable it looks. Today's fakes can mimic scientific style, citations, authors and institutions - and then get into AI responses.

So access to technology alone is not enough. You need verification skills: looking at the source, the authors, the publication date, the reputation of the journal or platform, the presence of independent corroboration, and the absence of obvious oddities in the work itself.

Source

The fake disease that fooled the internet - and what it says about all of us is published by The Conversation and reprinted by VaccinesWork/Gavi. An additional case study of bixonimania is published in Nature on 7 April 2026.