NewsLies

Arabic Fake News & Source Leakage Demo

Model: AraBERT v0.2
⚠️ Research Warning: The Source Leakage Illusion
Our research proved that models trained on the AFND dataset do not learn to fact-check. Instead, they memorize publisher writing styles. This demo model scores 70% on random data, but collapses to 37% when tested on unseen publishers. Try it yourself: Notice how the model focuses on source names and boilerplate rather than factual claims.
Arabic News Text
Loading model…
Result
About this Research
87.8%
AraBERT Random F1
38.1%
AraBERT Disjoint F1
38%
Live Demo Acc
Leakage
Primary Flaw

This project audited the AFND dataset and discovered catastrophic source leakage. The models associate words like "الحكومة" (Government) or publisher names with credibility, failing entirely on unseen sources. The web demo natively runs the AraBERT ONNX model in your browser via Hugging Face.

Read the full research paper on GitHub →