NewsLies

Arabic News Credibility Classifier · Research Demo

Model: AraBERT v0.2
⚠️ Research Warning
This model learns credibility patterns from the weakly supervised AFND dataset. It does not verify individual claims against external evidence. Our research shows that apparent accuracy on this dataset is largely driven by publisher-identifying information: the research model scores high on random data (87.8% Macro-F1) but collapses on articles from unseen publishers (37.0%). Treat every prediction as a research artifact about publisher style — not a fact-check.
Arabic News Text
Loading model…
Result
About this Research
87.8%
Research model · Random Macro-F1
37.0%
Research model · Unseen publishers
—
Deployed student · Disjoint F1 (pending)
Not a fact-checker
Weak source-derived labels

AFND labels were assigned at the publisher level, so models can succeed by recognizing who wrote an article instead of evaluating its claims. This project quantifies that shortcut: strong text-to-source predictability (>98%) plus the random→unseen-publisher collapse shown above is consistent with substantial publisher-dependent shortcuts. The browser demo runs an ONNX model locally in your browser via Hugging Face tooling. Predictions are calibrated (temperature scaling fitted on validation data only).

Read the full research on GitHub →