Model description
Pangram 4.1 is an AI detection model released by Pangram Labs on October 19, 2026. It is the successor to Pangram 4, with improvements to overall accuracy and performance under uncertainty.
As with Pangram 4, Pangram 4.1 uses technology detailed in the EditLens technical report.
Inputs
No change from Pangram 4.
Outputs
Pangram 4.1 returns one additional output, class_probabilities, which indicates how sure the model is of each possible label prediction.
Like Pangram 4, Pangram 4.1 first partitions scanned text into a sequence of segments of variable length. The model makes token-level predictions, which it then decodes into one of three labels for each segment: Human, AI-Assisted, or AI-Generated.
Segment-level outputs
Unchanged from Pangram 4: text, label, ai_assistance_score, start_index / end_index, word_count / token_length, humanizer_score, is_humanized.
New in Pangram 4.1:
class_probabilities (Array of 3 floats)
The segment's posterior probabilities for Human, AI-Assisted, and AI-Generated.
Document-level outputs
No change from Pangram 4.
Model resolution
Slightly improved from Pangram 4. You may occasionally see shorter segments in the detection readout.
Supported languages
English Spanish French Portuguese Arabic Chinese Japanese Korean Russian Turkish Hungarian German Dutch Swedish Romanian Ukrainian Polish Italian Czech Greek Hindi Persian Urdu Vietnamese
Pangram can work effectively on languages outside the official support set. Please contact us to request an evaluation on another language.
Model training
No change from Pangram 4.
Model inference
No change from Pangram 4.
Evaluations
Methodology
For evaluation we use the prediction_short field.
For binary evaluation (false positives and false negatives), we report any Mixed prediction as a misclassification.
Evaluation sets
For Pangram 4.1, we made two major changes to our headline evaluation sets for English and multilingual FPR. In previous versions of Pangram, we have evaluated on 1 million pre-ChatGPT FineWeb texts for our headline numbers. For Pangram 4.1, we decided to further clean this data, finding and excluding junk text such as spam, advertising and programmatically generated documents. We also preprocessed the documents, removing a selection of non-textual features from evaluated text.
We elected to make these changes to improve the fidelity of our headline evaluation results. Because this is a more challenging dataset, for Pangram 4.1 we return to our standard of calibrating our headline false positive rate to roughly 1 in 10,000.
False positive rate
Pangram 4.1's English false positive rate is measured on 781,981 human-written FineWeb examples. We evaluate Pangram 4.1's multilingual false positive rates on pre-2022 subsets of FineWeb2.
Overall false positive rate
| Evaluation corpus | N | False positives | False positive rate | 95% confidence interval |
|---|---|---|---|---|
| FineWeb English, in-scope | 781,981 | 70 | 0.0090% | 0.0071% to 0.0113% |
| FineWeb2, all 104 evaluated languages | 1,000,000 | 33 | 0.0033% | 0.0023% to 0.0046% |
False positive rate by language
Multilingual false positive rate is evaluated on FineWeb2.
| Language | Pangram 4.1 |
|---|---|
| Arabic | 0.0026% |
| Chinese | 0.0000% |
| Czech | 0.0090% |
| Dutch | 0.0000% |
| French | 0.0052% |
| German | 0.0052% |
| Greek | 0.0000% |
| Hindi | 0.0090% |
| Hungarian | 0.0090% |
| Italian | 0.0026% |
| Japanese | 0.0000% |
| Korean | 0.0052% |
| Persian | 0.0000% |
| Polish | 0.0026% |
| Portuguese | 0.0157% |
| Romanian | 0.0000% |
| Russian | 0.0052% |
| Spanish | 0.0026% |
| Swedish | 0.0000% |
| Turkish | 0.0026% |
| Ukrainian | 0.0090% |
| Urdu | 0.0000% |
| Vietnamese | 0.0026% |
False negative rate
The English evaluation contains generations from 26 frontier open-source and closed-weight AI models.
Overall false negative rate
| Evaluation corpus | Coverage | N | False negatives | False negative rate | 95% Wilson confidence interval |
|---|---|---|---|---|---|
| English AI generations | 26 generator models | 520,000 | 1,899 | 0.3652% | 0.3492% to 0.3820% |
| Multilingual AI generations | 18 languages | 190,149 | 3,793 | 1.99% | 1.93% to 2.06% |
False negative rate by AI model
| Company | Generator | Pangram 4.1 |
|---|---|---|
| Anthropic | Claude Fable 5 | 0.235% |
| Anthropic | Claude Opus 5 | 0.145% |
| Anthropic | Claude Sonnet 5 | 0.130% |
| Anthropic | Claude Opus 4.8 | 0.215% |
| Anthropic | Claude Sonnet 4.6 | 0.200% |
| Anthropic | Claude Haiku 4.5 | 0.160% |
| Anthropic | Claude family aggregate | 0.181% |
| DeepSeek | DeepSeek V4 Flash | 0.475% |
| DeepSeek | DeepSeek V4 Pro | 0.335% |
| DeepSeek | V4 family aggregate | 0.405% |
| Gemini 3.1 Pro Preview | 0.600% | |
| Gemini 3.5 Flash | 0.595% | |
| Gemini family aggregate | 0.598% | |
| Gemma 4 31B IT | 0.500% | |
| Meta | Llama 3.3 70B Instruct | 0.720% |
| Mistral AI | Mistral Medium 3.5 | 0.400% |
| Moonshot AI | Kimi K2.6 | 0.450% |
| NVIDIA | Nemotron 3 Ultra 550B-A55B | 0.290% |
| OpenAI | GPT-5.6 Sol | 0.465% |
| OpenAI | GPT-5.6 Terra | 0.405% |
| OpenAI | GPT-5.5 | 0.330% |
| OpenAI | GPT-5.4 | 0.245% |
| OpenAI | GPT-5.4 Mini | 0.340% |
| OpenAI | GPT-OSS 120B | 0.155% |
| OpenAI | GPT family aggregate | 0.323% |
| Alibaba Cloud | Qwen 3.7 Max | 0.345% |
| Tencent | HY3 Preview | 0.385% |
| Thinking Machines | Inkling | 0.165% |
| xAI | Grok 4.3 | 0.720% |
| Z.ai | GLM 5.2 | 0.490% |
| Overall | All generators | 0.3652% |
3rd party benchmarks
| Benchmark | Evaluation split | N | Pangram 4.1 FPR | Pangram 4.1 FNR |
|---|---|---|---|---|
| Liang et al. (2023): Non-native English | Human-authored non-native English | 91 | 0.000% | — |
| Jabarian and Imas (2025) | Standard, full length | 15,936 | 0.000% | 0.000% |
| Van Vassalaer et al. (2026) | Public fully AI-generated subset | 39 | — | 0.000% |
Changes from Pangram 4
- +Predictions are more stable under small edits. Across 29,103 paired edits, the incidence of unintended verdict flips decreases from 1.08% in Pangram 4 to 0.15% in Pangram 4.1.
- +Dashboard UI has changed to reflect less certainty for text under 150 words.
Intended usage and limitations
Pangram 4.1 is intended for long-form natural-language writing samples of at least 50 words, written primarily in complete sentences.
There are some kinds of text that we consider out of scope at this time, and for which Pangram can produce less reliable results. These include: code, markup, or Markdown text; tables of contents and reference sections; templated or automated writing; AI translations, such as from Google Translate; instructions, recipes and technical manuals; text dominated by mathematical notation; and figure text like axis labels or tabular data.
Raw text and .docx files are recommended over PDFs when available, as PDF parsing can occasionally introduce unintended artifacts.
Ethics & safety
False accusations of AI usage can lead to serious consequences, including reputational damage, emotional trauma, and other undue harm.
We acknowledge that our model has a non-zero error rate and its errors may result in such harms. We commit to continuing to engage with our users and the academic community to educate others on appropriately contextualizing and communicating the results of AI detection software.
We take reports of false positives extremely seriously and work to mitigate their occurrences to the best of our team's ability.
We commit to training and releasing models with the lowest possible false positive rate, and improving our evaluations to test and monitor future model releases for regressions.