AI Detection Accuracy Tests: The Most Transparent Benchmark

1. Research Methodology

Key Takeaway: We tested seven detectors on 400 documents across ten categories. Our goal was not to crown a winner, but to show how each tool performs under realistic conditions and where each falls short.

Benchmarks in the AI detection space are often criticized for being opaque, cherry-picked, or outdated. To avoid those problems, we disclose our dataset composition, scoring thresholds, and limitations.

Dataset Composition

Our corpus contains 400 documents, balanced across the following categories:

• Human Essays (n=60): Undergraduate argumentative essays from public academic datasets.
• Research Papers (n=40): Abstracts and introductions from STEM and social science papers.
• Technical Writing (n=30): Software documentation, API guides, and engineering reports.
• Creative Writing (n=20): Short fiction and personal narratives.
• ChatGPT Outputs (n=50): GPT-4o generated essays, summaries, and marketing copy.
• GPT-5.5 Outputs (n=50): Advanced reasoning and long-form content.
• Gemini Outputs (n=50): Google Gemini-generated articles and explanations.
• Claude Outputs (n=40): Anthropic Claude 3.5 summaries and analytical texts.
• Edited AI (n=40): AI drafts rewritten by human editors.
• Humanized AI (n=20): AI drafts processed through humanization tools.

All AI documents were generated with standard system prompts without jailbreaks or custom instructions, unless specified in the edited and humanized categories. Human documents were written before 2022 or sourced from public domain datasets with clear authorship attribution.

Tools Tested

2. Evaluation Metrics Explained

We report the standard binary classification metrics used in machine learning research:

• Accuracy: The proportion of correct predictions out of all predictions. This metric can be misleading if the dataset is imbalanced.
• Precision: Of all documents flagged as AI, what proportion actually were AI? High precision means fewer false accusations.
• Recall: Of all AI documents, what proportion did the detector catch? High recall means fewer missed AI texts.
• F1 Score: The harmonic mean of precision and recall. Useful for comparing detectors when the dataset is balanced.
• False Positive Rate (FPR): The proportion of human documents incorrectly flagged as AI. Critical for fairness.
• False Negative Rate (FNR): The proportion of AI documents incorrectly labeled as human.

For each detector, we used the default threshold unless the tool explicitly allowed threshold tuning. Where available, we report 95% confidence intervals using the Wilson score interval for proportions.

3. Testing Results

Overall Performance (400 documents)

Note: These figures are from our June 2026 benchmark and should be treated as estimates. Detector models are updated continuously, and performance varies by domain, language, and document length.

Performance by Document Type

4. Why Detectors Disagree and Make Errors

Key Takeaway: Disagreement is normal. Detectors use different training data, model architectures, and thresholds. They also specialize in different text domains.

Why False Positives Occur

False positives are most common when human writing is highly structured, predictable, or formulaic. Examples include legal contracts, technical documentation, scientific abstracts, and writing by non-native English speakers who use simpler, more grammatically regular sentence constructions. In these cases, the detector sees low perplexity and assumes AI involvement.

Why False Negatives Occur

False negatives occur when AI text has been heavily edited, humanized, or generated with custom instructions designed to maximize variance and unpredictability. Skilled prompt engineers can produce AI drafts with perplexity and burstiness profiles that closely resemble human writing.

Why Scores Vary by Detector

One detector might flag a Gemini article at 90% while another reports 45%. This happens because each model was trained on different data and calibrated differently. Some detectors are conservative, flagging only when highly confident. Others are aggressive, flagging more liberally to catch edge cases.

5. Practical Recommendations by Role