Research map / Large language models / Language, safety and interpretability
DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DETree, which models human-AI collaborative text generation processes as a Hierarchical Affinity Tree trained with a tree-structured contrastive loss on RoBERTa-large, plus a new hybrid-text benchmark RealBench, with retrieval-based few-shot adaptation for OOD detection.
Problem
Detecting AI-involved text produced by diverse human-AI collaborative processes (AI-edited human text, human-edited AI text, multi-LLM refinement) is poorly handled by binary or flat multi-class detectors, which generalize poorly to new LLMs and out-of-distribution domains.
Approach
Build a hierarchical affinity tree over generation-source categories from the inter-class similarity matrix (agglomerative clustering plus silhouette-based subtree reorganization with predefined human/AI priors), then train the encoder with a tree-structured contrastive loss that makes closer tree relatives more similar than distant ones; at inference use K-means-compressed kNN retrieval with few-
Key results
- On supervised MAGE/M4/TuringBench/M4 multilingual, DETree (prior1) reaches 97.88 average AvgRec and 97.75 average F1, above DeTeCTive at 96.94/96.74.
- Few-shot DETree improves AUROC on MAGE-Paraphrase by +15.55 and on Beemo GPT4o-Edited/Llama3.1-Edited by +10.39/+15.01 over zero-shot baselines.
- On HART level-1 detection, DETree (prior1) achieves 0.998 AUROC with 99.5% TPR@5%FPR on the Essay-ALL aggregate, versus 0.780 for Binoculars.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the prior1 structure, placing hybrid texts under AI-generated content (ablation) | DETree · Supervised average | F1: 97.49 -> 97.75 | Helped |
| Place hybrid texts under human-written texts instead of using prior1 (ablation) | DETree · Supervised average | F1: 97.75 -> 96.03 | Hurt |
| Remove Tree-Structured Contrastive Loss (ablation) | DETree · Supervised average | F1: 97.75 -> 96.32 | Hurt |
| Set the maximum HAT depth to 6 rather than 3 (ablation) | DETree · MAGE | F1: 96.48 -> 96.96 | Helped |
| Increase maximum HAT depth from 6 to 9 (ablation) | DETree · MAGE | F1: 96.96 -> 96.84 | Hurt |
| Add target-domain support samples to rebuild the retrieval database | DETree · OOD benchmarks | AUROC: +15.55 on MAGE-Paraphrase; +7.94 on DetectRL; +10.39/+15.01 on Beemo | Helped |
| Use a 10K-representative compressed RealBench retrieval database | DETree · RealBench database compression | detection accuracy | Helped |
| Train with adversarial samples | DETree · Attack-augmented MAGE test set | TPR@5% FPR | Helped |
| Train using only 10% of the full category set (ablation) | DETree · MAGE | TPR@5% FPR: within 3% | Hurt |
Surprising findings
- Hybrid texts are closer to AI-generated classes than to human texts; treating collaborative text as AI-involved (prior1) worked best, and human-prior assumption (prior2) performed worst.
- Machine-humanized human texts are indistinguishable from machine-generated base texts but distinguishable when the base text is human-written.
Limitations
- Adversarial evasion via model fine-tuning to bypass detection was not explored.
- Only hybrid texts involving up to three authors were analyzed.
- Unseen rare domains still require a small number of in-domain samples to adjust decision boundaries.
Experimental setup
Datasets: MAGE M4 TuringBench OUTFOX RAID RealBench
Benchmarks: MAGE M4 TuringBench DetectRL Beemo HART RAID
Metrics: F1 AvgRec AUC-ROC AUROC TPR@5% FPR
Baselines: Fast-DetectGPT Binoculars Glimpse HART UAR SCL MAGE T5-Sentinel DeTeCTive RADAR RoBERTa-Base
Builds on: supervised contrastive learning agglomerative hierarchical clustering HART Fast-DetectGPT
Closest papers by meaning
- Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education (AAAI 2024 · similarity 0.89)
- DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning (NeurIPS 2024 · similarity 0.88)
- Raidar: geneRative AI Detection viA Rewriting (ICLR 2024 · similarity 0.88)
- Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · similarity 0.88)
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection (NeurIPS 2025 · similarity 0.87)
- Few-Shot Detection of Machine-Generated Text using Style Representations (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.