Research map / Large language models / Language, safety and interpretability
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes AI content detection in human-AI hybrid essays as boundary detection and proposes TriBERT, a triplet-fine-tuned sentence encoder plus adjacent-prototype distance maximization, evaluated on a ChatGPT-generated hybrid essay dataset built from ASAP essays.
Problem
Detecting where human-written text transitions to AI-generated text within collaboratively written hybrid essays, rather than classifying whole texts as human or AI.
Approach
Fine-tune a sentence encoder with triplet loss to separate AI from human sentences, then compute prototypes (averages of p consecutive sentence embeddings) and place boundaries at the adjacent prototype pairs with the largest Euclidean distance.
Key results
- TriBERT (p=2) achieves the best overall In-Domain F1 of 0.575 vs. 0.536 for fine-tuned BERT.
- Out-of-domain, TriBERT (p=2) reaches overall F1 0.510 vs. 0.486 for BERT, and removing encoder fine-tuning (NT) drops performance to about 0.23-0.27.
- For single-boundary essays, TriBERT with p=4 improves over the best baseline by 22% (In-Domain) and 18% (Out-of-Domain).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TriBERT with prototype size p=2 instead of BERT | Boundary detection · In-Domain, All | F1@3: 0.536 -> 0.575 | Helped |
| use TriBERT with prototype size p=2 instead of BERT | Boundary detection · Out-of-Domain, All | F1@3: 0.486 -> 0.510 | Helped |
| use prototype size p=4 instead of BERT | TriBERT · In-Domain, #Bry=1 | F1@3: 22% | Helped |
| use prototype size p=4 instead of BERT | TriBERT · Out-of-Domain, #Bry=1 | F1@3: 18% | Helped |
| increase prototype size from p=1 to p=2 (ablation) | TriBERT · In-Domain, All | F1@3: 0.571 -> 0.575 | Helped |
| increase prototype size beyond p=2 (ablation) | TriBERT · In-Domain, All | F1@3: 0.575 -> 0.566 (p=3); 0.575 -> 0.549 (p=4); 0.575 -> 0.519 (p=5); 0.575 -> 0.492 (p=6) | Hurt |
| use larger prototype size p=4 rather than p=1 (ablation) | TriBERT · In-Domain, #Bry=1 | F1@3: 0.430 -> 0.486 | Helped |
| use larger prototype size p=4 rather than p=1 (ablation) | TriBERT · Out-of-Domain, #Bry=1 | F1@3: 0.379 -> 0.436 | Helped |
| use prototype size p=2 rather than p=1 (ablation) | TriBERT · In-Domain, #Bry=2 | F1@3: 0.646 -> 0.692 | Helped |
| use prototype size p=2 rather than p=1 (ablation) | TriBERT · Out-of-Domain, #Bry=2 | F1@3: 0.559 -> 0.597 | Helped |
| use prototype size p=1 rather than p=2 (ablation) | TriBERT · In-Domain, #Bry=3 | F1@3: 0.692 -> 0.646 | Helped |
| use prototype size p=1 rather than p=2 (ablation) | TriBERT · Out-of-Domain, #Bry=3 | F1@3: 0.545 -> 0.640 | Helped |
Surprising findings
- The best prototype size depends on the number of boundaries: larger prototypes help single-boundary essays while smaller ones (p=1 or 2) are preferred for multi-boundary essays.
- The commercial detector GPTZero performs no better than logistic regression on sentence-level boundary detection, and the untrained encoder performs far worse than the fine-tuned one.
Limitations
- Hybrid essays are generated with ChatGPT only and by a specific sentence-level removal-and-fill scheme; boundaries within sentences and multi-turn collaboration are not covered.
- Evaluation uses only essays from junior high school ASAP-AES prompts, limiting generalizability.
- Results rely on F1@K with K fixed to 3, which is tied to the dataset's maximum boundary count.
Experimental setup
Datasets: ASAP-AES student essays (Kaggle) ChatGPT-generated hybrid essays
Benchmarks: Hybrid essay boundary detection dataset (in-domain) Hybrid essay boundary detection dataset (out-of-domain, prompt-wise 8-fold)
Metrics: F1@3
Baselines: BERT LR GPTZero RANDOM
Builds on: Sentence-BERT triplet network BERT prototype-based representations
Closest papers by meaning
- DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning (NeurIPS 2025 · similarity 0.89)
- Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · similarity 0.87)
- DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning (NeurIPS 2024 · similarity 0.87)
- Raidar: geneRative AI Detection viA Rewriting (ICLR 2024 · similarity 0.86)
- Few-Shot Detection of Machine-Generated Text using Style Representations (ICLR 2024 · similarity 0.86)
- Multiscale Positive-Unlabeled Detection of AI-Generated Texts (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.