Research map / Large language models / Language, safety and interpretability
Where It Really Matters: Few-Shot Environmental Conservation Media Monitoring for Low-Resource Languages
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NewsSerow, a GPT-3.5-based pipeline (summarization, 10-shot chain-of-thought classification, self-reflection) to flag conservation news in Nepali and Spanish, deployed by WWF.
Problem
Automated environmental conservation news monitoring normally needs many expert-labeled examples, which is infeasible for low-resource languages such as Nepali used in the global south.
Approach
A few-shot LLM pipeline that summarizes each article, classifies it with 10 in-context examples carrying chain-of-thought explanations, and then runs a self-reflection filter on positive predictions to reduce false positives.
Key results
- NewsSerow reaches F1 0.70 on Nepali and 0.79 on Spanish with 10 demonstrations, vs 0.62 and 0.56 for few-shot GPT-3.5-Turbo.
- NewsSerow's Nepali F1 (0.70) matches fully fine-tuned XLM-R (0.70) while using 10 target-language examples plus English data, and its Spanish F1 (0.79) is slightly below fine-tuned mBERT (0.81).
- In deployment (April-July 2023, subset labeled by WWF), NewsSerow achieved aggregate F1 0.64 for Nepali and 0.67 for Spanish.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add chain-of-thought explanations for the in-context examples (ablation) | NewsSerow · Nepali and Spanish test sets | F1-score | Helped |
| Use zero-shot summaries instead of the first three article sentences (ablation) | NewsSerow · Nepali test set | F1-score | Helped |
| Use zero-shot summaries instead of the first three article sentences (ablation) | NewsSerow · Spanish test set | F1-score | Mixed |
| Add reflection after a positive classification (ablation) | NewsSerow · Nepali and Spanish test sets | Precision and recall | Mixed |
| Increase in-context examples from 2 to 10 (ablation) | NewsSerow · Nepali and Spanish test sets | Performance | Helped |
| Use NewsSerow instead of GPT-3.5-Turbo few-shot prompting without its prompt-engineering features | GPT-3.5-Turbo few-shot classification · Nepali and Spanish test sets | F1-score: 0.62 -> 0.70; 0.56 -> 0.79 | Helped |
Surprising findings
- Few-shot mBERT and XLM-R fine-tuned on 10 examples underperform even vanilla GPT-3.5-Turbo.
- Chain-of-thought explanations improved F1 in every ablation setting, and reflection trades recall for precision.
Limitations
- Deployment metrics are computed only on articles that earlier versions flagged positive and are not comparable to offline results.
- Small test sets (276 Nepali, 125 Spanish) and tendency for false positives from tenuous connections to conservation.
Experimental setup
Datasets: Nepali news articles scraped from four Nepali news sites (548) Colombian Spanish news articles from NewsAPI (248) 1647 English conservation-labeled articles shared by prior work
Benchmarks: Nepali news test set (276 articles) Spanish news test set (125 articles) WWF weekly deployment labels (Apr-Jul 2023)
Metrics: precision recall F1-score
Baselines: Zero-shot GPT-3.5-Turbo Few-shot GPT-3.5-Turbo few-shot mBERT few-shot XLM-R fully fine-tuned mBERT fully fine-tuned XLM-R Translate-Test with RoBERTa
Builds on: in-context learning chain-of-thought prompting self-reflection
Closest papers by meaning
- UniChange: Unifying Change Detection with Multimodal Large Language Model (CVPR 2026 · similarity 0.81)
- Fit for our purpose, not yours: Benchmark for a low-resource, Indigenous language (NeurIPS 2024 · similarity 0.81)
- MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps (CVPR 2025 · similarity 0.81)
- INQUIRE: A Natural World Text-to-Image Retrieval Benchmark (NeurIPS 2024 · similarity 0.81)
- BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity (NeurIPS 2024 · similarity 0.81)
- Combining Observational Data and Language for Species Range Estimation (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.