Research map / Label-efficient and robust learning / Recognition and long-tail
PUBLICSPEAK: Hearing the Public with a Probabilistic Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PublicSpeak, a probabilistic soft logic framework that fuses meeting structure, speaker roles, and linguistic/GenAI signals to extract public comments from local government meeting transcripts across seven US cities.
Problem
Identifying remarks made by the public at local government meetings at scale, since small meetings are not covered by newsrooms and existing methods ignore meeting structure and generalize poorly across cities.
Approach
Encodes meeting structure, speaker roles, domain knowledge, and linguistic cues (fine-tuned PLMs and GPT-4 section detection) as weighted logical rules in Probabilistic Soft Logic, jointly inferring discussion sections, speaker roles, and remark types.
Key results
- PUBLICSPEAK achieves the best PC-F1 in all cities and settings, improving over the RoBERTa baseline by about 10% on average and up to 40% in some cities.
- In the leave-one-city-out setting, PUBLICSPEAK (3-class) reaches overall PC-F1 0.688 versus 0.610 for RoBERTa, about a 13% gain in generalization to unseen cities.
- Zero-shot GPT-4o performs poorly on PC detection (overall PC-F1 0.443), below fine-tuned RoBERTa and DistilBERT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PUBLICSPEAK instead of RoBERTa (fine-tuned) | Public comment detection · Overall PC (3-Class) | PC-F1: 0.630 -> 0.688 | Helped |
| Use PUBLICSPEAK instead of the strongest baseline | Public comment detection · LOCO PC (3-Class) | PC-F1: 0.646 -> 0.663 | Helped |
| Use PUBLICSPEAK instead of DistilBERT (fine-tuned) | Public comment detection · Seattle PC (3-Class) | PC-F1: 0.957 -> 0.929 | Hurt |
| Use PUBLICSPEAK instead of RoBERTa (fine-tuned) | Public comment detection · Oakland PC (3-Class) | PC-F1: 0.717 -> 0.785 | Helped |
| Use PUBLICSPEAK instead of GPT4o | Public comment detection · Richmond PC (3-Class) | PC-F1: 0.857 -> 1.000 | Helped |
| Use PUBLICSPEAK instead of RoBERTa (fine-tuned) | Public comment detection · Ann Arbor PC (3-Class) | PC-F1: 0.642 -> 0.894 | Helped |
| Use PUBLICSPEAK instead of RoBERTa (fine-tuned) | Public comment detection · Lansing PC (3-Class) | PC-F1: 0.587 -> 0.612 | Helped |
| Use PUBLICSPEAK instead of DistilBERT (fine-tuned) | Public comment detection · Royal Oak PC (3-Class) | PC-F1: 0.785 -> 0.811 | Helped |
| Use PUBLICSPEAK instead of DistilBERT (fine-tuned) | Public comment detection · Jackson PC (3-Class) | PC-F1: 0.795 -> 0.870 | Helped |
| Use the 3-Class formulation instead of the 2-Class formulation | PUBLICSPEAK · Overall PC | PC-F1: 0.663 -> 0.688 | Helped |
Surprising findings
- Fine-tuned PLMs alone perform nearly as well as the structured approach in Seattle, where public hearings are rare and comment topics are restricted.
- Zero-shot GPT-4o underperforms fine-tuned smaller encoder models on public comment detection.
Limitations
- Each pipeline stage (video quality, ASR, diarization) introduces errors and hallucinations.
- Known racial disparities in ASR may differentially affect cities.
- Performance is highly variable across cities, e.g. zero F1 in Richmond for several methods due to very few public comments.
Experimental setup
Datasets: city council meeting transcripts from seven US cities (Seattle, Oakland, Richmond, Ann Arbor, Lansing, Royal Oak, Jackso
Benchmarks: PUBLICSPEAK annotated city council meeting dataset
Metrics: F1 (PC class) statistical significance
Baselines: PHRASES+ROLES GENAI+ROLES SBert + SVM TFIDF + SVM RoBERTa DistilBERT GPT-4o
Builds on: Probabilistic Soft Logic WhisperX BERTopic
Closest papers by meaning
- Representative Ranking for Deliberation in the Public Sphere (ICML 2025 · similarity 0.81)
- Freeing the Law with LOCUS: A Local Ordinance Corpus for the United States (NeurIPS 2026 · similarity 0.78)
- The Distortion of Public-Spirited Participatory Budgeting (AAAI 2025 · similarity 0.78)
- Topic-Conversation Relevance (TCR) Dataset and Benchmarks (NeurIPS 2024 · similarity 0.78)
- CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models (AAAI 2024 · similarity 0.78)
- Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes (ICLR 2025 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.