Research map / Large language models / Language, safety and interpretability
ACID Test: A Benchmark for Cultural Safety and Alignment in LALMs
Bikash Dutta, Adit Jain, Rishabh Ranjan, Mayank Vatsa, Richa Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compare MERaLiON against SALMONN | LALMs · ACID Set-B, French audio | relevance score: 0.4129 vs 0.4269 | Mixed |
| use Set-B multi-turn dialogues instead of Set-A single-turn prompts | LTU-AS · ACID, Arabic audio | relevance score: 0.2020 -> 0.3924 | Improved |
| use Set-B multi-turn dialogues instead of Set-A single-turn prompts | MERaLiON · ACID, English audio | relevance score: 0.3186 -> 0.3907 | Improved |
| use Set-B multi-turn dialogues instead of Set-A single-turn prompts | LTU · ACID, English audio | relevance score: 0.1190 -> 0.1277 | No clear effect |
| penalise relevance by x/2% when output copies the audio transcript | SALMONN / LTU-AS · ACID | relevance score | Mixed |
| compare closed-source GPT-4o Mini against Gemini-2.5 Flash | closed LALMs · ACID subset (1317 samples) | sentiment score: 0.3728 -> 0.5943 | Improved |
| compare GPT-4o Mini against Gemini-2.5 Flash | closed LALMs · ACID subset (1317 samples) | relevance score: 0.4321 -> 0.5399 | Improved |
| compare closed-source GPT-4o Mini against best open-source MERaLiON | LALMs · ACID | relevance score: 0.3907 -> 0.5399 | Improved |
| compare LTU against MERaLiON on sentiment | LTU · ACID English Set-A | positive sentiment score: 0.6592 vs 0.5578 | Mixed |
Closest papers by meaning
- Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models (ICLR 2026 · similarity 0.88)
- CulturePark: Boosting Cross-cultural Understanding in Large Language Models (NeurIPS 2024 · similarity 0.88)
- Audio Large Language Models Can Be Descriptive Speech Quality Evaluators (ICLR 2025 · similarity 0.87)
- Mellow: a small audio language model for reasoning (NeurIPS 2025 · similarity 0.87)
- MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.87)
- All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
