Research map / Video / Video-language understanding
Listening Between the Frames: Bridging Temporal Gaps in Large Audio-Language Models
Hualei Wang, Yiming Li, Shuo Ma, Hong Liu, Xiangdong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temporal marker tokens with numeral-based init (ablation) | SALMONN FTAR-tuned · AudioSet-Strong dense captioning | Eb-F1: 32.4 -> 36.0 | Improved |
| initialize temporal marker embeddings from numeral token embeddings instead of random (ablation) | temporal markers · AudioSet-Strong dense captioning | Eb-F1: 33.5 -> 36.0 | Improved |
| add absolute time-aware encoding to temporal markers (ablation) | SALMONN + TM · AudioGrounding | mIoU: 54.9 -> 56.0 | Improved |
| add absolute time-aware encoding only (ablation) | SALMONN FTAR-tuned · AudioGrounding | mIoU: 51.9 -> 53.8 | Improved |
| add segment-level token merging (ablation) | SALMONN + TM + ATE · CNN/DailyMail speech timeline | mIoU: 90.4 -> 94.2 | Improved |
| add segment-level token merging (ablation) | SALMONN + TM + ATE · AudioSet-Strong dense captioning | Eb-F1: 37.8 -> 37.4 | Hurt |
| add segment-level token merging (ablation) | SALMONN + TM + ATE · AudioGrounding | mIoU: 56.0 -> 57.8 | Improved |
| raise retained token ratio from 0.25 to 0.30 (ablation) | TimeAudio · CNN/DailyMail speech | ROUGE-1: 42.4 -> 42.5 | No clear effect |
| lower retained token ratio from 0.25 to 0.10 (ablation) | TimeAudio · CNN/DailyMail speech | ROUGE-1: 42.4 -> 24.3 | Hurt |
| TimeAudio vs fine-tuned Qwen2-Audio | TimeAudio · AudioSet-Strong dense captioning | METEOR: 22.4 -> 20.4 | Hurt |
| TimeAudio vs FTAR-tuned SALMONN-7B | TimeAudio · AudioGrounding | mIoU: 51.9 -> 57.8 | Improved |
Closest papers by meaning
- Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding (AAAI 2025 · similarity 0.89)
- MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks (ICML 2026 · similarity 0.88)
- MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.88)
- TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding (CVPR 2024 · similarity 0.88)
- FAVE: A Structured Benchmark for Fine-Grained Audio-Visual Temporal Evaluation in Multimodal LLMs (CVPR 2026 · similarity 0.88)
- FLAM: Frame-Wise Language-Audio Modeling (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
