Research map / Vision-language and multimodal / Visual question answering
Query-Guided Spatial–Temporal–Frequency Interaction for Music Audio–Visual Question Answering
Kun Li, Michael Ying Yang, Sami Sebastian Brandt
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add query-guided multimodal correlation (QGMC) (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 76.80 -> 78.98 | Improved |
| add query context reasoning (QCR) prompting block (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.19 -> 78.98 | Improved |
| add spatial-temporal interaction (STI) (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 77.80 -> 78.98 | Improved |
| add temporal-frequency interaction (TFI, AST features) (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 77.41 -> 78.98 | Improved |
| add query guidance at the beginning stage (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 77.93 -> 78.98 | Improved |
| add question embeddings in TFI (middle-stage guidance) (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.55 -> 78.98 | Improved |
| use AST-only features for the temporal audio pathway instead of VGGish (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.98 -> 78.14 | Hurt |
| combine VGGish and AST features in the audio pathway (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.98 -> 78.65 | No clear effect |
| use alternative prompts (translation, caption, GPT-4o generative) instead of unified keyword prompts (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.98 -> 78.44 / 78.28 / 78.37 | Hurt |
| QA-TIGER-style sequential early processing instead of QGMC (ablation) | QSTar · MUSIC-AVQA test | overall accuracy (%): 78.98 -> 78.89 | No clear effect |
| add prompting to QSTar (ablation) | QSTar · AVQA test | total accuracy (%): 90.9 -> 91.2 | Improved |
| QSTar vs QA-TIGER on Visual QA | QSTar · MUSIC-AVQA test Visual QA | accuracy (%): 85.14 (QA-TIGER) vs 84.17 | Hurt |
Closest papers by meaning
- Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering (NeurIPS 2024 · similarity 0.93)
- AVQACL: A Novel Benchmark for Audio-Visual Question Answering Continual Learning (CVPR 2025 · similarity 0.91)
- Audio-Visual Adaptive Fusion Network for Question Answering Based on Contrastive Learning (AAAI 2025 · similarity 0.91)
- Object-Aware Adaptive-Positivity Learning for Audio-Visual Question Answering (AAAI 2024 · similarity 0.90)
- Question-Aware Gaussian Experts for Audio-Visual Question Answering (CVPR 2025 · similarity 0.90)
- MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
