Research map / Vision-language and multimodal / Cross-modal learning
Look, Listen and Learn — A Multimodal LSTM for Speaker Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace per-frame CNN/CNN+SVM classification with an LSTM over CNN face features | face sequence classifier · BBT face sequence classification | accuracy (%): 92.42 -> 95.61 | Helped |
| replace CNN with CNN+SVM on features | face sequence classifier · BBT face sequence classification | accuracy (%): 92.33 -> 92.42 | No clear effect |
| add cross-modal sharing of recurrent weights and output weights W_y (full sharing) vs no sharing (ablation) | multimodal LSTM · BBT distractor rejection ROC | ROC (false alarm vs accuracy) | Helped |
| use modality-specific W_y (half sharing) instead of shared W_y (ablation) | multimodal LSTM · BBT distractor rejection ROC | false alarm rate | Hurt |
| replace Hu et al. with multimodal LSTM | speaker identification · BBT S01E03 speaker naming, 0.5s window | accuracy (%): 74.93 -> 86.59 | Helped |
| replace Hu et al. with multimodal LSTM at 2.5s window | speaker identification · BBT S01E03 speaker naming, 2.5s window | accuracy (%): 82.81 -> 91.17 | Helped |
Closest papers by meaning
- Massive Sound Embedding Benchmark (MSEB) (NeurIPS 2025 · similarity 0.85)
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.85)
- Mirasol3B: A Multimodal Autoregressive Model for Time-Aligned and Contextual Modalities (CVPR 2024 · similarity 0.84)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.84)
- LLM-driven Multimodal and Multi-Identity Listening Head Generation (CVPR 2025 · similarity 0.84)
- $\texttt{AVROBUSTBENCH}$: Benchmarking the Robustness of Audio-Visual Recognition Models at Test-Time (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.