Research map / Vision-language and multimodal / Multimodal LLMs
SpecEyes: Accelerating Agentic Multimodal LLM via Speculative Planning and Perception
Haoyu Huang, Jinfa Huang, Zhongwei Wan, Xiawu Zheng, Rongrong Ji, Jiebo Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| route queries with min-separability gating to a small tool-free model before the agentic model | DeepEyes · Average (V*, HR-Bench, POPE) | accuracy / speedup: 81.39 -> 84.26 at 1.73x | Improved |
| route queries with min-separability gating | Thyme · Average (V*, HR-Bench, POPE) | accuracy / speedup: 82.29 -> 83.99 at 1.42x | Improved |
| apply SpecEyes to high-resolution inputs | SpecEyes (min) · HR-Bench 8K | speedup: 1.08x (DeepEyes), 0.95x (Thyme) | No clear effect |
| use min aggregation of token separability instead of mean (ablation) | SpecEyes (DeepEyes) · Average (V*, HR-Bench, POPE) | accuracy: 80.53 -> 84.26 | Improved |
| use separability score instead of softmax geometric-mean confidence (ablation) | SpecEyes (DeepEyes) · Average (V*, HR-Bench, POPE) | accuracy: 82.31 -> 84.26 | Improved |
| use min instead of bottom-r aggregation (ablation) | SpecEyes (Thyme) · Average (V*, HR-Bench, POPE) | accuracy: 81.95 -> 83.99 | Improved |
| disable tools in the large backbone instead of routing (ablation) | DeepEyes · POPE Adversarial | accuracy: 78.43 -> 46.90 | Hurt |
| replace Qwen3-VL-2B draft with larger Qwen3-VL-8B (ablation) | SpecEyes (min), DeepEyes · Average (V*, HR-Bench, POPE) | accuracy / speedup: V* Attr 92.17 vs 90.43; speedup 1.67x vs 1.73x | Mixed |
| increase Top-K in separability computation (ablation) | SpecEyes (min) · V* and HR-Bench | accuracy / speedup | Mixed |
| lower the gating threshold (ablation) | SpecEyes (min) · V*, HR-Bench, POPE | accuracy / speedup | Mixed |
| increase serving batch size (ablation) | SpecEyes · V*, HR-Bench, POPE | speedup | Improved |
| use token-level speculative reasoning (SpecReason) instead of agentic-level speculation | DeepEyes · Average (V*, HR-Bench, POPE) | speedup: 0.43x average | Hurt |
Closest papers by meaning
- From an LLM Swarm to a PDDL-empowered Hive: Planning Self-executed Instructions in a Multi-modal Jungle (ICLR 2025 · similarity 0.89)
- Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents (NeurIPS 2025 · similarity 0.89)
- EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.88)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.88)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.88)
- Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
