Research map / Vision-language and multimodal / CLIP and zero-shot
Composing Ensembles of Pre-trained Models via Iterative Consensus
Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Igor Mordatch
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CLIP, classifier (CLS), and classifier-free guidance scorers together to GLIDE generator (ablation) | GLIDE image generation · ImageNet 64x64 | IS: 25.017 -> 34.952 | Improved |
| Add CLS and CLS-FREE scorers to CLIP scorer (multi-scorer ensemble) (ablation) | GLIDE image generation · ImageNet 64x64 | FID: 30.462 -> 29.184 | Improved |
| Compose CLIP scorer with CLS scorer rather than CLIP alone (ablation) | GLIDE image generation · ImageNet 64x64 | FID: 30.462 -> 29.543 | Improved |
| Use multiple CLIP scorers instead of a single CLIP-32 scorer (ablation) | GPT-2 captioning for video QA · ActivityNet-QA | accuracy: 58.389 -> 61.168 | Improved |
| Compose GPT-2 generator with CLIP scorer (zero-shot, no finetuning) instead of JustAsk pretrained | video QA · ActivityNet-QA | accuracy: 50.671 -> 58.389 (+7.72) | Improved |
| Compose GPT-2 generator with a GSM8K question-solution classifier scorer with iterative token-level refinement | GSM8K math · GSM8K | accuracy (BS=1): 3.487 (GPT-FT) -> 16.831 | Improved |
| Iterative refinement of scorer feedback at every step vs scorer feedback only at final step (ablation) | GPT-2 Medium generator with CLS scorer · GSM8K | accuracy (BS=1): 9.704 -> 17.210 | Improved |
| Iterative refinement vs final-step-only feedback with finetuned generator (ablation) | GPT-2 Medium (FT) with CLS scorer · GSM8K | accuracy (BS=1): 14.481 -> 17.210 (versus PIC) | Improved |
| Compose 5 camera-view scorers summed instead of a single view scorer | MPC+World model robot manipulation · Ravens, 2 relations, text | success rate: 35.0 -> 67.5 | Improved |
| Compose 5 camera-view scorers summed instead of a single view scorer | MPC+World model robot manipulation · Ravens, 3 relations, text | success rate: 50.0 -> 75.0 | Improved |
| Remove iterative refinement (scorers only select among 100 sampled trajectories) (ablation) | MPC+World model robot manipulation with 5 views · Ravens, 2 relations, text | success rate: 30.0 -> 67.5 | Improved |
| Remove iterative refinement (scorers only select among 100 sampled trajectories) (ablation) | MPC+World model robot manipulation with 5 views · Ravens, 3 relations, text | success rate: 46.6 -> 75.0 | Improved |
Closest papers by meaning
- Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models (ICLR 2026 · similarity 0.85)
- Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models (ICML 2024 · similarity 0.85)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.85)
- Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models (ICML 2026 · similarity 0.84)
- AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.84)
- SEED-Bench: Benchmarking Multimodal Large Language Models (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
