AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Composing Ensembles of Pre-trained Models via Iterative Consensus

Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Igor Mordatch

ICLR 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: Open-vocabulary alignment from VLMs · 7 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add CLIP, classifier (CLS), and classifier-free guidance scorers together to GLIDE generator (ablation)GLIDE image generation · ImageNet 64x64IS: 25.017 -> 34.952Improved
Add CLS and CLS-FREE scorers to CLIP scorer (multi-scorer ensemble) (ablation)GLIDE image generation · ImageNet 64x64FID: 30.462 -> 29.184Improved
Compose CLIP scorer with CLS scorer rather than CLIP alone (ablation)GLIDE image generation · ImageNet 64x64FID: 30.462 -> 29.543Improved
Use multiple CLIP scorers instead of a single CLIP-32 scorer (ablation)GPT-2 captioning for video QA · ActivityNet-QAaccuracy: 58.389 -> 61.168Improved
Compose GPT-2 generator with CLIP scorer (zero-shot, no finetuning) instead of JustAsk pretrainedvideo QA · ActivityNet-QAaccuracy: 50.671 -> 58.389 (+7.72)Improved
Compose GPT-2 generator with a GSM8K question-solution classifier scorer with iterative token-level refinementGSM8K math · GSM8Kaccuracy (BS=1): 3.487 (GPT-FT) -> 16.831Improved
Iterative refinement of scorer feedback at every step vs scorer feedback only at final step (ablation)GPT-2 Medium generator with CLS scorer · GSM8Kaccuracy (BS=1): 9.704 -> 17.210Improved
Iterative refinement vs final-step-only feedback with finetuned generator (ablation)GPT-2 Medium (FT) with CLS scorer · GSM8Kaccuracy (BS=1): 14.481 -> 17.210 (versus PIC)Improved
Compose 5 camera-view scorers summed instead of a single view scorerMPC+World model robot manipulation · Ravens, 2 relations, textsuccess rate: 35.0 -> 67.5Improved
Compose 5 camera-view scorers summed instead of a single view scorerMPC+World model robot manipulation · Ravens, 3 relations, textsuccess rate: 50.0 -> 75.0Improved
Remove iterative refinement (scorers only select among 100 sampled trajectories) (ablation)MPC+World model robot manipulation with 5 views · Ravens, 2 relations, textsuccess rate: 30.0 -> 67.5Improved
Remove iterative refinement (scorers only select among 100 sampled trajectories) (ablation)MPC+World model robot manipulation with 5 views · Ravens, 3 relations, textsuccess rate: 46.6 -> 75.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy