AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

GENIXER: Empowering Multimodal Large Language Models as a Powerful Data Generator

Henry Hengyuan Zhao, Pan Zhou, Mike Zheng Shou

ECCV 2024 · Vision-language and multimodal › Multimodal LLMs · approach: Visual instruction tuning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add 915K Genixer synthetic VQA data to LLaVA-1.5 pretrainingLLaVA-1.5 7B · VizWizaccuracy: 50.0 -> 53.8Improved
add Genixer-915K synthetic VQA dataLLaVA-1.5 7B · MMEscore: 1465.0 -> 1502.7Improved
add Genixer-915K synthetic VQA dataLLaVA-1.5 7B · SEED-Bench imageaccuracy: 66.2 -> 66.6No clear effect
add Genixer-915K synthetic VQA dataLLaVA-1.5 7B · LLaVA-Bench (in the wild), MM-VetGPT-4 judged score: 65.4 -> 64.0; 31.1 -> 30.1Hurt
add Genixer-350K synthetic REC data to Shikra trainingShikra · RefCOCO/+/g averageaccuracy: 82.92 -> 83.49Improved
add Genixer-350K synthetic REC data to Shikra trainingShikra · RefCOCOg valaccuracy: 82.27 -> 81.99Hurt
use the Genixer-L generator model directly (zero-shot) as a VQA modelGenixer-L · GQA, ScienceQA, SEEDaccuracy: GQA 62.0 -> 61.3; SQA 66.8 -> 65.6; VQAv2 78.5 -> 79.7Mixed
retrain Genixer-L with the 665K LLaVA finetuning data mixed in (MixT) (ablation)Genixer-L · six benchmarksaccuracy: VizWiz 50.0 -> 54.1; GQA 62.0 -> 63.1Improved
scale synthetic data from 300K to 915K (ablation)LLaVA-1.5 pretraining · ScienceQA-IMGaccuracy: 68.5 -> 69.7Improved
filter samples with Fuyu-8B probability threshold 0.7 instead of none (ablation)Genixer-915K · ScienceQA-IMGaccuracy: 69.6 -> 69.7 (VizWiz 53.5 -> 53.8) with 1.4M -> 0.9M samplesImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy