AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Analyzing Finetuning Representation Shift for Multimodal LLMs Steering

Pegah Khayatan, Mustafa Shukor, Jayneel Parekh, Arnaud Dapogny, Matthieu Cord

ICCV 2025 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply Yes->No steering vector at last layerLLaVA-1.5 steering · VQAv2 val subsettarget answer count: +828Improved
apply Yes->No steering vectorLLaVA-1.5 steering · VQAv2 yes/no accuracyaccuracy (%): 90.82 -> 69.03Hurt
apply White->Black steering vectorLLaVA-1.5 steering · VQAv2 other accuracyaccuracy (%): 71.10 -> 58.36Hurt
coarse steering toward yes/no, number, other answer typesLLaVA-1.5 · VQAv2 answer-type countstarget type count: 366 -> 557; 122 -> 201; 494 -> 501Improved
steer captions toward colorsLLaVA-1.5 steering · COCO caption stylecaption count: 1309 -> 2561Improved
use fine-grained instead of coarse steering for gender debiasingLLaVA-1.5 steering · COCO gender debiasingcaptions converted: 232 -> 632Improved
use fine-grained instead of coarse steering for gender debiasingQwen2-VL-Instruct steering · COCO gender debiasingcaptions converted: 134 -> 300Improved
apply safety steering vectorQwen2-VL-Instruct · MM-SafetyBench subsetASR: 45/100 -> 5/100Improved
add shift magnitude alpha=1 for concept shift (ablation)concept shift vectors · VG fine-tunings concept recoveryT-OverlapImproved
increase number of concepts K (ablation)concept recovery · concept recovery T-OverlapT-OverlapNo clear effect
extract concepts from deeper layers (ablation)concept recovery · concept recovery T-OverlapT-Overlap gapImproved
steer at deeper layers (ablation)LLaVA-1.5 steering · VQAv2target answer countImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy