Research map / Vision-language and multimodal / Multimodal LLMs
Multimodal Large Language Model-Guided ISP Hyperparameter Optimization with Dynamic Preference Learning
Xinyu Sun, Zhikun Zhao, Congyan Lang, Bing Li, Juan Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MLLM-generated text and multimodal fusion (MDPM) to image-only ISP optimization (ablation) | MLLM-ISP · COCO detection | mAP 0.5: 0.612 -> 0.653 | Improved |
| add DPG-DPO preference fine-tuning (ablation) | MLLM-ISP · COCO detection | mAP 0.5: 0.634 -> 0.653 | Improved |
| add DPG-DPO preference fine-tuning (ablation) | MLLM-ISP · IMX766 IQA Qualcomm ISP | PSNR: 32.788 -> 34.264 | Improved |
| add text information versus image-only baseline (ablation) | MLLM-ISP · IMX766 IQA Qualcomm ISP | PSNR: 27.265 -> 32.788 | Improved |
| use GPT-4V or mPLUG-Owl-2 instead of Q-Instruct LLaVA-7B for text generation (ablation) | MLLM-ISP · COCO detection | mAP 0.5: 0.653 -> 0.650 (mPLUG-Owl-2), 0.647 (GPT-4V) | No clear effect |
| use a stronger downstream detector (YOLOv8 instead of YOLOv3) (ablation) | MLLM-ISP · COCO detection | mAP 0.5: 0.653 -> 0.675 | Improved |
| use MLLM-ISP instead of RL-SepISP at instance segmentation mAP 0.5 | ISP tuning · COCO instance segmentation | mAP 0.5: 0.630 -> 0.632 | Improved |
Closest papers by meaning
- RL-SeqISP: Reinforcement Learning-Based Sequential Optimization for Image Signal Processing (AAAI 2024 · similarity 0.86)
- Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs (NeurIPS 2025 · similarity 0.86)
- Dynamics-Aware Preference Optimization for Vision-Language Models (CVPR 2026 · similarity 0.86)
- Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization (ICLR 2026 · similarity 0.85)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.84)
- MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
