Research map / Vision-language and multimodal / Human and embodied modeling
Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a DQN-based post-hoc RL framework to explore and summarize failure landscapes of pretrained vision, NLP, and text-to-image models, then uses a few human feedback points to fine-tune models away from discovered failure modes.
Problem
Exhaustively testing deep vision and language models for failures across combinations of factors is infeasible, and existing failure-finding methods do not characterize the whole failure landscape or help mitigate it.
Approach
Frames failure discovery as a deep RL problem over an actionable concept space (macroscopic and microscopic exploration with DQN), summarizes discovered failures with Q-value-based probability landscapes and Wasserstein barycenters, and restructures the model by fine-tuning on failure-free or user-preferred data guided by a few human feedback points.
Key results
- On AlexNet, DQN found 499 failures versus 33 for random search and 44 for threshold search, with lower entropy (5.69 vs 6.92).
- Fine-tuning Stable Diffusion v1-4 with LoRA on DALL-E 3 data reduced the male-to-female bias ratio from 1.65 to 1.16 and cut ambiguous image generation by 43%.
- Human feedback and CLIP-based rewards yielded similar learned policies, with a reported Wasserstein distance of 0.0011 between reward distributions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DQN instead of the strongest listed search baseline | Image classification failure discovery · ImageNet-1K, AlexNet | failure count: 44 -> 499 | Helped |
| Use DQN instead of the strongest listed search baseline | Image classification failure discovery · ImageNet-1K, ResNet | failure count: 22 -> 153 | Helped |
| Use DQN instead of the strongest listed search baseline | Image classification failure discovery · ImageNet-1K, EfficientNet | failure count: 22 -> 109 | Helped |
| Use DQN instead of the strongest listed search baseline | Summarization failure discovery · Summarize-from-Feedback, BART | failure count: 42 -> 109 | Helped |
| Use DQN instead of the strongest listed search baseline | Summarization failure discovery · Summarize-from-Feedback, T5 | failure count: 35 -> 83 | Helped |
| Use DQN instead of the strongest listed search baseline | Image-generation failure discovery · Stable Diffusion v1-4 | failure count: 31 -> 85 | Helped |
| Use DQN instead of the lowest-entropy listed search baseline | Failure discovery · AlexNet, ResNet, EfficientNet, BART, T5, and Stable Diffusion v1-4 | entropy: 6.91 -> 5.69; 6.79 -> 5.88; 6.79 -> 5.99; 3.80 -> 3.20; 3.70 -> 3.10; 4.25 -> 3.65 | Helped |
| Fine-tune the failure-selected BART model | BART summarization · Summarize-from-Feedback | reward: 13584.5 to -10077 | Helped |
| Fine-tune the failure-selected T5 model | T5 summarization · Summarize-from-Feedback | reward: 115317.21 to -276394 | Helped |
| LoRA fine-tune on balanced male and female generated images | Stable Diffusion v1-4 · Bias prompts | male-to-female bias ratio: 1.65 to 1.16 | Helped |
| LoRA fine-tune on balanced male and female generated images | Stable Diffusion v1-4 · Bias prompts | ambiguous image generation: 43% drop | Helped |
| Use CLIP embeddings instead of human feedback for reward | Stable Diffusion v1-4 failure discovery · Human survey reward comparison | Wasserstein distance: 0.0011 | No clear effect |
Surprising findings
- Fine-tuning on the single highest-failure action can shift failures to other actions rather than eliminate them, and for AlexNet the reward even increased after fine-tuning due to its low accuracy.
- CLIP-based rewards were found to be an effective stand-in for human feedback in driving the RL policy.
Limitations
- Fine-tuning on one failure mode can shift failures to other modes; authors note the need to unify fine-tuning approaches.
- Requires domain-specified concepts and reward functions defined by humans; the method is not applicable to LLMs and generative models in the Wasserstein analysis.
- Human feedback uses a small number of participants (50) and a limited number of instances.
Experimental setup
Datasets: ImageNet-1K openai/summarize_from_feedback DALL-E 3 generated images
Benchmarks: ImageNet-1K openai/summarize_from_feedback
Metrics: failure count entropy of failure distribution Wasserstein distance BLEU male-to-female bias ratio running time
Baselines: Random search Greedy search Threshold search FGSM adversarial training Bayesian Optimization
Builds on: deep Q-network Bradley-Terry model LoRA FGSM
Closest papers by meaning
- Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search (ICLR 2024 · similarity 0.86)
- ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.86)
- Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.86)
- FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges (NeurIPS 2025 · similarity 0.86)
- TTRV: Test-Time Reinforcement Learning for Vision Language Models (CVPR 2026 · similarity 0.86)
- A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.