Research map / Image generation / Autoregressive and latent generation
Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Manipulating Human Perceptual Variability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Samples images along ANN decision boundaries using classifier-guided diffusion, collects human digit-recognition data to build variMNIST, fine-tunes classifiers to predict individual human variability, and generates dual-participant controversial stimuli to steer divergent choices.
Problem
Human perceptual variability across individuals on simple visual decision tasks (e.g., handwritten digit classification) is under-studied, and it is unclear whether ANN decision boundaries can induce and predict such variability and be used to manipulate individual perceptual decisions.
Approach
Generate stimuli along ANN perceptual boundaries using classifier-guided diffusion (uncertainty and controversial guidance with a diffusion prior), label them with large-scale human behavior, fine-tune per-person classifiers to predict individual responses, and use pairs of individualized models as adversarial generators to produce stimuli that push two participants toward different answers.
Key results
- Individual fine-tuning (IndivNet) improved over BaseNet by about 20% on variMNIST and about 5% over GroupNet on variMNIST-i; accuracy rose for 241 of 246 participants.
- For VGG, Spearman correlation between model and human entropy rose from 0.08 to 0.74 after group fine-tuning.
- In the dual-participant manipulation study (18 participants, 18 pairs), individualized stimuli raised success rate by about 3%, reduced failure by about 4%, and improved targeted ratio by about 12% over variMNIST.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the classifier with group-level human behavioral data | BaseNet · variMNIST | accuracy: ~20% | Helped |
| Fine-tune the group-aligned model with individual behavioral data | GroupNet · variMNIST-i | accuracy: ~5% | Helped |
| Fine-tune the model individually using participant behavioral data | BaseNet · variMNIST-i | accuracy across participants: 241 participants; 5 participants | Mixed |
| Fine-tune VGG with group-level human behavioral data | VGG · Human entropy prediction | Spearman rank correlation: ρ=0.08 to ρ=0.74 | Helped |
| Fine-tune classifiers with group-level human behavioral data | VIT, VGG, CORNet, MLP, and LRM · MNIST | accuracy | Mixed |
| Fine-tune classifiers with group-level human behavioral data | VIT, VGG, CORNet, MLP, and LRM · variMNIST | accuracy | Helped |
| Fine-tune classifiers with individual-level human behavioral data | VIT, VGG, CORNet, MLP, and LRM · variMNIST-i | accuracy | Helped |
| Use individually customized stimuli and IndivNets instead of variMNIST stimuli | Individual manipulation framework · Human guidance outcomes | guidance success rate: increased by 3% | Helped |
| Use individually customized stimuli and IndivNets instead of variMNIST stimuli | Individual manipulation framework · Human guidance outcomes | guidance bias rate: increased by 1% | Hurt |
| Use individually customized stimuli and IndivNets instead of variMNIST stimuli | Individual manipulation framework · Human guidance outcomes | guidance failure rate: decreased by 4% | Helped |
| Use individually customized stimuli and IndivNets instead of variMNIST stimuli | Individual manipulation framework · Successful human guidance trials | targeted ratio: improved by 12% | Helped |
| Use individually fine-tuned models to generate second-round stimuli | Individual manipulation framework · Human manipulation experiment | guidance success rate: ~3% (p < 0.001) | Helped |
Surprising findings
- Eliciting human perceptual variability was harder for handwritten digits than for natural images, since human observers agree strongly on digit classification.
Limitations
- Dataset is limited to digit/object recognition and may not capture cultural or other sources of human variability.
- Models are trained on MNIST-style data from specific populations, limiting generality.
- Individual manipulation tested only on 18 participants in small-scale in-lab experiments.
Experimental setup
Datasets: MNIST variMNIST variMNIST-i ImageNet
Benchmarks: MNIST variMNIST variMNIST-i
Metrics: accuracy Spearman rank correlation between model and human entropy guidance outcome success/bias/failure rate targeted ratio
Baselines: BaseNet GroupNet random stimuli
Builds on: classifier guidance DDPM controversial stimuli model metamers DreamSim
Closest papers by meaning
- When does perceptual alignment benefit vision representations? (NeurIPS 2024 · similarity 0.87)
- Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.87)
- Evaluating alignment between humans and neural network representations in image-based learning tasks (NeurIPS 2024 · similarity 0.87)
- Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.86)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.86)
- Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.