Research map / Vision-language and multimodal / Human and embodied modeling
A Computational Framework for Modeling Emergence of Color Vision in the Human Brain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Simulates a biophysically based retina (random cone mosaic, foveation, lateral inhibition) and trains a bio-plausible cortical model via self-supervised prediction of optic nerve signal fluctuations under fixational eye drift, showing N-cone retinas yield N-dimensional color vision verified with a color-matching test simulator.
Problem
How the brain infers the correct color dimensionality from optic nerve signals alone, without knowing the eye's unknown spectral encoding properties or assuming a fixed dimensionality such as RGB.
Approach
Model cortical color as an N-dimensional vector space and learn decoder/translator/encoder functions plus retinal "neural buckets" (cone types, positions, lateral inhibition weights) by predicting the next optic nerve signal under random fixational eye motion; color dimensionality then emerges and is measured with a black-box color-matching simulator (CMF-Sim) and a learnable projection to RGB (Ne
Key results
- Cortical models trained on retinas with 1, 2, 3, or 4 cone types converge to mono-, di-, tri-, and tetrachromatic color vision, formally measured with CMF-Sim.
- A control with cortical learning removed measures 1-D color despite a 3-cone retina, showing learning is necessary.
- Simulated gene therapy on a dichromat retina (adding L opsin) re-converges to 3-D color after continued cortical learning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use self-supervised cortical learning rather than directly treating optic nerve signals as percepts | Trichromat retina model · Trichromat retina | color dimensionality: 1-D to 3-D | Helped |
| Add demosaicking network D (ablation) | Proposed cortical model · Trichromat retina | color dimensionality: 1D to 3-D | Helped |
| Use the proposed cortical learning mechanism with a retina containing K cone types | Cortical model · Simulated retinas with 1, 2, 3, or 4 cone types | color dimensionality: 1, 2, 3, or 4 cone types -> 1D, 2D, 3D, or 4D | Helped |
| Continue self-supervised cortical learning after introducing L opsin into a subset of cones | Adult protanopic squirrel-monkey model · Simulated gene therapy, dichromacy to trichromacy | color dimensionality: 2D to 3-dimensional | Helped |
| Add a fourth cone type and allow cortical learning to re-converge | Trichromat model · Simulated gene therapy, trichromacy to tetrachromacy | color dimensionality: 3D to 4D | Helped |
Surprising findings
- Without cortical learning, a trichromat retina's optic nerve signal yields only 1-D color under color-matching tests.
- Simulation suggests adding a fourth cone type to a normal trichromat could boost color to 4-D tetrachromacy.
Limitations
- Simulated retina is a textbook model; the paper acknowledges it is an existence proof rather than a full biological account.
- Foveal tritanopia omitted from the eye simulation.
- Neural Scope not applicable to 4-D case; tetrachromacy only verified formally via CMF-Sim.
Experimental setup
Datasets: 900 hyperspectral everyday-scene images (31 channels)
Benchmarks: CMF-Sim color matching tests against human psychophysical data
Metrics: minimum number of primaries (color dimensionality) perceptual matching error
Baselines: no-cortical-learning baseline cone-activation-as-percept baseline no-demosaicking ablation
Builds on: U-Net normalizing flow Difference-of-Gaussians lateral inhibition model Maxwellian color matching
Closest papers by meaning
- Color Visual Illusions: A Statistics-based Computational Model (NeurIPS 2020 · similarity 0.89)
- A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs (ICLR 2019 · similarity 0.88)
- From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction (NeurIPS 2019 · similarity 0.87)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.85)
- Neuro-3D: Towards 3D Visual Decoding from EEG Signals (CVPR 2025 · similarity 0.84)
- The Art of Deception: Color Visual Illusions and Diffusion Models (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.