Research map / Vision-language and multimodal / Human and embodied modeling

Paleoinspired Vision: From Exploring Colour Vision Evolution to Inspiring Camera Design

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Human-like visual cognition in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an opsin-layer model of retinal transduction (1x1 conv with Gaussian-parameterised spectral sensitivities) optimised end-to-end with a segmentation network under conservative regularisation, to simulate colour-vision evolution and design task-specific camera filters.

Problem

Testing long-standing hypotheses in colour-vision evolution is hard with traditional methods, and camera spectral response functions are rarely optimised for specific imaging tasks.

Approach

Model retinal opsins as a 1x1 convolution layer whose kernels are Gaussian spectral sensitivities parameterised by lambda_max only, trained jointly with a MiT segmentation encoder/decoder under a cap of 0.5 nm lambda_max shift per epoch, so that task accuracy acts as an evolutionary pressure; the same layer is reused as a learnable camera filter.

Key results

• Trichromatic vision yields higher fruit-camouflage reconstruction score than dichromatic (0.3161 vs 0.2195 bright light).
• Learned Martian camera filters reach 39.20 mIoU vs 39.15 for general filters, a marginal gain.
• Cancer-detection filter design reaches 53.55 mIoU vs 53.17 for general filters; U-Net variant improves IoU from 42.56 to 44.42.

Surprising findings

• Colour-weak simulated observers outperform normal trichromats on khaki rock segmentation (41.01 vs 40.75 mIoU), and on fruit detection under dim light the normal trichromat is worse than several colour-blind variants.
• Simulated dichromatic vision performs best for hypothetical Martian life, not trichromatic or tetrachromatic.

Limitations

• Gains for camera design are marginal (e.g. 39.15 to 39.20 mIoU on Mars-Seg), and no physical prototype was built.
• Hyperspectral data is reconstructed from RGB rather than measured, and evolutionary claims rest on simplified simulations.
• Bioluminescence experiments show separations of only about 5 nm, far smaller than biological data, so the mechanism is not validated.

Experimental setup

Datasets: LIB-HSI MinneApple VOC2012 Mars-Seg MHSI Choledoch

Benchmarks: MinneApple LIB-HSI Mars-Seg MHSI Choledoch

Metrics: mIoU camouflage score (reconstruction fidelity SR) IoU of cancer

Baselines: General RGB-like filters (580, 540, 425 nm) Standard U-Net filters (590, 540, 460 nm)

Builds on: MiT / SegFormer deep optics minimalist vision with freeform pixels

Closest papers by meaning

• A Computational Framework for Modeling Emergence of Color Vision in the Human Brain (ICLR 2025 · similarity 0.84)
• BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning (NeurIPS 2025 · similarity 0.82)
• Connectome-Based Modelling Reveals Orientation Maps in the Drosophila Optic Lobe (NeurIPS 2025 · similarity 0.82)
• EnColor: Improving Visual Accessibility with a Deep Encoder-Decoder Image Corrector for Color Vision Deficient Individuals (AAAI 2024 · similarity 0.81)
• Visual Pinwheel Centers Act as Geometric Saliency Detectors (NeurIPS 2024 · similarity 0.81)
• BioCLIP: A Vision Foundation Model for the Tree of Life (CVPR 2024 · similarity 0.81)