Research map / Architectures and efficiency / Attention and transformers
Binarized Mamba-Transformer for Lightweight Quad Bayer HybridEVS Demosaicing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BMTNet, a binarized hybrid Mamba-Swin Transformer with a binarized global visual encoder for lightweight Quad Bayer HybridEVS demosaicing, binarizing Mamba projections while keeping the selective scan full precision.
Problem
Demosaicing Quad Bayer RAW images from Hybrid event-based vision sensors (HybridEVS) is hard due to the Quad Bayer CFA and event-pixel color loss, and existing long-range models are too computationally heavy for mobile deployment.
Approach
Binarize all projections of a Mamba block (Bi-Mamba) while keeping the Selective Scan in full precision, combine it in parallel with a binarized Swin Transformer, and inject global visual embeddings from a binarized visual encoder (distilled from RAM) into the SSM control matrix B.
Key results
- BMTNet reaches 35.52 dB average PSNR over six image datasets with 1.28M params and 6.56G OPs, the best among BNNs (BBCU 34.85, ReActNet 35.08).
- On video (REDS, Vid4) BMTNet averages 37.70 dB, outperforming all BNNs by over 0.6 dB.
- Bi-Mamba cuts parameters by 79% and OPs by 88% versus FP Mamba, at a 0.79 dB PSNR drop (36.95 vs 37.73).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BMTNet instead of the strongest non-BMTNet binary baseline | Quad Bayer HybridEVS demosaicing · MIPI, Kodak, McM, BSD100, Urban100, Wed | PSNR (dB) | Mixed |
| Use BMTNet instead of the strongest non-BMTNet binary baseline | Quad Bayer HybridEVS demosaicing · REDS, Vid4 | PSNR (dB) | Mixed |
| Remove Mamba (ablation) | BMTNet · MIPI | PSNR (dB): 36.95 -> 35.17 | Hurt |
| Replace Bi-Mamba with full-precision Mamba (ablation) | BMTNet · MIPI | PSNR (dB): 36.95 -> 37.73 | Helped |
| Add global visual embeddings at the B control matrix (ablation) | Bi-Mamba · MIPI | PSNR (dB): 36.90 -> 36.95 | Helped |
| Apply global visual embeddings at all control-matrix positions instead of only B (ablation) | Bi-Mamba · MIPI | PSNR (dB): 36.95 -> 36.89 | Hurt |
| Apply global visual embeddings to C instead of B (ablation) | Bi-Mamba · MIPI | PSNR (dB): 36.95 -> 36.91 | Hurt |
| Apply global visual embeddings to Δ instead of B (ablation) | Bi-Mamba · MIPI | PSNR (dB): 36.95 -> 36.83 | Hurt |
Surprising findings
- Embedding global visual information into the Delta matrix or all control matrices slightly hurts performance, while embedding into B helps.
Limitations
- Full-precision upsampling/downsampling and the selective scan remain in higher precision, so the model is not fully binarized.
- Slightly higher OPs (0.44G) than the smallest BNN baseline BTM.
- Evaluation on real HybridEVS data is qualitative only, with no ground truth.
Experimental setup
Datasets: MIPI 2024 demosaicing dataset Kodak McM BSD100 Urban100 Wed REDS Vid4
Benchmarks: MIPI Kodak McM BSD100 Urban100 Wed REDS Vid4
Metrics: PSNR SSIM Params OPs
Baselines: BNN ReActNet BBCU BTM BiViT DFormer NAFNet Restormer SAGAN PIPNet CycleISP
Builds on: Mamba BBCU ReActNet BiViT RAM
Closest papers by meaning
- MambaVision: A Hybrid Mamba-Transformer Vision Backbone (CVPR 2025 · similarity 0.87)
- MobileMamba: Lightweight Multi-Receptive Visual Mamba Network (CVPR 2025 · similarity 0.85)
- MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods (ICLR 2025 · similarity 0.85)
- MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining (CVPR 2025 · similarity 0.85)
- MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imaging (NeurIPS 2024 · similarity 0.85)
- Demystify Mamba in Vision: A Linear Attention Perspective (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.