Research map / Image generation / Autoregressive and latent generation
Bridging Protein Sequences and Microscopy Images with Unified Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CELL-Diff, a unified continuous (image) plus discrete (sequence) diffusion transformer-U-Net that bidirectionally generates fluorescence protein microscopy images from sequences and sequences from images, trained on HPA and fine-tuned on OpenCell.
Problem
Linking protein amino acid sequences to their subcellular localization as seen in fluorescence microscopy, with bidirectional generation; prior sequence-to-image generators (CELL-E, CELL-E2) produce blurry, coarse images.
Approach
Jointly train a latent continuous diffusion model for protein images and an order-agnostic masked discrete diffusion (OA-ARDM) for sequences within one transformer-based U-Net with ESM2 sequence embeddings and cross-attention, so both conditional directions are learned together.
Key results
- On HPA with nucleus image, CELL-Diff reaches MSFR 641 nm and IoU 0.484 vs CELL-E2 MSFR 1824 nm and IoU 0.462; FID-O 51.1 vs 166.4.
- Adding ER and microtubule cell images raises HPA IoU to 0.635 (nucleus, ER, microtubule) with MSFR 644 nm.
- On OpenCell, CELL-Diff gives MSFR 529 nm vs 1165 nm for CELL-E2 and FID-O 68.9 vs 245.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CELL-E2 with CELL-Diff using nucleus conditioning | Protein image generation · HPA | FID-O: 166.4 -> 51.1 | Helped |
| Replace CELL-E2 with CELL-Diff using nucleus conditioning | Protein image generation · OpenCell | FID-O: 245.8 -> 68.9 | Helped |
| Add ER image conditioning | CELL-Diff with nucleus conditioning · HPA | IoU: 0.484 -> 0.580 | Helped |
| Add microtubule image conditioning | CELL-Diff with nucleus conditioning · HPA | IoU: 0.484 -> 0.616 | Helped |
| Add ER and microtubule image conditioning | CELL-Diff with nucleus conditioning · HPA | IoU: 0.484 -> 0.635 | Helped |
| Add cross-attention (ablation) | CELL-Diff · HPA | IoU: 0.434 -> 0.484 | Helped |
| Add cross-attention (ablation) | CELL-Diff · HPA | MSFR (nm): 687 -> 641 | Helped |
| Add cross-attention (ablation) | CELL-Diff · HPA | FID-T: 55.8 -> 60.1 | Hurt |
| Add cross-attention (ablation) | CELL-Diff · HPA | FID-O: 41.3 -> 51.1 | Hurt |
Limitations
- Evaluation limited to a 100-protein test set due to dataset size and sequence diversity.
- Relies on fixed-stain HPA and live-cell OpenCell data; biological validation of generated sequences and applications is qualitative.
- Only one baseline (CELL-E2) compared quantitatively.
Experimental setup
Datasets: Human Protein Atlas (HPA) OpenCell UniProt
Benchmarks: HPA test set (714 data points) OpenCell test set (473 data points)
Metrics: MSFR IoU FID-T FID-O
Baselines: CELL-E2
Builds on: Stable Diffusion DiT (adaLN-Zero) OA-ARDM CELL-E2 UniDiffuser
Closest papers by meaning
- Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.86)
- PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protein-Protein Complex Design (ICML 2025 · similarity 0.85)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.85)
- MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays (NeurIPS 2026 · similarity 0.85)
- DPLM-2: A Multimodal Diffusion Protein Language Model (ICLR 2025 · similarity 0.85)
- 4D Diffusion for Dynamic Protein Structure Prediction with Reference and Motion Guidance (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.