Research map / 3D vision / 3D generation

At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections

NeurIPS 2026 · 3D vision › 3D generation · approach: 2D-to-3D pretrained representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FullTilt, a DETR-like end-to-end open-set detector that localizes 3D macromolecules directly from aligned 2D tilt-series, avoiding slow 3D sliding-window inference.

Problem

Open-set 3D macromolecule detection in cryo-ET is slow because VRAM limits force sliding-window inference over subvolumes or 2D slices of reconstructed tomograms, and tomograms contain redundant information relative to the tilt-series.

Approach

Operate directly on the aligned 2D tilt-series with a tilt-series encoder that uses alternating local attention and global row attention (exploiting the y-aligned geometry), a multiclass visual prompt encoder, a tilt-aware query initializer that anchors 3D queries from the near-0° tilt, and an auxiliary on-the-fly geometric primitives module.

Key results

• On CZII zero-shot intra-instance with 4 prompts, FullTilt reaches mAP@1r 0.180 and F1 0.221 versus TomoTwin's 0.105 and 0.102, at 0.510 s runtime and 3430 MB VRAM.
• On EMPIAR-10499 with 4 prompts, FullTilt achieves mAP@0.5r 0.173 and F1 0.441 in cross-instance detection, outperforming TomoTwin (0.064 and 0.272) at roughly 0.39 s runtime.
• Ablation shows that removing the tilt-series encoder drops performance dramatically (e.g., CZII mAP@0.5r 0.040 to 0.099 with E_Tilt added alongside I_Tilt), and full model with all three modules is best.

Surprising findings

• Naive adaptations of DETR3D and PETR for multi-view tilt-series perform poorly, with PETR near zero mAP.
• A single-image prompt setting only slightly degrades performance compared with prompting in all tilt images.

Limitations

• Struggles with particularly small macromolecules, a known DETR-like limitation.
• Suboptimal localization for highly non-globular particles due to single-diameter formulation.
• Performance drops substantially when the number of tilt images is reduced (e.g., 12.5% of tilts).

Experimental setup

Datasets: simulated cryo-ET tomograms and tilt-series (1,138, 119 proteins) CZII CryoET Object Identification Challenge EMPIAR-10304 EMPIAR-10499

Benchmarks: CZII EMPIAR-10304 EMPIAR-10499 simulated dataset with 10 unseen proteins

Metrics: mAP@0.5r mAP@1r F1 runtime (s) peak VRAM (MB)

Baselines: TomoTwin ProPicker CryoSAM Zeng et al. DETR3D PETR TomoPicker Huang et al.

Builds on: DETR DETR3D PETR Grounding DINO T-Rex2 Swin Transformer

Closest papers by meaning

• BOE-ViT: Boosting Orientation Estimation with Equivariance in Self-Supervised 3D Subtomogram Alignment (CVPR 2025 · similarity 0.88)
• CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference (NeurIPS 2024 · similarity 0.86)
• CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM (NeurIPS 2024 · similarity 0.86)
• cryoSPHERE: Single-Particle HEterogeneous REconstruction from cryo EM (ICLR 2025 · similarity 0.85)
• Vox-UDA: Voxel-wise Unsupervised Domain Adaptation for Cryo-Electron Subtomogram Segmentation with Denoised Pseudo-Labeling (AAAI 2025 · similarity 0.85)
• CryoFM: A Flow-based Foundation Model for Cryo-EM Densities (ICLR 2025 · similarity 0.84)