Research map / Video / Video generation
CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRePE, a UCM-compatible expected-RoPE positional encoding that integrates a learned log-radial distance distribution along curved projected camera rays, implemented as a lightweight adapter on a frozen Wan2.1 video DiT for camera-controlled generation, with an external radial-map interface (Radial MixForcing).
Problem
Camera-conditioned video generation needs positional encodings that stay reliable under wide-angle and fisheye (UCM) lenses; existing ray-only or pinhole-based attention encodings do not model where scene content lies along each ray.
Approach
Represent each key token as a depth-aware uniform distribution over log radial distance along its UCM ray, project breakpoints into each query camera, and average the RoPE phasor analytically over the resulting curved path, with pseudo radial supervision from UniK3D used as a training anchor.
Key results
- On the merged PanShot set, CRePE reduces CamMC from 21.87 (UCPE) to 18.15 and translation error from 17.54 to 13.78, while UCPE stays slightly better on FVD (463.5 vs 465.3).
- Supervision-free breakpoint ablation: K=3 reduces CamMC from 14.75 (endpoint K=2) to 12.31; K=5 gives best average rank (1.71).
- On external radial-map adherence on DAVIS-2016, CRePE-Mix achieves AbsRel 0.2719 and scale jitter 0.1781, best among compared methods including VACE and OmniVDiff.
Surprising findings
- Depth-Concat, which gives the model the same pseudo radial signal as features, underperforms even depth-free UCPE on pose, indicating that radial distance must enter attention as a query-dependent phase.
- Under the affine-path analysis, subdividing an affine phase gives no benefit, so K-dependent gains must come from non-affine phase; pinhole TransErr actually worsens at K=5 (+4.1%).
Limitations
- Main-table results are single-run point estimates without multi-seed uncertainty.
- Pose and radial metrics rely on fixed estimators (ViPE, GeoCalib, UniK3D) that may carry shared bias; supervision uses monocular pseudo-labels, not metric moving-object depth.
- Covers only central UCM cameras, not non-central or rolling-shutter models.
Experimental setup
Datasets: PanShot UniK3D pseudo radial maps KITTI-360 RealCam-Vid
Benchmarks: PanShot KITTI-360 RealCam-Vid DAVIS-2016 VBench
Metrics: CamMC RotErr TransErr k1 k2 pitch gravity up-vector error FVD FID
Baselines: ReCamMaster UCPE RayRoPE-style endpoint PE Depth-Concat ControlVideo Ctrl-Adapter Make-Your-Video OmniVDiff VACE
Builds on: UCPE RoPE RayRoPE Wan2.1 UniK3D
Closest papers by meaning
- Cameras as Relative Positional Encoding (NeurIPS 2025 · similarity 0.86)
- Imagine360: Immersive 360 Video Generation from Perspective Anchor (NeurIPS 2025 · similarity 0.84)
- CRAYM: Neural Field Optimization via Camera RAY Matching (NeurIPS 2024 · similarity 0.83)
- IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation (NeurIPS 2025 · similarity 0.83)
- World Tracing: Pixel-Aligned Geometry Beyond the Visible (NeurIPS 2026 · similarity 0.83)
- AKiRa: Augmentation Kit on Rays for Optical Video Generation (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.