Research map / Image generation / Autoregressive and latent generation

Stage Light is Sequence$^2$: Multi-Light Control via Imitation Learning

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Controllable music generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SeqLight, a hierarchical method that predicts full-light hue/value distributions with a customized Skip-BART and then decomposes them into per-light controls via goal-conditioned imitation learning (BC, AIRL, GRPO) trained on HER-style expert data from mixed lights.

Problem

Automatic stage lighting control has been limited to single primary light color prediction or to music-to-DMX mappings that do not transfer across venues with different light setups and need costly professional data.

Approach

Decouple multi-light control into a venue-independent music-to-full-light-distribution predictor (Skip-BART adapted to hue/value histograms) and a venue-specific goal-conditioned decomposition policy trained with a three-phase IL pipeline (BC, AIRL reward learning, GRPO fine-tuning) on HER-relabeled expert trajectories built from mixed light simulations, requiring no professional demonstrations.

Key results

• In the human study with 30 participants, SeqLight scored 4.54 ± 0.88 overall on the in-domain group versus 3.90 ± 0.84 for Skip-BART and 2.70 ± 1.26 for the rule-based method.
• On the out-of-domain group (Suno-generated music), SeqLight scored 3.94 ± 1.32 overall versus 3.47 ± 1.01 for Skip-BART.
• In the quantitative decomposition study, Phase 3 (GRPO) gives the best OOD results on several hue metrics, while Phase 2 (GRPO) is best in-domain on value metrics.

Surprising findings

• Phase 3 fine-tuning degrades in-domain performance relative to Phase 2, which the authors attribute to overfitting to the original domain.
• PPO did not improve in Phase 3, while GRPO did, which the authors attribute to critic–reward-model interference.

Limitations

• Experiments are conducted in a simulated multi-light environment rather than real venues.
• Light positions and control order are assumed fixed in advance; saturation fixed at 100%, so only pure colors are considered.
• Phase 3 fine-tuning reduces in-domain performance, indicating a generalization–overfitting trade-off.

Experimental setup

Datasets: RPMC-L2

Benchmarks: RPMC-L2 test split

Metrics: L1 Wasserstein distance JS divergence KL divergence Bhattacharyya distance cosine similarity human rating (1-7)

Baselines: Skip-BART rule-based method PPO-based Phase 2 Ground Truth

Builds on: Skip-BART AIRL GRPO Hindsight Experience Replay PPO

Closest papers by meaning

• Learning Adaptive Lighting via Channel-Aware Guidance (ICML 2025 · similarity 0.82)
• ScribbleLight: Single Image Indoor Relighting with Scribbles (CVPR 2025 · similarity 0.82)
• Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Dataset (CVPR 2025 · similarity 0.82)
• LightIt: Illumination Modeling and Control for Diffusion Models (CVPR 2024 · similarity 0.82)
• LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting (CVPR 2025 · similarity 0.81)
• DiffusionLight: Light Probes for Free by Painting a Chrome Ball (CVPR 2024 · similarity 0.81)