Research map / Architectures and efficiency / RNNs and normalization
AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching
Pengfei ZHANG, Tianxin Xie, Minghao Yang, Li Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align FoG-A-selected top-3 layers instead of the best single fixed layer (REPA @ L8) | AG-REPA on DiT flow matching · Speech FAD, Config B | FAD: 1.58 -> 1.29 | Improved |
| add attribution-weighted sparse top-3 selection on top of Shallow REPA (L1-L3) (ablation) | AG-REPA · Speech FAD, Config B | FAD: 1.36 -> 1.29 | Improved |
| align deep representation-rich layers (L20-L22) (ablation) | REPA · Speech FAD, Config B | FAD: 1.84 -> 1.79 | No clear effect |
| align random layers (L5, L14, L19) (ablation) | REPA · Speech FAD, Config B | FAD: 1.84 -> 1.75 | No clear effect |
| select layers by highest LASP score instead of random (ablation) | REPA layer selection · Speech FAD, Config B | FAD: 1.75 -> 1.68 | Improved |
| select layers by FoG-A instead of Gradient Norm (ablation) | AG-REPA layer selection · Speech FAD / Audio FAD | FAD: 1.35/2.71 -> 1.29/2.56 | Improved |
| select layers by FoG-A instead of LASP (ablation) | AG-REPA layer selection · Steps to reach Speech FAD 1.5 | convergence steps: 720k -> 220k | Improved |
| apply AG-REPA to Voicebox | Voicebox · Voicebox | FAD: 1.20 -> 0.95 | Improved |
| apply AG-REPA to CosyVoice | CosyVoice · CosyVoice | FAD: 0.88 -> 0.72 | Improved |
| apply AG-REPA to F5-TTS | F5-TTS · F5-TTS | FAD: 1.45 -> 1.15 | Improved |
| re-run FoG-A per architecture instead of hard-coding shallow layers L1-L3 (ablation) | AG-REPA vs Shallow REPA · F5-TTS | FAD: 1.34 -> 1.15 | Improved |
| continuously refresh layer set S every epoch instead of static selection (ablation) | AG-REPA · Speech/Audio FAD | FAD: 1.29/2.56 -> 1.28/2.55 | No clear effect |
Closest papers by meaning
- UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities (NeurIPS 2026 · similarity 0.89)
- Native Audio-Visual Alignment for Generation (NeurIPS 2026 · similarity 0.88)
- Foley-Flow: Coordinated Video-to-Audio Generation with Masked Audio-Visual Alignment and Dynamic Conditional Flows (CVPR 2025 · similarity 0.88)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.87)
- Model-Guided Dual-Role Alignment for High-Fidelity Open-Domain Video-to-Audio Generation (NeurIPS 2025 · similarity 0.87)
- Asymmetric Hierarchical Anchoring for Robust Audio–Visual Cross-Modal Generalization (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
