Research map / Reinforcement learning / Visual and representation RL

Fourier Guided Adaptive Adversarial Augmentation for Generalization in Visual Reinforcement Learning

AAAI 2025 · Reinforcement learning › Visual and representation RL · approach: Robust visual encoders for RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FGA3, a frequency-domain adversarial augmentation for visual RL that perturbs only Fourier amplitude (keeping phase) with a value-loss and cosine-similarity worst-case objective, improving generalization on DMControl-GB and Procgen.

Problem

Visual RL agents degrade in unseen environments due to domain gaps; existing augmentations either break semantic consistency or (for adversarial methods like PAADA) rely on policy objectives or need extra models.

Approach

Apply adaptive adversarial perturbations only to the Fourier amplitude (phase preserved for semantics), optimized by gradient ascent on a Lagrangian worst-case objective combining TD/value loss with cosine-similarity distance in the encoder's latent space, without training an extra generator.

Key results

• SAC+FGA3 improved over SAC baseline by 52.6% on Random Colors and 69.6% on Natural Videos in DMControl-GB.
• On Procgen, DrAC+FGA3 achieved average rank 2.88 versus 4.81 for DrAC (lower is better), the best among compared methods.
• FGA3 with SAC scored 757 on Random Colors and 646 on Natural Videos, outperforming SRM-based SAC (682 and 527).

Measured effects reported in the paper

Surprising findings

• Mixup substantially reduced performance in visual RL (avg 173 vs 496 for NoAug), attributed to superposing two states.
• Applying perturbation to the phase component decreased performance relative to baseline in the ablation.

Limitations

• PAADA and RADIAL/SAR were excluded or reimplemented due to unavailable code, complicating comparisons.
• Performance gains depend on the number of perturbation iterations K; K=8 gave best results but adds compute cost.
• Evaluated only on DMControl-GB and Procgen with a limited set of backbone algorithms.

Experimental setup

Datasets: DMControl-GB Procgen

Benchmarks: DMControl-GB (Random Colors, Natural Videos) Procgen

Metrics: average episode return average rank

Baselines: SAC PAD DrQ SVEA SRM PPO RAD MixStyle DrAC RADIAL SAR PAADA

Builds on: Volpi et al. worst-case domain generalization SAC DrQ SVEA PPO DrAC SRM

Closest papers by meaning

• Prompt-based Visual Alignment for Zero-shot Policy Transfer (ICML 2024 · similarity 0.87)
• VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
• Diffusion Guided Adversarial State Perturbations in Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation (NeurIPS 2025 · similarity 0.85)
• Unified Reinforcement and Imitation Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.85)
• Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization (NeurIPS 2024 · similarity 0.85)