Research map / Adversarial robustness and security / Certified and empirical robustness

Representation-Based Robustness in Goal-Conditioned Reinforcement Learning

AAAI 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a semi-contrastive representation (SCR) attack that needs only the policy encoder, plus ARTs defenses (semi-contrastive adversarial augmentation and a sensitivity-aware regularizer on SimSR) to improve robustness of goal-conditioned RL agents on Fetch tasks.

Problem

Goal-conditioned RL agents are vulnerable to adversarial perturbations of states and goals, and existing RL attacks and robust representation methods transfer poorly to GCRL's sparse binary reward setting.

Approach

A negative-tuple-only adversarial contrastive attack (SCR) that perturbs state/goal inputs to push the encoder representation away from the clean tuple toward negated tuples, requiring no critic or pseudo-labels; defended by SCAA augmentation and a SimSR-based sensitivity-aware Lipschitz regularizer.

Key results

• SCR-PGD reduces DDPG discounted return by 47.17% on FetchPick, 57.69% on FetchPush, 93.78% on FetchReach and 100% on FetchSlide relative to nature returns.
• ARTs (SCAA) raises DDPG under best attack from 7.83 to 14.53 on FetchPick and from 1.86 to 27.68 on FetchReach.
• SimSR+SAR for DDPG reaches 16.78 on FetchPick and 12.63 on FetchPush under attack, comparable to nature returns.

Measured effects reported in the paper

Surprising findings

• GoFar with SimSR is more robust than the attack expects, and the attack is less effective against GoFar, with GCSL on FetchReach dropping only 4.46%.
• Under SCR attacks the reductions in return for GoFar on FetchSlide reach 100%, yet ARTs gains on GoFar are inconsistent across tasks.

Limitations

• Experiments limited to four Fetch manipulation tasks in simulation.
• ARTs gains are inconsistent: GoFar with SCAA does not improve on several tasks and the paper notes weaker defense for GoFar with SimSR.
• Results are averaged over only 5 seeds with high variance in several cells.

Experimental setup

Datasets: FetchPick offline dataset FetchPush offline dataset FetchReach offline dataset FetchSlide offline dataset

Benchmarks: FetchPick FetchPush FetchReach FetchSlide

Metrics: discounted return under attack

Baselines: DDPG GCSL GoFar SA-FGSM SA-PGD Uniform noise SimSR

Builds on: DDPG GCSL GoFar SimSR MICo

Closest papers by meaning

• Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies (ICLR 2024 · similarity 0.88)
• Adversarial Diffusion for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations (NeurIPS 2025 · similarity 0.88)
• Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL (ICLR 2024 · similarity 0.87)
• Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula (ICLR 2024 · similarity 0.87)
• Diffusion Guided Adversarial State Perturbations in Reinforcement Learning (NeurIPS 2025 · similarity 0.87)