Research map / Architectures and efficiency / RNNs and normalization
Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FCNet, a Fourier-domain sequence controller that uses a causal spectral convolution with FFT-based parallel training and sliding-DFT recurrent inference to get low-latency, data-efficient robot policies.
Problem
Transformer-based robot policies trained on trajectories suffer from low data efficiency and high inference latency, making them unsuitable for real-time control on robots.
Approach
Model windows of robot states in the frequency domain, keeping only the m lowest DFT modes with a learned complex linear map (causal spectral convolution), and exploit FFT convolution for parallel training and sliding DFT for O(m) recurrent inference.
Key results
- On D4RL-Mujoco, FCNet averages 75.1 versus 74.7 for DT and 63.6 for DT-RetNet.
- On D4RL-Adroit, FCNet averages 41.0 versus 27.8 for DT, and beats DT on 9 of 12 tasks.
- FCNet's CPU inference latency stays below 20ms and is 3 to 5 times faster than Transformer at 32 layers or hidden size 2048.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the Transformer architecture with FCNet | D4RL-Mujoco offline RL · D4RL-Mujoco | average return: 74.7 -> 75.1 | Helped |
| replace DT-RetNet with FCNet | D4RL-Mujoco offline RL · D4RL-Mujoco | average return: 63.6 -> 75.1 | Helped |
| replace DT with FCNet | D4RL-Adroit · D4RL-Adroit | average return: 27.8 -> 41.0 | Helped |
| replace CQL with FCNet | D4RL-Adroit · D4RL-Adroit | average return: 40.3 -> 41.0 | Helped |
| use FCNet instead of Transformer | multi-environment legged robotics imitation learning · Multi-environment legged robotics dataset | mean return | Helped |
| increase context length (ablation) | FCNet · Legged robotics dataset | mean return | Helped |
| increase context length (ablation) | FCNet · Legged robotics dataset | inference latency | No clear effect |
| reduce context length to 8 (ablation) | Transformer · Legged robotics dataset | mean return | Hurt |
| increase model size by varying layers and hidden size (ablation) | FCNet · Legged robotics dataset | mean return | Helped |
| increase layer count or hidden size (ablation) | FCNet compared with Transformer · CPU inference tests | inference latency: 3 to 5 times faster | Helped |
Surprising findings
- Transformer with context length reduced to 8 produces completely ineffective policies on the legged locomotion dataset.
Limitations
- Scalability not validated on larger embodied datasets or pretraining.
- High-frequency information may matter in some scenarios and is filtered out.
- No multimodal inputs considered.
Experimental setup
Datasets: D4RL multi-environment legged robot locomotion dataset (self-collected in Isaacgym)
Benchmarks: D4RL-Mujoco D4RL-Adroit multi-environment legged robot locomotion dataset
Metrics: normalized return mean return CPU inference latency
Baselines: Decision Transformer DT-RetNet Behavior Cloning CQL BEAR BRAC-v AWR Transformer
Builds on: Decision Transformer FNet Fourier Neural Operator FITS
Closest papers by meaning
- FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens (NeurIPS 2025 · similarity 0.87)
- Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D (ICLR 2024 · similarity 0.85)
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.84)
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.84)
- Fourier Head: Helping Large Language Models Learn Complex Probability Distributions (ICLR 2025 · similarity 0.84)
- A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.