Research map / Federated learning and privacy / Communication-efficient FL
Federated Online Adaptation for Deep Stereo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes federated online adaptation for deep stereo, where peer clients adapt a stereo network and a listening lightweight client receives averaged weights, plus FedMAD to reduce communication, built on a new MADNet 2 backbone.
Problem
Online adaptation of deep stereo networks to challenging domains is computationally prohibitive on resource-constrained devices, since adaptation lowers framerate.
Approach
Distribute the online adaptation workload to remote active clients on powerful hardware, which periodically send weights to a server that averages them (FedAvg-style) and dispatches them to passive listening clients; FedMAD sends only one sampled block per round to cut traffic.
Key results
- On DSEC nighttime sequences, FedFULL++ with MADNet 2 reaches 8.36% D1-all on Night#2 versus 9.16% for single-agent FULL, while running near 12 ms on RTX 3090 and 111 ms on AGX.
- FedMAD reduces traffic to the server (e.g., 4.3 MB/s vs 20.2 MB/s for FedFULL on KITTI) with only small accuracy loss.
- MADNet 2 with proxy-label federated adaptation runs at about 20 FPS on Jetson AGX while adapting, versus about 2 FPS-range single-agent full adaptation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the MADNet cost-volume module with RAFT-Stereo all-pairs correlation and remove the context network | MADNet 2 on KITTI City · KITTI City | D1-all (%): 37.42 -> 4.04 | Helped |
| Add full single-agent photometric adaptation | MADNet 2 on KITTI City · KITTI City | D1-all (%): 4.04 -> 1.32 | Helped |
| Add federated full adaptation with photometric loss | MADNet 2 on DSEC Night#2 · DSEC Night#2 | D1-all (%): 13.86 -> 8.36 | Helped |
| Add federated full adaptation with photometric loss | MADNet 2 on DrivingStereo Dusky · DrivingStereo Dusky | D1-all (%): 13.16 -> 10.65 | Helped |
| Add federated adaptation with proxy labels | MADNet 2 on DrivingStereo Rainy · DrivingStereo Rainy | D1-all (%): 10.34 -> 8.33 | Helped |
| Add federated full adaptation with photometric loss | MADNet 2 on DSEC Night#1 · DSEC Night#1 | D1-all (%): 5.65 -> 5.50 | Helped |
| Use modular updates instead of full federated updates (ablation) | MADNet 2 FedMAD on KITTI City · KITTI City | D1-all (%): 1.42 -> 1.48 | Hurt |
| Use modular updates instead of full federated updates (ablation) | MADNet 2 FedMAD++ on DSEC Night#2 · DSEC Night#2 | D1-all (%): 7.03 -> 7.13 | Hurt |
| Average only decoder weights rather than all weights (ablation) | MADNet 2 FedDEC on KITTI City · KITTI City | D1-all (%): 1.42 -> 1.43 | No clear effect |
| Average only the last decoder weights (ablation) | MADNet 2 FedLAST on KITTI City · KITTI City | D1-all (%): 1.42 -> 2.72 | Hurt |
| Average only encoder weights (ablation) | MADNet 2 FedENC on KITTI City · KITTI City | D1-all (%): 1.42 -> 3.44 | Hurt |
| Increase the number of active clients (ablation) | FedFULL and FedMAD on KITTI · KITTI | D1-all (%) | Helped |
Surprising findings
- Federated adaptation by remote clients can outperform on-device single-agent adaptation in challenging domains such as nighttime and rainy scenes.
Limitations
- Active clients receive no reciprocal benefit from the listening client.
- Adapting nodes are assumed to process images with similar properties to the listening client.
- Connection delays are ignored because all clients run on the same server.
Experimental setup
Datasets: FlyingThings3D KITTI DrivingStereo DSEC
Benchmarks: KITTI DrivingStereo DSEC
Metrics: D1-all EPE runtime (ms) data traffic (MB/s)
Baselines: MADNet RAFT-Stereo CREStereo IGEV-Stereo UniMatch CoEX HITNet TemporalStereo FULL MAD
Builds on: FedAvg MADNet MAD RAFT-Stereo
Closest papers by meaning
- Learning to Adapt for Stereo (CVPR 2019 · similarity 0.87)
- Real-Time Self-Adaptive Deep Stereo (CVPR 2019 · similarity 0.85)
- RobIA: Robust Instance-aware Continual Test-time Adaptation for Deep Stereo (NeurIPS 2025 · similarity 0.84)
- Stereo Risk: A Continuous Modeling Approach to Stereo Matching (ICML 2024 · similarity 0.82)
- Cross-Device Collaborative Test-Time Adaptation (NeurIPS 2024 · similarity 0.82)
- Robust Synthetic-to-Real Transfer for Stereo Matching (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.