Research map / 3D vision / Human pose estimation
A Category Agnostic Model for Visual Rearrangment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CAVR, a category-agnostic modular visual rearrangement agent that detects scene changes via point-cloud diff-clouds, matches changes by appearance features, and uses a closer-aligned-retrace exploration policy, and introduces the MrMiR multi-room dataset.
Problem
Visual rearrangement (recovering a shuffled scene to its goal configuration from experience-goal observations) in existing modular methods relies on category labels from detectors, which introduces errors and limits generalization to unseen categories.
Approach
Represent scene changes as a point-cloud diff-cloud (moved and protruding parts) instead of semantic labels, match changed entities across locations using category-agnostic geometric and visual appearance similarity via bipartite matching, and explore with a policy that minimizes observation distance to objects.
Key results
- On RoomR, CAVR reaches 14.2% success and 33.1% Fixed Strict versus TIDEE's 11.7% and 28.9%, and MaSS's 4.7% and 16.5%.
- On the new MrMiR dataset, CAVR achieves 5.0% success and 28.7% Fixed Strict versus TIDEE's 1.0% and 14.1%.
- Ablation: replacing the closer-aligned-retrace policy with coverage-based exploration drops success to 13.1%, and using only visual or only geometric matching features yields 14.0% and 14.2% success respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CAVR instead of TIDEE | Visual rearrangement · RoomR | Success (%): 11.7 -> 14.2 | Helped |
| use CAVR instead of TIDEE | Visual rearrangement · MrMiR | Success (%): 1.0 -> 5.0 | Helped |
| add closer-aligned-retrace exploration instead of coverage-based exploration (ablation) | CAVR · RoomR | Success (%): 13.1 -> 14.2 | Helped |
| add closer-aligned-retrace exploration instead of MaSS's semantic policy (ablation) | CAVR · RoomR | Success (%): 8.7 -> 14.2 | Helped |
| add appearance-similarity matching instead of uniform matching weights (ablation) | CAVR · RoomR | Success (%): 11.3 -> 14.2 | Helped |
| add geometric-feature similarity to visual-feature matching (ablation) | CAVR · RoomR | Success (%): 14.0 -> 14.2 | No clear effect |
| add visual-feature similarity to geometric-feature matching (ablation) | CAVR · RoomR | FixedStrict (%): 32.3 -> 33.1 | Helped |
Surprising findings
- Geometric-only matching achieves the same success rate (14.2%) as the full model, so the gain is mainly in fixed-strict and energy metrics.
Limitations
- Success rates remain low in absolute terms (14.2% on RoomR, 5.0% on MrMiR).
- Matching relies on appearance and geometry similarity, which may fail for visually identical instances.
- Hyper-parameters such as the distance threshold were tuned on RoomR and transferred to MrMiR.
Experimental setup
Datasets: RoomR AI2THOR point cloud pairs ProcTHOR apartments
Benchmarks: RoomR MrMiR
Metrics: Success Fixed Strict Energy Remaining Misplaced
Baselines: TIDEE MaSS
Builds on: TIDEE MaSS PointNet++ DBSCAN Kuhn-Munkres algorithm
Closest papers by meaning
- Task Planning for Object Rearrangement in Multi-Room Environments (AAAI 2024 · similarity 0.86)
- Multi-View Pose-Agnostic Change Localization with Zero Labels (CVPR 2025 · similarity 0.85)
- SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent (NeurIPS 2025 · similarity 0.84)
- Learning to Evolve Scenes: Reasoning about Human Activities with Scene Graphs (NeurIPS 2026 · similarity 0.84)
- STAR-R1: Multi-View Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs (CVPR 2026 · similarity 0.84)
- InfiniBench: Infinite Benchmarking for Visual Spatial Reasoning with Customizable Scene Complexity (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.