Research map / Large language models / AI and society
Benchmarking PhD-Level Coding in 3D Geometric Computer Vision
Wenyi Li, Renkai Luo, Yue Yu, Huan-ang Gao, Mingju Gao, Li Yuan, Chaoyou Fu, Hao Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| give only Introduction-to-Method paper text instead of the full paper (ablation) | LLMs (Gemini-2.5-Pro, DeepSeek-R1) · GeoCodeBench | overall pass rate | Improved |
| give full paper instead of text up to the Method section (ablation) | most LLMs · GeoCodeBench | overall pass rate | Hurt |
| provide more paper context (full paper) (ablation) | Qwen3-Coder-480B and Doubao-Seed-1.6 · GeoCodeBench | overall pass rate | Improved |
| vary paper input (none, up to Method, full) (ablation) | GPT-5 and Kimi-K2-Instruct · GeoCodeBench | overall pass rate | No clear effect |
| add more paper content (ablation) | LLMs on Geometric Transformations and Mechanics/Optics · GeoCodeBench | pass rate | No clear effect |
| provide full paper instead of up-to-method input (ablation) | LLMs on Algorithm Implementation and Geometric Logic Routing · GeoCodeBench | pass rate | Hurt |
| evaluate research tasks instead of general 3D tasks | all eight LLMs · GeoCodeBench | pass rate: GPT-5 42.8% -> 29.1% | Hurt |
| evaluate research tasks instead of general 3D tasks | Claude-Sonnet-4.5 · GeoCodeBench | pass rate: 37.2% -> 23.7% | Hurt |
| use GPT-5 instead of Llama-3.1-405B-Instruct | LLM coding · GeoCodeBench | overall pass rate: 14.3% -> 36.6% | Improved |
| use Qwen3-Coder-480B on research versus general tasks | Qwen3-Coder-480B · GeoCodeBench | pass rate: 22.7% -> 24.6% | No clear effect |
Closest papers by meaning
- 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code (NeurIPS 2026 · similarity 0.89)
- GIQ: Benchmarking 3D Geometric Reasoning of Vision Foundation Models with Simulated and Real Polyhedra (ICLR 2026 · similarity 0.88)
- Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code (ICML 2026 · similarity 0.88)
- Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis (NeurIPS 2026 · similarity 0.87)
- Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning (CVPR 2026 · similarity 0.87)
- GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
