Research map / Vision-language and multimodal / Multimodal LLMs

BAMI: Training-Free Bias Mitigation in GUI Grounding

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BAMI, a training-free test-time method for GUI grounding that combines coarse-to-fine cropping with masked multi-candidate generation and an external LLM/VLM candidate selector guided by GUI priors, raising TianXi-Action-7B on ScreenSpot-Pro from 51.9% to 57.8%.

Problem

GUI grounding models underperform on high-resolution, dense professional interfaces such as ScreenSpot-Pro (below 50% accuracy), with errors attributed to precision and ambiguity biases in the model's inference.

Approach

Diagnose grounding errors with a masked prediction distribution attribution method, then apply training-free bias-aware inference: iterative coarse-to-fine cropping to fix precision bias, and masked multi-candidate generation with a prompted correction model (GUI-prior rules) to fix ambiguity bias.

Key results

• BAMI with GPT-5 as correction model raises TianXi-Action-7B on ScreenSpot-Pro from 51.9% to 57.8%, the best among 7B models reported.
• BAMI raises OS-Atlas-7B on ScreenSpot-Pro from 18.9% to 41.6% and UI-TARS-1.5-7B from 40.8% to 51.9% (average).
• A local Qwen3-VL-8B correction model fine-tuned with LoRA reaches 56.2% on ScreenSpot-Pro; on ScreenSpot-V2 BAMI improves OS-Atlas-7B from 81.2 to 82.2 and UI-TARS-1.5-7B from 86.4 to 86.5.

Surprising findings

• Naive vanilla prompting of the correction model yields only modest gains; chain-of-thought and GUI-prior key principles are needed for ambiguity-bias correction.
• Language-space expansion (query or context expansion) did not significantly improve accuracy and sometimes introduced errors.

Limitations

• Multi-step inference with multiple candidates and an external correction model increases inference cost.
• Gains on ScreenSpot-V2 are small because baseline models already perform well there.
• Reliance on GPT-5 API for best results raises privacy and dependency concerns; local variant is weaker.

Experimental setup

Datasets: ScreenSpot-Pro ScreenSpot-V2 GUIAct Desktop domain data

Benchmarks: ScreenSpot-Pro ScreenSpot-V2

Metrics: accuracy

Baselines: OS-Atlas-7B UI-TARS-1.5-7B TianXi-Action-7B UGround-7B DiMo-GUI-7B GUI-G2-7B GUI-RC Claude Computer Use GPT-4o

Builds on: Shapley values GUI-RC ScreenSeekeR

Closest papers by meaning

• GUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding (ICML 2026 · similarity 0.87)
• UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction as Reasoning (ICLR 2026 · similarity 0.87)
• Learning GUI Grounding with Spatial Reasoning from Visual Feedback (ICML 2026 · similarity 0.87)
• Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding (NeurIPS 2026 · similarity 0.87)
• Test-Time Reinforcement Learning for GUI Grounding via Region Consistency (AAAI 2026 · similarity 0.87)
• GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents (NeurIPS 2025 · similarity 0.87)