Research map / Vision-language and multimodal / Visual question answering

Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue

AAAI 2025 · Vision-language and multimodal › Visual question answering · approach: Open-domain multimodal QA · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TSADE, a tree-structured question-generation strategy for goal-oriented visual dialogue that uses an answer distribution estimator and two RL rewards (binary divide-and-conquer and candidate-minimization) to plug into existing QGen models on GuessWhat?! and VisDial.

Problem

Goal-oriented visual dialogue (GuessWhat?!, GuessWhich) where existing question generators ask aimless, repetitive questions and fail to narrow the candidate object set efficiently within limited rounds.

Approach

Guide question generation with a divide-and-conquer tree strategy: each question should split current candidates roughly in half, measured by an answer distribution estimator (a simulated Oracle) and rewarded via a binary reward plus a candidate-minimization reward under policy gradient RL, without injecting ground truth into the reward.

Key results

• FS+TSADE reduces question repetition rate on New Object greedy from 66.27% to 26.02%.
• ADVSE+TSADE reaches 65.99% greedy task success rate on New Image with r_s, a new state-of-the-art claimed under RL.
• On VisDial v1.0, ReeQ+AugG+TSADE improves MRR from 31.21 to 33.21 without r_o.

Measured effects reported in the paper

Surprising findings

• GPT-4-vision-preview used as Oracle, Guesser and QGen performs much worse than small traditional models on GuessWhat?!, which the authors attribute to lacking spatial reasoning.

Limitations

• Gains are reported mainly as relative improvements within existing RL frameworks; absolute gains on some metrics are small (e.g., ADVSE+TSADE with r_s improves greedy New Image success by only 0.68%).
• Candidate-minimization reward is tuned via grid search (alpha, beta, gamma) and the method depends on an approximated oracle/answer distribution estimator.
• Generalization to other tasks (e.g., goal navigation) is only suggested, not evaluated.

Experimental setup

Datasets: GuessWhat?! VisDial v1.0

Benchmarks: GuessWhat?! VisDial v1.0

Metrics: task success rate question repetition rate average rounds T T/R efficiency MRR R@1 R@5 R@10

Baselines: DV FS ADVSE GDSE VDST CSQG Vilbert Bayesian VQG ReeQ+AugG GPT-4-vision-preview

Builds on: FS (Fast-Strategy RL QGen) DV (Deep Visual dialogue) ADVSE AQM ReeQ

Closest papers by meaning

• Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog (NeurIPS 2018 · similarity 0.90)
• From Retrieval to Generation: A Simple and Unified Generative Model for End-to-End Task-Oriented Dialogue (AAAI 2024 · similarity 0.86)
• ConVQG: Contrastive Visual Question Generation with Multimodal Guidance (AAAI 2024 · similarity 0.86)
• Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding (NeurIPS 2026 · similarity 0.86)
• Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation (CVPR 2025 · similarity 0.86)
• Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering (NeurIPS 2025 · similarity 0.86)