Research map / Multi-agent systems and games / LLM agents

Mix-Ecom: Towards Mixed-Type E-Commerce Dialogues with Complex Domain Rules

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Mix-ECom, a real-world mixed-type e-commerce customer-service benchmark with 82 domain rules and tools, and proposes E-ReAct and E-Plan&Solve, which add a dynamic module that filters domain rules and trajectories to reduce hallucination.

Problem

Existing e-commerce agent benchmarks test simplistic user issues and simple policies, not mixed-type dialogues (QA, recommendation, task-oriented, chit-chat) under complex domain rules with multimodal inputs.

Approach

A dynamic module, invoked after each user utterance, selects the task-relevant subset of domain rules and prunes hallucinated trajectory steps (or re-plans) before the ReAct or Plan-and-Solve agent acts.

Key results

• Gemini-2.5-pro with E-Plan&Solve achieves the highest total Score of 62.2% on Mix-ECom, while GPT-4o with ReAct scores 43.1%.
• E-ReAct improves over ReAct for every model tested, e.g., GPT-4o total Score rises from 43.1 to 49.2 and Gemini-2.5-pro from 53.5 to 60.5.
• Qwen-2.5-VL-7B fine-tuned on the logistics and after-sales subsets improves its logistics Score from 0.9 to 19.3 and after-sales from 0.0 to 17.7.

Surprising findings

• Stripping non-text inputs drops GPT-4o scores by only 3.3 and 6.0 points, suggesting the model barely exploits visual cues.
• Removing domain rules drops logistics and after-sales scores by 44.5 and 11.3 points, showing rule compliance is a major bottleneck.

Limitations

• Only 299 test samples, and the 4,500 training instances were used for only a subset of tasks in SFT due to resource constraints.
• Fine-tuned model is a single small open model; pre-sales and video inputs were excluded from training.
• Dataset is described as to-be-released, and the evaluation relies on GPT-4o as judge for key answers and database equivalence.

Experimental setup

Datasets: 70,000 real-world customer-service conversations user portraits from historical behavior

Benchmarks: Mix-ECom

Metrics: Key Answer score Database score Score (joint success) human evaluation (human-likeness, informativeness, key answe

Baselines: ReAct Plan&Solve Qwen-2.5-VL 7B Tau-retail EcomScriptBench CBYS RECBENCH-MD ECom-Bench

Builds on: ReAct Plan-and-Solve

Closest papers by meaning

• EcomGPT: Instruction-Tuning Large Language Models with Chain-of-Task Tasks for E-commerce (AAAI 2024 · similarity 0.85)
• eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data (ICML 2024 · similarity 0.83)
• ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents (AAAI 2026 · similarity 0.83)
• MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (NeurIPS 2024 · similarity 0.83)
• ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents (NeurIPS 2026 · similarity 0.83)
• Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.82)