Research map / Vision-language and multimodal / Human and embodied modeling
TIGER-FG: Text-Guided Implicit Fine-Grained Grounding for E-commerce Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TIGER-FG, a detection-free dual-encoder that uses structured item text to guide target-focused pooling over full item images for cropped-query e-commerce retrieval, and releases the ECom-RF-IMMR benchmark suite.
Problem
Image-to-multimodal retrieval in e-commerce, where a cropped visual query must match full-scene item images with structured text; existing approaches either need costly detection/grounding pipelines or global CLIP-style embeddings that get distracted by backgrounds and other items.
Approach
Text-guided two-step cross-attention over learnable query slots plus a CLS-guided patch pooling branch yields a target-focused item embedding without boxes at inference; training adds region-text alignment, target-anchored fused-region alignment with cross-category hard negatives, spatial-relational distillation from a DINOv3 teacher, and similarity-distribution distillation from a MoCo image retr
Key results
- On ECom-RF-IMMR-Normal, TIGER-FG reaches 80.1 Recall@1, 6.1 points above the strongest baseline, with 85.7M query-side parameters and 256-dim embeddings.
- On ECom-RF-IMMR-Mosaic, TIGER-FG reaches 75.2 Recall@1, 34.4 points above the strongest baseline; Qwen3-VL-Embedding drops from 74.0 to 40.8 Recall@1 on Normal vs Mosaic.
- On LookBench, TIGER-FG achieves best HitRate, Recall, MRR and NDCG across subsets (e.g., HitRate@1 39.8 vs 33.1 for Qwen3-VL-Emb).
Surprising findings
- Removing spatial-relational distillation has essentially no effect on ranking metrics (w/o SRD Recall@1 75.3 vs 75.2 on Mosaic), so it acts mainly as a grounding regularizer.
- Training only on original samples without Mosaic augmentation drops Recall@1 on Mosaic from 75.2 to 47.8, indicating clutter-aware training is critical.
Limitations
- Evaluated only in the e-commerce setting with Chinese-language structured item text; generalization to other text fields, category systems and languages is untested.
- Benchmarks cover item-level retrieval under clean and synthetic cluttered layouts and do not model personalization or live serving dynamics.
- Training relies on GroundingDINO pseudo boxes and a Mosaic synthetic augmentation, and item boxes are used as auxiliary supervision.
Experimental setup
Datasets: ECom-RF-IMMR-10M eSSPR LookBench
Benchmarks: ECom-RF-IMMR-Normal ECom-RF-IMMR-Mosaic eSSPR LookBench
Metrics: Recall@K MRR@K NDCG@K HitRate@K
Baselines: CLIP-SF CLIP-FF BLIP-SF BLIP-FF BGE-VL-Large GME-2B Qwen3-VL-Embedding-2B Ops-MM-embedding-v1-2B UniEcs OClear
Builds on: CLIP DINOv3 BLIP-2 DETR GroundingDINO
Closest papers by meaning
- Transformer-Empowered Multi-Modal Item Embedding for Enhanced Image Search in E-commerce (AAAI 2024 · similarity 0.87)
- TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models (ICLR 2025 · similarity 0.86)
- MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions (ICML 2024 · similarity 0.85)
- MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents (ICLR 2026 · similarity 0.85)
- Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.85)
- Visual Abstraction: A Plug-and-Play Approach for Text-Visual Retrieval (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.