Research map / Label-efficient and robust learning / Noisy labels

MOGIC: Metadata-infused Oracle Guidance for Improved Extreme Classification

ICML 2025 · Label-efficient and robust learning › Noisy labels · approach: Loss correction and noise robustness theory

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MOGIC, which trains a text-based early-fusion oracle with ground-truth metadata and uses its embeddings to regularize a late-fusion memory-based XC disciple (OAK, NGAME, DEXA), improving accuracy at no extra inference cost.

Problem

Memory-based extreme classification needs accurate, low-latency use of metadata; early-fusion text metadata is accurate with clean metadata but slow and noise-sensitive, while late-fusion is robust but less accurate.

Approach

Train an early-fusion DistilBERT oracle on query- and label-side ground-truth metadata, then guide a late-fusion memory-based disciple via alignment (cross-model triplet) and matching (L2 embedding) losses, so the disciple keeps low latency while mimicking oracle representations.

Key results

• MOGIC (OAK) reaches P@1 34.62 on LF-WikiSeeAlsoTitles-320K vs 33.71 for OAK, and 47.28 vs 44.82 on LF-WikiTitles-500K.
• MOGIC improves NGAME P@1 from 30.72 to 32.37 and DEXA P@1 from 31.57 to 32.75 on LF-WikiSeeAlsoTitles-320K.
• With predicted metadata, MOGIC (OAK) loses only about 1-2% while the oracle drops about 22% in P@1 when given noisy metadata.

Measured effects reported in the paper

Surprising findings

• The DistilBERT oracle outperforms LoRA-finetuned LLaMA-2 (7B) and Phi-2 oracles, so a smaller encoder works best as the oracle.
• The oracle is far less robust than the distilled disciple: its P@1 falls by about 22% with predicted metadata, while MOGIC (OAK) falls only 1-2%.

Limitations

• Oracle accuracy depends on ground-truth metadata, which is unavailable at deployment, so the gap to the oracle remains (about 6-8% in P@1).
• Gains are modest, about 1-2% in P@1 and 2-3% in propensity-scored metrics.

Experimental setup

Datasets: LF-WikiSeeAlsoTitles-320K LF-WikiTitles-500K LF-AmazonTitles-131K LF-WikiSeeAlso-320K LF-Wikipedia-500K LF-Amazon-131K

Benchmarks: LF-WikiSeeAlsoTitles-320K LF-WikiTitles-500K LF-AmazonTitles-131K

Metrics: P@1 P@5 N@5 PSP@1 PSP@5

Baselines: OAK DEXA NGAME ANCE DEXML GraphFormers GraphSAGE

Builds on: OAK NGAME DEXA knowledge distillation

Closest papers by meaning

• OAK: Enriching Document Representations using Auxiliary Knowledge for Extreme Classification (ICML 2024 · similarity 0.83)
• ELMO : Efficiency via Low-precision and Peak Memory Optimization in Large Output Spaces (ICML 2025 · similarity 0.82)
• ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities (ICLR 2025 · similarity 0.81)
• What Does It Take to Build a Performant Selective Classifier? (NeurIPS 2025 · similarity 0.81)
• LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content (CVPR 2024 · similarity 0.81)
• Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.81)