Research map / Label-efficient and robust learning / Domain adaptation and generalization
Decoupling Representation and Knowledge for Few-Shot Intent Classification and Slot Filling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes JMRM, which decouples general semantic representation from domain-specific intent-slot and slot-slot relation knowledge in few-shot NLU, transferring only the representation via two masking modules.
Problem
Few-shot joint intent classification and slot filling suffers from transfer gaps between source and target domains, especially for domain-specific knowledge such as intent-slot and slot-slot relations.
Approach
Explicitly decouple general semantic representation from domain-specific knowledge: an I2S-Mask built from support-set intent-slot co-occurrence and a Masked Slot Decoding module using BIO transition masks are applied during source training and target evaluation, while only the representation is transferred.
Key results
- On FewJoint 1-shot, JMRM improves Joint Acc from 27.72% to 42.20%.
- On FewJoint 5-shot, JMRM improves Joint Acc from 46.54% to 60.79%.
- On Snips 5-shot, Joint Acc rises from 52.95% (ConProm+FT+TR) to 59.24%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use JMRM instead of ConProm+FT+TR | Few-shot intent classification and slot filling · Snips 1-shot | Joint Acc.: 27.80% to 41.76% | Helped |
| Use JMRM instead of ConProm+FT+TR | Few-shot intent classification and slot filling · FewJoint 1-shot | Joint Acc.: 27.72% to 42.20% | Helped |
| Use JMRM instead of ConProm+FT+TR | Few-shot intent classification and slot filling · Snips 5-shot | Joint Acc.: 52.95% to 59.24% | Helped |
| Use JMRM instead of ConProm+FT+TR | Few-shot intent classification and slot filling · FewJoint 5-shot | Joint Acc.: 46.54% to 60.79% | Helped |
| Add I2S to joint modeling (ablation) | JM · FewJoint 1-shot, Cos similarity | Joint Acc.: 24.29 -> 36.56 | Helped |
| Add MSD to joint modeling (ablation) | JM · FewJoint 1-shot, Cos similarity | Joint Acc.: 24.29 -> 32.99 | Helped |
| Add I2S and MSD to joint modeling (ablation) | JM · FewJoint 1-shot, Cos similarity | Joint Acc.: 24.29 -> 41.35 | Helped |
| Add I2S to joint modeling (ablation) | JM · FewJoint 5-shot, L2 similarity | Joint Acc.: 37.24 -> 50.86 | Helped |
| Add MSD to joint modeling (ablation) | JM · FewJoint 5-shot, L2 similarity | Joint Acc.: 37.24 -> 46.54 | Helped |
| Add I2S and MSD to joint modeling (ablation) | JM · FewJoint 5-shot, L2 similarity | Joint Acc.: 37.24 -> 58.82 | Helped |
| Add I2S to JointProto (ablation) | JointProto · FewJoint 1-shot, L2 similarity | Joint Acc.: 26.11 -> 33.46 | Helped |
| Add I2S to JointProto (ablation) | JointProto · FewJoint 5-shot, L2 similarity | Joint Acc.: 37.69 -> 50.93 | Helped |
Surprising findings
- SepProto with two separate BERT encoders achieves the highest intent accuracy in 1-shot, above JMRM, which the authors attribute to shared-encoder focus on slots.
- Applying the relation masks only at target evaluation (+RM) already improves all baselines, showing benefit even without decoupled training.
Limitations
- Intent accuracy is not competitive with SepProto in the 1-shot setting.
- Snips evaluation uses only 10 testing episodes and few testing intents.
Experimental setup
Datasets: Snips FewJoint
Benchmarks: Snips FewJoint
Metrics: Intent Acc Slot F1 Joint Acc
Baselines: SepProto JointProto ConProm ConProm+FT+TR
Builds on: Prototypical Networks ConProm JointProto VPB
Closest papers by meaning
- Towards Explainable Joint Models via Information Theory for Multiple Intent Detection and Slot Filling (AAAI 2024 · similarity 0.83)
- Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation (ICML 2025 · similarity 0.83)
- Uni-MIS: United Multiple Intent Spoken Language Understanding via Multi-View Intent-Slot Interaction (AAAI 2024 · similarity 0.82)
- Slot State Space Models (NeurIPS 2024 · similarity 0.82)
- OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP (CVPR 2025 · similarity 0.82)
- BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.