AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

UNIREX: A Unified Learning Framework for Language Model Rationale Extraction

Aaron Chan, Maziar Sanjabi, Lambert Mathias, Liang Tan, Shaoliang Nie, Xiaochang Peng, Xiang Ren, Hamed Firooz

ICML 2022 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 19 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Jointly train the task model and learned extractor for faithfulness and plausibilityUNIREX · Five English text classification datasets, Dataset MeanCNRG relative gain: 32.9%Improved
Use UNIREX (AA-F) instead of strongest baseline SGTRationale extraction without plausibility optimization · Dataset MeanCNRG: 9.2%Improved
Use UNIREX (DLM-FP) or UNIREX (SLM-FP) instead of A2R+PRationale extraction with plausibility optimization · Dataset MeanCNRG: over 30%Improved
Use UNIREX (DLM-FP) rationales rather than other non-oracle methodsSST user study · SSTalignment: 3.96~($\pm$0.22) vs 3.64~($\pm$0.28)Improved
Use an MLP pooler instead of sum pooling (ablation)UNIREX AA-FP · SSTAUPRC: 47.60~($\pm$2.44) -> 54.82~($\pm$1.97)Improved
Use a learned Dual LM extractor instead of an attribution-algorithm extractor (ablation)UNIREX FP · SSTAUPRC: 47.60~($\pm$2.44) -> 85.80~($\pm$0.74)Improved
Optimize comprehensiveness without sufficiency (ablation)UNIREX SLM-FP · SSTComp: 0.302~($\pm$0.039) -> 0.350~($\pm$0.048)Improved
Add sufficiency optimization to comprehensiveness optimization (ablation)UNIREX SLM-FP · SSTSuff: 0.310~($\pm$0.049) -> 0.113~($\pm$0.013)Improved
Use sufficiency alone instead of comprehensiveness plus sufficiency (ablation)UNIREX SLM-FP · SSTComp: 0.302~($\pm$0.039) -> 0.166~($\pm$0.003)Hurt
Replace margin sufficiency loss with KL divergence (ablation)UNIREX SLM-FP · SSTSuff: 0.113~($\pm$0.013) -> 0.131~($\pm$0.005)Hurt
Replace margin sufficiency loss with MAE (ablation)UNIREX SLM-FP · SSTSuff: 0.113~($\pm$0.013) -> 0.143~($\pm$0.008)Hurt
Replace the linear extractor head with a two-layer 2048-dim MLP (ablation)UNIREX SLM-FP · SSTComp: 0.302~($\pm$0.039) -> 0.323~($\pm$0.071)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy