Research map / Architectures and efficiency / RNNs and normalization
Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings
Hanji Du
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AIR structured retrieval readout (vs sequential decoding) (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 10.85 -> 61.31 | Improved |
| add chiral regularization R_chi (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 7.45 -> 61.31 | Improved |
| use Householder involutive operator instead of a linear layer (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 1.02 -> 61.31 | Improved |
| use Householder reflection instead of orthogonal map in AIR (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 32.14 -> 61.31 | Improved |
| use Householder reflection instead of unconstrained linear map in AIR (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 18.45 -> 61.31 | Improved |
| use Householder reflection instead of identity map in AIR (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 8.12 -> 61.31 | Improved |
| use learned adjoint operator vs plain kNN retrieval (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 0.95 -> 61.31 | Improved |
| use learned operator instead of external retrievers (Contriever, DPR) in AIR (ablation) | A1 · Wiki-Inverse-10K | inverse accuracy (%): 14.28 -> 61.31 | Improved |
| A1 vs bidirectional training (BDP) | A1 · Fact-Inv-300 | inverse accuracy (%): 41.13 -> 65.07 | Improved |
| A1 vs data augmentation (DA) | A1 · Fact-Inv-300 | inverse accuracy (%): 32.47 -> 65.07 | Improved |
| A1 vs standard Llama-3 readout (forward accuracy) | A1 · Fact-Inv-300 | inverse accuracy (%): 88.47 -> 82.13 | Hurt |
| scale standard Transformer from 125M to 8B parameters | Standard Transformer · Wiki-Inverse-10K scaling | inverse accuracy (%): < 2% | No clear effect |
Closest papers by meaning
- A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse (NeurIPS 2026 · similarity 0.85)
- Learning Infinitesimal Generators of Continuous Symmetries from Data (NeurIPS 2024 · similarity 0.84)
- Geometric Factual Recall in Transformers (NeurIPS 2026 · similarity 0.84)
- A Generative Model of Symmetry Transformations (NeurIPS 2024 · similarity 0.84)
- Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics (NeurIPS 2024 · similarity 0.84)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
