Research map / Large language models / Language, safety and interpretability
Just Add Functions: A Neural-Symbolic Language Model
David Demeter, Doug Downey
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add numeric micro-models based on class-specific metrics and PDFs | LSTM language model · Wikitext-2 Numbers | perplexity: 790.2 -> 475.5 | Improved |
| Add numeric micro-models based on class-specific metrics and PDFs | LSTM language model · Wikitext-103 Numbers | perplexity: 364.6 -> 262.0 | Improved |
| Add geographic micro-models using location distance | LSTM language model · Wikitext-2 Locations | perplexity: 4,812.1 -> 1,801.1 | Improved |
| Add geographic micro-models using location distance | LSTM language model · Wikitext-103 Locations | perplexity: 585.7 -> 488.6 | Improved |
| Add numeric micro-models | AWD-LSTM · Wikitext-2 Numbers | perplexity: 421.1 -> 336.4 | Improved |
| Add geographic micro-models | AWD-LSTM · Wikitext-2 Locations | perplexity: 2,066.3 -> 1,340.7 | Improved |
| Initialize word embeddings with GloVe trained on the 6B token corpus | NNLM · Wikitext-2 Numbers | perplexity: 831.2 -> 640.0 | Improved |
| Initialize word embeddings with GloVe trained on the 6B token corpus | NNLM · Wikitext-2 Locations | perplexity: 4,969.4 -> 3,888.3 | Improved |
| Append numeric values to embeddings and freeze the appended dimensions | NNLM · Wikitext-2 Numbers | perplexity: 790.2 -> 717.9 | Improved |
| Append longitude and latitude to embeddings and freeze the appended dimensions | NNLM · Wikitext-2 Locations | perplexity: 4,812.1 -> 3,890.2 | Improved |
| Add micro-models to the LSTM while monitoring overall language-model performance | LSTM language model · Wikitext-2 global | perplexity: 92.0 -> 89.8 | Improved |
Closest papers by meaning
- Training Neural Networks as Recognizers of Formal Languages (ICLR 2025 · similarity 0.88)
- In-Context Language Learning: Architectures and Algorithms (ICML 2024 · similarity 0.87)
- Neural-Symbolic Recursive Machine for Systematic Generalization (ICLR 2024 · similarity 0.87)
- Logically Consistent Language Models via Neuro-Symbolic Integration (ICLR 2025 · similarity 0.87)
- Language Models Are Implicitly Continuous (ICLR 2025 · similarity 0.86)
- Large Language Models are Interpretable Learners (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
