NeurIPS 2021 accepted papers on Large language models
66 of the 2,508 papers accepted at NeurIPS 2021 (2.6%) are on Large language models, compared with 1.2% at NeurIPS 2020. They fall into 6 topics.
See NeurIPS 2021 in the interactive map
Submitting to NeurIPS? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Language, safety and interpretability · 32 papers
- True Few-Shot Learning with Language Models (189 citations)
- COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining (128 citations)
- CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review (96 citations)
- KLUE: Korean Language Understanding Evaluation (76 citations)
- FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information (52 citations)
- Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (29 citations)
- Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models (26 citations)
- Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence (23 citations)
- MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers (23 citations)
- Multimodal and Multilingual Embeddings for Large-Scale Speech Mining (21 citations)
- FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark (20 citations)
- Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking (16 citations)
and 20 more in the interactive map.
Agents, code and math · 12 papers
- Computer-Aided Design as Language (40 citations)
- NaturalProofs: Mathematical Theorem Proving in Natural Language (13 citations)
- Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages (11 citations)
- Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning (7 citations)
- Pipeline Combinators for Gradual AutoML (6 citations)
- How to transfer algorithmic reasoning knowledge to learn new algorithms? (5 citations)
- Grammar-Based Grounded Lexicon Learning (2 citations)
- Measuring Coding Challenge Competence With APPS (2 citations)
- MIND dataset for diet planning and dietary healthcare with machine learning: Dataset creation using combinatorial optimization and controllable generation with domain experts (2 citations)
- Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLP (1 citation)
- Programming Puzzles
- Subgoal Search For Complex Reasoning Tasks
AI and society · 10 papers
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (408 citations)
- Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (162 citations)
- CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks (72 citations)
- The Utility of Explainable AI in Ad Hoc Human-Machine Teaming (19 citations)
- AI and the Everything in the Whole Wide World Benchmark (15 citations)
- SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning (7 citations)
- Distributed Deep Learning In Open Collaborations (4 citations)
- Artsheets for Art Datasets (2 citations)
- Addressing "Documentation Debt" in Machine Learning: A Retrospective Datasheet for BookCorpus
- Think Big, Teach Small: Do Language Models Distil Occam’s Razor?
Retrieval and knowledge · 8 papers
- A Dataset for Answering Time-Sensitive Questions (37 citations)
- CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge (26 citations)
- Mind the Gap: Assessing Temporal Generalization in Neural Language Models (24 citations)
- Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management (12 citations)
- Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval (4 citations)
- Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs (4 citations)
- CSFCube - A Test Collection of Computer Science Research Articles for Faceted Query by Example (2 citations)
- Open Rule Induction
