Research map / Architectures and efficiency / Attention and transformers
CoFrGeNet: Continued Fraction Architectures for Language Generation
Amit Dhurandhar, Vijil Chenthamarakshan, Dennis Wei, Tejaswini Pedapati, Karthikeyan Natesan Ramamurthy, Rahul Nair
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dyadic incremental parameter update schedule (ablation) | CoFrGeNet-F · Wikitext103 (OWT) | perplexity: 22.65 -> 16.14 | Improved |
| add dyadic incremental parameter update schedule (ablation) | CoFrGeNet-A · Wikitext103 (OWT) | perplexity: 24.25 -> 17.26 | Improved |
| add dyadic incremental parameter update schedule (ablation) | CoFrGeNet · Wikitext103 (OWT) | perplexity: 23.33 -> 17.17 | Improved |
| add dyadic incremental parameter update schedule (ablation) | CoFrGeNet-F · Wikitext103 (GW) | perplexity: 20.44 -> 18.11 | Improved |
| add dyadic incremental parameter update schedule (ablation) | CoFrGeNet-F · Wikitext2 (OWT) | perplexity: 26.71 -> 17.12 | Improved |
| use continuant-based computation instead of per-layer division (ablation) | CoFrGeNet · OWT inference time | inference time (us): 5898.72 -> 628.73 | Improved |
| use continuant-based computation instead of per-layer division (ablation) | CoFrGeNet · OWT train time | train time (hrs): 203 -> 178 | Improved |
| replace FFN with CoFrNet ladders | GPT2-xl · Wikitext103 (OWT) | perplexity: 17.50 -> 16.14 | Improved |
| replace FFN with CoFrNet ladders | GPT2-xl · GLUE MNLI (OWT) | accuracy: 86.89 -> 87.26 | Improved |
| replace attention with CoFrNet ladders | GPT2-xl · Wikitext103 (OWT) | perplexity: 17.50 -> 17.26 | No clear effect |
| replace FFN with CoFrNet ladders | Llama-3.2B · BoolQ zero-shot | accuracy: .691 -> .668 | Hurt |
| replace FFN with CoFrNet ladders | Llama-3.2B · train time | train time (days): 8.5 -> 6.6 | Improved |
Closest papers by meaning
- Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models (NeurIPS 2025 · similarity 0.87)
- Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges (ICML 2025 · similarity 0.87)
- Separations in the Representational Capabilities of Transformers and Recurrent Architectures (NeurIPS 2024 · similarity 0.86)
- Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA (ICLR 2025 · similarity 0.86)
- Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks (ICML 2024 · similarity 0.86)
- Understanding Transformers via N-Gram Statistics (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
