Research map / Architectures and efficiency / RNNs and normalization
EVALUATION OF NEURAL ARCHITECTURES TRAINED WITH SQUARE LOSS VS CROSS-ENTROPY IN CLASSIFICATION TASKS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace cross-entropy with square loss | BERT · MRPC test | accuracy: 82.1 -> 83.8 | Helped |
| Replace cross-entropy with square loss | BERT · MRPC test | F1: 86.7 -> 88.1 | Helped |
| Replace cross-entropy with square loss | Transformer-XL · enwik8 test | accuracy: 77.5 -> 76.7 | Hurt |
| Replace cross-entropy with square loss | LSTM+CNN · QQP test | F1: 60.5 -> 60.3 | Hurt |
| Replace cross-entropy with square loss | VGG+BLSTMP · WSJ test | WER: 5.3 -> 5.1 | Helped |
| Replace cross-entropy with square loss | VGG+BLSTM · Librispeech test | WER: 10.6 -> 9.8 | Helped |
| Replace cross-entropy with square loss | Transformer · Librispeech test | WER: 9.2 -> 9.4 | Hurt |
| Replace cross-entropy with square loss | TCNN · MNIST test | accuracy: 97.7 -> 97.7 | No clear effect |
| Replace cross-entropy with square loss | Wide ResNet · CIFAR-10 test | accuracy: 96.3 -> 95.9 | Hurt |
| Replace cross-entropy with square loss | ResNet-50 · ImageNet validation | accuracy: 76.1 -> 76.2 | Helped |
| Replace cross-entropy with square loss | EfficientNet · ImageNet validation | accuracy: 77.0 -> 74.6 | Hurt |
| Use square loss with the same training epochs as cross-entropy (ablation) | Square-loss training · NLP tasks | accuracy | Helped |
Closest papers by meaning
- Generative Cross-Entropy: A Strictly Proper Loss for Data-Efficient Classification (NeurIPS 2026 · similarity 0.88)
- Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss (Student Abstract) (AAAI 2024 · similarity 0.87)
- Cut Your Losses in Large-Vocabulary Language Models (ICLR 2025 · similarity 0.87)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.86)
- Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics (NeurIPS 2026 · similarity 0.85)
- Scaling Law with Learning Rate Annealing (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.