Research map / Architectures and efficiency / RNNs and normalization
Learning Linear Block Error Correction Codes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes jointly training binary linear block codes and a Transformer decoder end-to-end over GF(2), using STE binarization, polarization for the XOR operation, and a differentiable Tanner-graph-derived attention mask.
Problem
Designing short binary linear block codes that are efficiently decodable is hard because the code's binary structure is non-differentiable, so neural decoders have only been applied to fixed, classically designed codes.
Approach
Parameterize the systematic parity-check matrix via a trainable real matrix binarized with a straight-through estimator, convert GF(2) XOR into a differentiable product of bipolar terms, and build the Transformer's self-attention mask from H so gradients flow from the decoder back to the code (DC-ECCT / E2E DC-ECCT).
Key results
- On BCH(31,16) at Eb/N0=6 dB, E2E DC-ECCT (N=6,d=128) reaches -ln BER 12.84 versus 10.63 for ECCT, and the paper reports the learned codes improve BP and ML decoding too (e.g., BP on (64,32) at 6 dB: 12.51 vs 4.45 baseline).
- Even with fixed (untrained) parity-check matrices, DC-ECCT outperforms the ECCT neural decoder across the tested codes in the reported table.
- Training with the all-ones message (m=1) is reported as important; using m=0 completely fails training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly train the code and decoder rather than use the conventional code with ECCT | DC-ECCT · POLAR(32,11), Eb/N0=6 | -ln(BER): 10.19 -> 11.71 | Helped |
| Jointly train the code and decoder rather than use the conventional code with ECCT | DC-ECCT · BCH(31,16), Eb/N0=6 | -ln(BER): 10.63 -> 12.54 | Helped |
| Use a fully trainable mask independent of the code (ablation) | DC-ECCT · (31,16) and (32,11), Eb/N0=4 | -ln(BER): 5.16 -> 4.83; 4.40 -> 4.04 | Hurt |
| Use STE for modulo calculations instead of polarization (ablation) | DC-ECCT · (31,16) and (32,11), Eb/N0=4 | -ln(BER): 5.16 -> 5.11; 4.40 -> 4.63 | Mixed |
| Stop gradients from backpropagating through the mask (ablation) | DC-ECCT · (31,16) and (32,11), Eb/N0=4 | -ln(BER): 5.16 -> 4.54; 4.40 -> 4.26 | Hurt |
| Sample random messages instead of using all-ones messages (ablation) | DC-ECCT · (31,16) and (32,11), Eb/N0=4 | -ln(BER): 5.16 -> 5.04; 4.40 -> 4.38 | Hurt |
Surprising findings
- Using m=0 (as in most neural decoder training) completely fails the joint code-decoder training.
- Learned codes improve classical BP decoding by large margins even when presented in standard form, suggesting the codes are good beyond the neural decoder.
Limitations
- Optimization is highly non-convex; results depend on initialization and no theoretical guarantees are provided.
- Training is limited to a single code at a time due to training complexity; multi-code optimization is not explored.
- Acceleration of the decoder (pruning, quantization, distillation) is left for future work.
Experimental setup
Datasets: LDPC(49,24) Polar(32,11) Polar(64,32) BCH(31,16) BCH(63,45) RS(60,52)
Benchmarks: LDPC(49,24) Polar(32,11) Polar(64,32) BCH(31,16) BCH(63,45) RS(60,52)
Metrics: negative natural log of bit error rate (-ln BER)
Baselines: BP SCL ECCT FECCT Maximum likelihood decoding
Builds on: ECCT FECCT straight-through estimator Transformer
Closest papers by meaning
- Error Correction Code Transformer (NeurIPS 2022 · similarity 0.92)
- A Foundation Model for Error Correction Codes (ICLR 2024 · similarity 0.87)
- Denoising Diffusion Error Correction Codes (ICLR 2023 · similarity 0.87)
- Friendly Attacks to Improve Channel Coding Reliability (AAAI 2024 · similarity 0.85)
- DeepPolar: Inventing Nonlinear Large-Kernel Polar Codes via Deep Learning (ICML 2024 · similarity 0.84)
- CrossMPT: Cross-attention Message-passing Transformer for Error Correcting Codes (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.