Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Linear Block Error Correction Codes (2405.04050v1)

Published 7 May 2024 in cs.IT, cs.AI, and math.IT

Abstract: Error correction codes are a crucial part of the physical communication layer, ensuring the reliable transfer of data over noisy channels. The design of optimal linear block codes capable of being efficiently decoded is of major concern, especially for short block lengths. While neural decoders have recently demonstrated their advantage over classical decoding techniques, the neural design of the codes remains a challenge. In this work, we propose for the first time a unified encoder-decoder training of binary linear block codes. To this end, we adapt the coding setting to support efficient and differentiable training of the code for end-to-end optimization over the order two Galois field. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient. Our results show that (i) the proposed decoder outperforms existing neural decoding on conventional codes, (ii) the suggested framework generates codes that outperform the {analogous} conventional codes, and (iii) the codes we developed not only excel with our decoder but also show enhanced performance with traditional decoding techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. End-to-end learning of communications systems without a channel model. In 2018 52nd Asilomar Conference on Signals, Systems, and Computers, pp.  298–303. IEEE, 2018.
  2. Arikan, E. Channel polarization: A method for constructing capacity-achieving codes. In 2008 IEEE International Symposium on Information Theory, pp.  1173–1177. IEEE, 2008.
  3. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013.
  4. Deep learning for decoding of linear codes-a syndrome-based approach. In 2018 IEEE International Symposium on Information Theory (ISIT), pp.  1595–1599. IEEE, 2018.
  5. On a class of error correcting binary group codes. Information and control, 3(1):68–79, 1960.
  6. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  7. Scaling deep learning-based decoding of polar codes via partitioning. In GLOBECOM 2017-2017 IEEE Global Communications Conference, pp.  1–6. IEEE, 2017.
  8. Graph neural networks for channel decoding. In 2022 IEEE Globecom Workshops (GC Wkshps), pp.  486–491. IEEE, 2022.
  9. Aff3ct: A fast forward error correction toolbox! Elsevier SoftwareX, 10:100345, October 2019. ISSN 2352-7110. doi: https://doi.org/10.1016/j.softx.2019.100345. URL http://www.sciencedirect.com/science/article/pii/S2352711019300457.
  10. Error correction code transformer. Advances in Neural Information Processing Systems (NeurIPS), 2022.
  11. Denoising diffusion error correction codes. In The Eleventh International Conference on Learning Representations (ICLR), 2023.
  12. Deep quantum error correction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.  64–72, 2024a.
  13. A foundation model for error correction codes. In The Twelfth International Conference on Learning Representations (ICLR), 2024b. URL https://openreview.net/forum?id=7KDuQPrAF3.
  14. Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in neural information processing systems, 28, 2015.
  15. ESTI. 5g nr multiplexing and channel coding. etsi 3gpp ts 38.212. https://www.etsi.org/deliver/etsi_ts/138200_138299/138212/16.02.00_60/ts_138212v160200p.pdf, 2021.
  16. Gallager, R. Low-density parity-check codes. IRE Transactions on information theory, 8(1):21–28, 1962.
  17. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp.  249–256. JMLR Workshop and Conference Proceedings, 2010.
  18. On deep learning-based channel decoding. In 2017 51st Annual Conference on Information Sciences and Systems (CISS), pp.  1–6. IEEE, 2017.
  19. Database of Channel Codes and ML Simulation Results. www.uni-kl.de/channel-codes, 2019.
  20. Sionna: An open-source library for next-generation physical layer research. arXiv preprint, Mar. 2022.
  21. Deepturbo: Deep turbo decoder. In 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), pp.  1–5. IEEE, 2019a.
  22. Turbo autoencoder: Deep learning based channel codes for point-to-point communication channels. Advances in neural information processing systems, 32, 2019b.
  23. Deepcode: Feedback codes via deep learning. In Advances in Neural Information Processing Systems (NIPS), pp.  9436–9446, 2018a.
  24. Communication algorithms via deep learning. In Sixth International Conference on Learning Representations (ICLR), 2018b.
  25. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  26. Opennmt: Open-source toolkit for neural machine translation. In Proc. ACL, 2017. doi: 10.18653/v1/P17-4012. URL https://doi.org/10.18653/v1/P17-4012.
  27. Boosting learning for ldpc codes to improve the error-floor performance. arXiv preprint arXiv:2310.07194, 2023.
  28. A survey of transformers. arXiv preprint arXiv:2106.04554, 2021.
  29. Hyper-graph-network decoders for block codes. In Advances in Neural Information Processing Systems, pp.  2326–2336, 2019.
  30. Autoregressive belief propagation for decoding block codes. arXiv preprint arXiv:2103.11780, 2021.
  31. Learning to decode linear codes using deep learning. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp.  341–346. IEEE, 2016.
  32. An introduction to machine learning communications systems. arXiv preprint arXiv:1702.00832, 2017.
  33. How to mask in error correction code transformer: Systematic and double masking. arXiv preprint arXiv:2308.08128, 2023.
  34. Pearl, J. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann, 1988.
  35. perm2vec: Graph permutation selection for decoding of error correction codes using self-attention. arXiv preprint arXiv:2002.02315, 2020.
  36. Crc-aided learned ensembles of belief-propagation polar decoders. arXiv preprint arXiv:2301.06060, 2023.
  37. Polynomial codes over certain finite fields. Journal of the society for industrial and applied mathematics, 8(2):300–304, 1960.
  38. Design of capacity-approaching irregular low-density parity-check codes. IEEE transactions on information theory, 47(2):619–637, 2001.
  39. Shannon, C. E. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423, 1948.
  40. Shazeer, N. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020.
  41. List decoding of polar codes. IEEE Transactions on Information Theory, 61(5):2213–2226, 2015.
  42. Attention is all you need. In Advances in neural information processing systems, pp.  5998–6008, 2017.
  43. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020.
  44. On layer normalization in the transformer architecture. In International Conference on Machine Learning, pp.  10524–10533. PMLR, 2020.
  45. Channel agnostic end-to-end learning based communication systems with conditional gan. In 2018 IEEE Globecom Workshops (GC Wkshps), pp.  1–5. IEEE, 2018.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now