An Introduction to Deep Learning for the Physical Layer

IEEE Transactions on Cognitive Communications and Networking

Volumn 3, Issue 4, 2017, Pages 563-575

  O'Shea, Timothy ^a   Hoydis, Jakob ^b  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b LUCENT TECHNOLOGIES   (United States)

Author keywords

cognitive radio; deep learning; digital communications; Machine learning; modulation; physical layer; radio communication

Indexed keywords

COGNITIVE RADIO; DIGITAL COMMUNICATION SYSTEMS; DIGITAL RADIO; ELECTRIC TRANSFORMERS; LEARNING SYSTEMS; MODULATION; NETWORK LAYERS; NEURAL NETWORKS; RADIO COMMUNICATION; RADIO TRANSMISSION; TRANSMITTERS;

COMMUNICATIONS SYSTEMS; CONVOLUTIONAL NEURAL NETWORK; DIGITAL COMMUNICATIONS; MACHINE LEARNING MODELS; MODULATION CLASSIFICATION; MULTIPLE TRANSMITTERS; PHYSICAL LAYERS; TRANSMITTER AND RECEIVER;

DEEP LEARNING;

EID: 85065897925     PISSN: None     EISSN: 23327731     Source Type: Journal    
DOI: 10.1109/TCCN.2017.2758370     Document Type: Conference Paper

References (66)

1. T. S. Rappaport, Wireless Communications: Principles and Practice, 2nd ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2002.
2. R. M. Gagliardi and S. Karp, Optical Communications, 2nd ed. New York, NY, USA: Wiley, 1995.
3. H. Meyr, M. Moeneclaey, and S. A. Fechtel, Digital Communication Receivers: Synchronization, Channel Estimation, and Signal Processing. New York, NY, USA: Wiley, 1998.
4. T. Schenk, RF Imperfections in High-Rate Wireless Systems: Impact and Digital Compensation. Dordrecht, The Netherlands: Springer, 2008.
5. J. Proakis and M. Salehi, Digital Communications, 5th ed. Boston, MA, USA: McGraw-Hill Educ., 2007.
6. Y. LeCun, "Generalization and network design strategies," in Connectionism in Perspective. Amsterdam, The Netherlands: North-Holland, 1989, pp. 143-155.
7. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proc. IEEE Int. Conf. Comput. Vis., Santiago, Chile, 2015, pp. 1026-1034.
8. D. G. Lowe, "Object recognition from local scale-invariant features," in Proc. IEEE Int. Conf. Comput. Vis., 1999, pp. 1150-1157.
9. Z. S. Harris, "Distributional structure," Word, vol. 10, nos. 2-3, pp. 146-162, 1954.
10. A. Goldsmith, "Joint source/channel coding for wireless channels," in Proc. IEEE Veh. Technol. Conf., vol. 2. Chicago, IL, USA, 1995, pp. 614-618.
11. E. Zehavi, "8-PSK trellis codes for a Rayleigh channel," IEEE Trans. Commun., vol. 40, no. 5, pp. 873-884, May 1992.
12. H. Wymeersch, Iterative Receiver Design, vol. 234. Cambridge, U.K.: Cambridge Univ. Press, 2007.
13. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Netw., vol. 2, no. 5, pp. 359-366, 1989.
14. S. Reed and N. de Freitas, "Neural programmer-interpreters," arXiv preprint arXiv:1511.06279, 2015.
15. H. T. Siegelmann and E. D. Sontag, "On the computational power of neural nets," in Proc. 5th Annu. Workshop Comput. Learn. Theory, Pittsburgh, PA, USA, 1992, pp. 440-449.
16. V. Vanhoucke, A. Senior, and M. Z. Mao, "Improving the speed of neural networks on CPUs," in Proc. Deep Learn. Unsupervised Feature Learn. NIPS Workshop, vol. 1. 2011, p. 4.
17. Y.-H. Chen, T. Krishna, J. S. Emer, and V. Sze, "Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks," IEEE J. Solid-State Circuits, vol. 52, no. 1, pp. 127-138, Jan. 2017.
18. R. Raina, A. Madhavan, and A. Y. Ng, "Large-scale deep unsupervised learning using graphics processors," in Proc. Int. Conf. Mach. Learn. (ICML), Montreal, QC, Canada, 2009, pp. 873-880.
19. M. Ibnkahla, "Applications of neural networks to digital communications-A survey," Elsevier Signal Process., vol. 80, no. 7, pp. 1185-1215, 2000.
20. M. Bkassiny, Y. Li, and S. K. Jayaweera, "A survey on machine-learning techniques in cognitive radios," IEEE Commun. Surveys Tuts., vol. 15, no. 3, pp. 1136-1159, 3rd Quart., 2013.
21. M. Zorzi, A. Zanella, A. Testolin, M. D. F. De Grazia, and M. Zorzi, "Cognition-based networks: A new perspective on network optimization using learning and distributed intelligence," IEEE Access, vol. 3, pp. 1512-1530, 2015.
22. E. Nachmani, Y. Be'ery, and D. Burshtein, "Learning to decode linear codes using deep learning," in Proc. IEEE Annu. Allerton Conf. Commun. Control Comput. (Allerton), Monticello, IL, USA, 2016, pp. 341-346.
23. E. Nachmani, E. Marciano, D. Burshtein, and Y. Be'ery, "RNN decoding of linear block codes," arXiv preprint arXiv:1702.07560, 2017.
24. N. Samuel, T. Diskin, and A. Wiesel, "Deep MIMO detection," arXiv preprint arXiv:1706.01151, 2017.
25. J. R. Hershey, J. L. Roux, and F. Weninger, "Deep unfolding: Model-based inspiration of novel deep architectures," arXiv preprint arXiv:1409.2574, 2014.
26. M. Borgerding and P. Schniter, "Onsager-corrected deep learning for sparse linear inverse problems," in Proc. IEEE Glob. Conf. Signal Inf. Process. (GlobalSIP), Washington, DC, USA, 2016, pp. 227-231.
27. Y.-S. Jeon, S.-N. Hong, and N. Lee, "Blind detection for MIMO systems with low-resolution ADCs using supervised learning," in Proc. IEEE Int. Conf. Commun. (ICC), Paris, France, 2017, pp. 1-6.
28. N. Farsad and A. Goldsmith, "Detection algorithms for communication systems using deep learning," arXiv preprint arXiv:1705.08044, 2017.
29. H. Sun et al., "Learning to optimize: Training deep neural networks for wireless resource management," arXiv preprint arXiv:1705.09412, 2017.
30. T. J. O'Shea, K. Karra, and T. C. Clancy, "Learning to communicate: Channel auto-encoders, domain specific regularizers, and attention," in Proc. IEEE Int. Symp. Signal Process. Inf. Technol. (ISSPIT), Limassol, Cyprus, 2016, pp. 223-228.
31. T. J. O'Shea, J. Corgan, and T. C. Clancy, "Convolutional radio modulation recognition networks," in Proc. Int. Conf. Eng. Appl. Neural Netw., Aberdeen, U.K., 2016, pp. 213-226.
32. T. J. O'Shea, J. Corgan, and T. C. Clancy, "Unsupervised representation learning of structured radio communication signals," in Proc. IEEE Int. Workshop Sens. Process. Learn. Intell. Mach. (SPLINE), Aalborg, Denmark, 2016, pp. 1-5.
33. T. Gruber, S. Cammerer, J. Hoydis, and S. T. Brink, "On deep learningbased channel decoding," in Proc. IEEE 51st Annu. Conf. Inf. Sci. Syst. (CISS), Baltimore, MD, USA, 2017, pp. 1-6.
34. S. Cammerer, T. Gruber, J. Hoydis, and S. T. Brink, "Scaling deep learning-based decoding of polar codes via partitioning," arXiv preprint arXiv:1702.06901, 2017.
35. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
36. N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting," J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929-1958, 2014.
37. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," in Proc. Int. Conf. Mach. Learn. (ICML), Haifa, Israel, 2010, pp. 807-814.
38. Y. Jia et al., "Caffe: Convolutional architecture for fast feature embedding," in Proc. 22nd ACM Int. Conf. Multimedia, Orlando, FL, USA, 2014, pp. 675-678.
39. T. Chen et al., "MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems," arXiv preprint arXiv:1512.01274, 2015.
40. M. Abadi et al. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. [Online]. Available: http://tensorflow.org/
41. R. Al-Rfou et al., "Theano: A Python framework for fast computation of mathematical expressions," arXiv preprint arXiv:1605.02688, 2016.
42. R. Collobert, K. Kavukcuoglu, and C. Farabet, "Torch7: A MATLABlike environment for machine learning," in Proc. BigLearn NIPS Workshop, 2011, pp. 1-6.
43. F. Chollet. (2015). Keras. [Online]. Available: https://github.com/fchollet/keras
44. T. O'Shea and J. Hoydis. (2017). Source Code. [Online]. Available: https://github.com/radioml/introdlphy/
45. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
46. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in Proc. Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, 2015, pp. 1-15.
47. Y. N. Dauphin et al., "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), Montreal, QC, Canada, 2014, pp. 2933-2941.
48. L. V. D. Maaten and G. Hinton, "Visualizing data using t-SNE," J. Mach. Learn. Res., vol. 9, pp. 2579-2605, Nov. 2008.
49. I. Goodfellow et al., "Generative adversarial nets," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2014, pp. 2672-2680.
50. M. Abadi and D. G. Andersen, "Learning to protect communications with adversarial neural cryptography," arXiv preprint arXiv:1610.06918, 2016.
51. M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu, "Spatial transformer networks," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2015, pp. 2017-2025.
52. J. Estaran et al., "Artificial neural networks for linear and non-linear impairment mitigation in high-baudrate IM/DD systems," in Proc. 42nd Eur. Conf. Opt. Commun. (ECOC), Düsseldorf, Germany, 2016, pp. 1-3.
53. A. K. Nandi and E. E. Azzouz, "Algorithms for automatic modulation recognition of communication signals," IEEE Trans. Commun., vol. 46, no. 4, pp. 431-436, Apr. 1998.
54. A. Fehske, J. Gaeddert, and J. H. Reed, "A new approach to signal classification using spectral correlation and neural networks," in Proc. IEEE Int. Symp. New Front. Dyn. Spectr. Access Netw. (DYSPAN), Baltimore, MD, USA, 2005, pp. 144-150.
55. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, 2015.
56. F. Pedregosa et al., "Scikit-learn: Machine learning in Python," J. Mach. Learn. Res., vol. 12, pp. 2825-2830, Feb. 2011.
57. A. Abdelmutalab, K. Assaleh, and M. El-Tarhuni, "Automatic modulation classification based on high order cumulants and hierarchical polynomial classifiers," Phys. Commun., vol. 21, pp. 10-18, Dec. 2016.
58. D. George and E. A. Huerta, "Deep neural networks to enable real-time multimessenger astrophysics," arXiv preprint arXiv:1701.00008, 2016.
59. D. Maclaurin, D. Duvenaud, and R. P. Adams, "Gradient-based hyperparameter optimization through reversible learning," in Proc. 32nd Int. Conf. Mach. Learn. (ICML), Lille, France, 2015, pp. 2113-2122.
60. J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, Jan. 2012.
61. A. Hirose, Complex-Valued Neural Networks. Heidelberg, Germany: Springer, 2006.
62. M. F. Amin, M. I. Amin, A. Y. H. Al-Nuaimi, and K. Murase, "Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks," in Proc. Int. Conf. Neural Inf. Process., Shanghai, China, 2011, pp. 550-559.
63. G. C. Goodwin and R. L. Payne, Dynamic System Identification: Experiment Design and Data Analysis. New York, NY, USA: Academic Press, 1977.
64. S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
65. F. Adib, C.-Y. Hsu, H. Mao, D. Katabi, and F. Durand, "Capturing the human figure through a wall," ACM Trans. Graph., vol. 34, no. 6, p. 219, 2015.
66. M. Zhao, F. Adib, and D. Katabi, "Emotion recognition using wireless signals," in Proc. ACM Annu. Int. Conf. Mobile Comput. Netw., New York, NY, USA, 2016, pp. 95-108.