Adaptive Quantum-Inspired Optimization for Nonlinear Distortion Compensation in Optical Fiber Networks
Keywords:
Bit Error Rate (BER), Digital Back Propagation (DBP), Quality factor, Quantum-Inspired Optimization (QIO), Wavelength Division Multiplexing (WDM)Abstract
Nonlinear impairments remain a major challenge in Wavelength Division Multiplexing (WDM) optical fiber networks, particularly at high launch powers and long transmission distances. Conventional compensation techniques such as Digital Backpropagation (DBP) and Machine Learning (ML)-based methods improve transmission quality but often suffer from high computational complexity, extensive training requirements, and limited adaptability. This study proposes an Adaptive Quantum-Inspired Optimization (QIO) framework for mitigating nonlinear impairments in WDM optical fiber communication systems. The proposed approach integrates a quantum-inspired annealing optimization engine with a Nonlinear Schrödinger Equation (NLSE)-based optical transmission model to adaptively optimize launch power, amplifier gain, and digital signal processing parameters. An 8-channel WDM system was modeled in MATLAB and OptiSystem, and its performance was evaluated using the Q-factor, Bit Error Rate (BER), and Optical Signal-To-Noise Ratio (OSNR). The proposed QIO method achieved a peak Q-factor of , compared with for ML, for DBP, and for the uncompensated system. It also produced the lowest BER of outperforming ML ( ), DBP ( ), and the uncompensated system ( at launch power. For long-haul transmission over 500km, the proposed method maintained an OSNR of 19.8 dB, exceeding ML ( ), DBP ( ), and the uncompensated system ( ). These results demonstrate that the proposed QIO framework provides effective nonlinear impairment mitigation while maintaining moderate computational complexity, making it a promising solution for next-generation high-capacity optical fiber communication systems.
References
Z. Tan, and C. Lu, “Optical fiber communication technology: Present status and prospect,” Strategic Study of CAE,” vol. 22, no. 3, pp. 100–107, 2020.
E. Agrell and M. Secondini, “Information-theoretic tools for optical communications engineers,” 2018 IEEE Photonics Conference (IPC), Reston, VA, USA, 2018, pp. 1–5.
R. Dar, M. Feder, A. Mecozzi, and M. Shtaif, ‘Accumulation of nonlinear interference noise in fiber-optic systems,’ Optics Express, vol. 22, no. 12, pp. 14199–14211, Jun. 2014.
S. Civelli, D. Cellini, E. Forestieri, and M. Secondini, “Fiber nonlinearity mitigation in coherent optical systems,” arXiv, 2025.
C. Häger and H. D. Pfister, “Nonlinear interference mitigation via deep neural networks,” 2018 Optical Fiber Communications Conference and Exposition (OFC), San Diego, CA, USA, 2018, pp. 1–3.
C. A. Montoya Ocampo, K. D. Martinez Zapata, and J. J. Granada Torre, “Techniques for physical layer equalization and monitoring in radio-over-fiber: DSP and machine learning perspective,” Physical Communication, vol. 75, Mar. 2026.
M. O. Butt, N. Waheed, T. Q. Duong and W. Ejaz, “Quantum-inspired resource optimization for 6G networks: A survey,” in IEEE Communications Surveys & Tutorials, vol. 27, no. 5, pp. 2973–3019, Oct. 2025.
J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, “Quantum machine learning,” Nature, vol. 549, pp. 195–202, 2017.
F. Arute et al., “Quantum supremacy using a programmable superconducting processor,” Nature, vol. 574, pp. 505–510, 2019.
Y. Qian, X. Wang, Y. Du, X. Wu and D. Tao, “The dilemma of quantum neural networks,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 4, pp. 5603–5615, Apr. 2024
S. Monaco, O. Kiss, A. Mandarino, S. Vallecorsa, and M. Grossi, “Quantum phase detection generalization from marginal quantum neural network models,” Physical Review B, vol. 107, no. 8, 2023.
Y. Kwak, W. J. Yun, S. Jung, and J. Kim, “Quantum neural networks: Concepts, applications, and challenges,” in Proceedings of 12th International Conference on Ubiquitous and Future Networks, Jeju Island, South Korea, 2021, pp. 413–416.
T. Haug, K. Bharti, and M. S. Kim, “Capacity and quantum geometry of parametrized quantum circuits,” PRX Quantum, vol. 2, Oct. 2021.
M. Larocca, N. Ju, D. García-Martín, P. J. Coles, and M. Cerezo, “Theory of overparametrization in quantum neural networks,” Nature Computational Science, vol. 3, pp. 542–551, Jun. 2023.
Z. Qu, Y. Li, and P. Tiwari, “QNMF: A quantum neural network-based multimodal fusion system for intelligent diagnosis,” Infusion Fusion, vol. 100, Dec. 2023.
G. Manavalan and S. Arnon, “Quantum neural network-based compensation of distorted orbital angular momentum beams in complex media,” Scientific Reports, vol. 16, Dec. 2026.
G. P. Agrawal, Nonlinear Fiber Optics, 6th ed. Cambridge, MA, USA: Academic Press, 2021.
T. Sabapathi and V. Bhavasri, “Analysis of fiber nonlinearities while deployed in 5G communication system,” in Cellular Communication System-The Evolution from 1G to 6G. London, U.K.: IntechOpen, 2026.
E. Ip and J. M. Kahn, “Compensation of dispersion and nonlinear impairments using digital backpropagation,” Journal of Lightwave Technology, vol. 26, no. 20, pp. 3416–3425, Oct.15, 2008.
O. Soman, Digital signal processing techniques for fiber nonlinearity compensation in coherent optical communication systems, Ph.D. dissertation, Memorial University of Newfoundland, St. John’s, NL, Canada, 2020.
I. Sajjad, and M. M. A. Sobh, “The quantum-inspired adaptive superposition optimization for neural network training,” AIMS Mathematics, vol. 11, no. 1, pp. 243–271, 2026.
F. N. Khan, C. Lu, and A. P. T. Lau, “Machine learning techniques for optical communication systems,” Advanced Photonics 2017, 2017.
V. Neskorniuk, Deep learning methods for nonlinearity mitigation in coherent fiber-optic communication links, Ph.D. dissertation, Aston University, Birmingham, U.K., Nov. 2022.
H. Zhang, R, Zo, Z. Lv, and Z. Wang,” Application and research of quantum optimisation control algorithms in adaptive optics systems,” Frontiers in Computing and Intelligent Systems, vol. 15, no. 3, pp. 89–99, 2026.
P. Botsinis, D. Alanis, Z. Babar, H. V. Nguyen, D. Chandra, S. X. Ng, and L. Hanzo, “Quantum search algorithms for wireless communications,” IEEE Communications Surveys & Tutorials, vol. 21, no. 2, pp. 1209–1242, 2019.
M. A. Nielsen and I. L. Chuang, Quantum computation and quantum information, 10th ed. New York, NY, USA: Cambridge University Press, 2011.
S. Imre and L. Gyongyosi, Advanced quantum communications: An engineering approach. Hoboken, NJ, USA: Wiley, 2013.
B. I. Bakare, W. Minah-Eeba, S. Orike, and C. O. Ahiakwo, “Optimization of hybrid optical amplifiers for enhanced WDM efficiency,” Journal of Optoelectronics and Communication, vol. 7, no. 3, 2025.
