AI-driven Intrusion Detection System Using Machine Learning: An Extensive Study
Keywords:
Anomaly detection, Cybersecurity, Deep learning, machine learning, Network intrusion detection system, Network securityAbstract
Network Intrusion Detection Systems (NIDS) have gained significant importance in modern cybersecurity owing to the escalating frequency and complexity of cyber-attacks on enterprise networks, cloud computing platforms, and Internet of Things (IoT). Traditional signature-based NIDS cannot identify zero-day attacks and evolving attacks, which is why intelligent techniques like machine learning (ML) and deep learning have been introduced to identify threats. This survey reviews recent advancements in machine-learning-based and deep-learning-based intrusion detection by comparatively analyzing five primary research studies and supporting literature covering different datasets, attack scenarios, learning approaches, and research challenges. In this survey, common datasets used, feature engineering techniques, classifiers, and metrics have been described along with their pros and cons. Moreover, key challenges faced by current NIDS include dataset imbalance, false positives, inadequate cross-dataset generalization, detection of unknown attacks, computational cost, and lack of interpretability. Additionally, potential future research directions for intrusion detection, namely explainable artificial intelligence (XAI), federated learning, transformer-based IDSs, continual learning, and adaptive hybrid models, have been discussed in this survey.
References
I. H. Sarker, A. S. M. Kayes, S. Badsha, H. Alqahtani, P. Watters, and A. Ng, “Cybersecurity data science: an overview from machine learning perspective,” Journal of Big Data. vol. 7, Jul. 2020.
M. H. Bhuyan, D. K. Bhattacharyya and J. K. Kalita, “Network anomaly detection: Methods, systems and tools,” in IEEE Communications Surveys & Tutorials, vol. 16, no. 1, pp. 303–336, 2014.
D. Dasgupta, Z. Akhtar, and S. Sen, “Machine learning in cybersecurity: a comprehensive survey,” The Journal of Defense Modeling and Simulation, vol. 19, no. 1, pp. 57–106, Sept. 2022.
R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran, A. Al-Nemrat, and S. Venkatraman, “Deep learning approach for intelligent intrusion detection system,” in IEEE Access, vol. 7, pp. 41525–41550, 2019.
O. H. Abdulganiyu, T. A. Tchakoucht, and Y. K. Saheed, “RETRACTED ARTICLE: Towards an efficient model for network intrusion detection system (IDS): systematic literature review,” Wireless Networks,” vol. 30, pp. 453–482, 2024.
I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization,” in Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP), Funchal, Madeira, Portugal, 2018, pp. 108–116.
G. Engelen, V. Rimmer, and W. Joosen, “Troubleshooting an intrusion detection dataset: the CICIDS2017 case study,” 2021 IEEE Security and Privacy Workshops (SPW), San Francisco, CA, USA, 2021, pp. 7–12.
M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, 2009, pp. 1–6.
N. Moustafa and J. Slay, “UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set),” 2015 Military Communications and Information Systems Conference (MilCIS), Canberra, ACT, Australia, 2015, pp. 1–6.
A. Khraisat, I. Gondal, P. Vamplew, and J. Kamruzzaman, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, Jul. 2019.
S. Ho, S. A. Jufout, K. Dajani, and M. Mozumdar, “A novel intrusion detection model for detecting known and innovative cyberattacks using convolutional neural network,” in IEEE Open Journal of the Computer Society, vol. 2, pp. 14–25, 2021.
Yogesh and L. M. Goyal, “RETRACTED ARTICLE: Deep learning based network intrusion detection system: a systematic literature review and future scopes,” International Journal of Information Security, vol. 23, pp. 3433–3463, 2024.
M. Lanvin, P.-F. Gimenez, Y. Han, F. Majorczyk, L. Mé, and É Totel, “Errors in the CICIDS2017 dataset and the significant differences in detection performances it makes,” in International Conference on Risks and Security of Internet and Systems, 2022, pp. 18–33.
C. Hutabarat and Y. Asnar, “Development of machine learning module in intrusion detection system for unknown threat detection,” 2024 IEEE International Conference on Data and Software Engineering (ICoDSE), Gorontalo, Indonesia, 2024, pp. 114–119.
V. Nitin, Aarti and V. N. Thatha, “Deep learning for web application security: A comprehensive survey on intrusion detection systems,” 2025 2nd International Conference on Intelligent Systems for Cybersecurity (ISCS), Gurugram, India, 2025, pp. 1–7.
R. Fu, “Design and implementation of network intrusion detection system based on machine learning,” 2025 International Conference on Intelligent Systems and Computational Networks (ICISCN), Bidar, India, 2025, pp. 1–6.
M. Cantone, C. Marrocco, and A. Bria, “Machine learning in network intrusion detection: A cross-dataset generalization study,” in IEEE Access, vol. 12, pp. 144489–144508, 2024.
T. Zoppi, A. Ceccarelli, T. Puccetti, and A. Bondavalli, “Which algorithm can detect unknown attacks? Comparison of supervised, unsupervised and meta-learning algorithms for intrusion detection,” Computers & Security, vol. 127, Apr. 2023.
R. Ahmad, I. Alsmadi, W. Alhamdani, L. Tawalbeh,” Zero-day attack detection: a systematic literature review,” Artificial Intelligence Review, vol. 56, pp. 10733–10811, Feb. 2023.
I. R. Sabu, S. Saju, E. A. M. Anita, and T. Sowmya, “Detection of DoS attacks using machine learning based intrusion detection system,” 2024 IEEE International Conference on Contemporary Computing and Communications (InC4), Bangalore, India, 2024, pp. 1–9.
