Clustering Techniques for Anomaly Detection in Cybersecurity Logs: A Survey

Abstract

A pivotal issue in the current digitally interconnected world is cyber security because networks and systems are continuously threatened by more advanced attacks. Checking cyber security logs is one of the main ways of detecting such threats, as the said documents capture the activities occurring on the system and can help detect unusual actions. The huge size, the dimensionality, as well as the unstructured data make it impractical to analyse the logs manually or on a rule-based approach that is usually ineffective in identifying new or latent threats. The paper is a review article that is based on the detection of anomalies in security log data using advanced clustering algorithms. The addressed problem is that it is very hard to correctly locate anomalies, which can be clues of cyber-attacks, in large and unlabeled datasets and that conventional detection methods cannot be scaled or generalized. Compared to other supervised approaches to learning, where labelled data are used, anomaly detection in the context of clustering is unsupervised and therefore applicable to real-world security situations, where novel attack patterns are often discovered. The task of the present survey is to analyze and contrast various clustering methods, both classic, such as K-Means, DBSCAN and hierarchical clustering, and newer algorithms, such as OPTICS, fuzzy C-means, deep clustering algorithms. These techniques cluster the data by similarity allowing outliers to be found and possibly be signifying an ill intent. These algorithms are reviewed on some parameters such as accuracy in detection, efficiencies, scalability towards large size of input, and robustness to noise or completeness which can be a common problem in log analysis. One of the findings that the study has emphasized is a trade-off which exists between precision of detection and false positives. The sensitivity may be so high that it has the potential to flood security analysts with alerts, and low sensitivity may miss important threats. This survey will give an account of the use of the clustering algorithm as an anomaly detection model in cybersecurity log information. It inherently tweaks out what the existing methods are today, strong and weak, and points out how there is a demand of better ways of doing this that are more adaptive, explainable, and efficient. Clustering-based anomaly detection is a potentially good research and development in the creation of pro-active and intelligent defense mechanisms based on connecting the gap between data science methods and practices as far as efficient cybersecurity requirements are concerned.