Authors: Richard Staňa, Patrik Pekarčík, Andrej Gajdoš, Pavol Sokol
Abstract
The increasing number of cybersecurity threats affects the security situation of organisations. The maintenance of the operational picture of the organisation, which integrates all relevant information for selecting appropriate countermeasures, becomes a vital role for organisations. In this paper, we focus on network security situation awareness forecasting. The paper aims to answer two questions—the influence of loss function in neural networks on network security situation awareness forecasting and a comparison of statistical methods and neural networks in network security situation awareness forecasting. For this purpose, we used two-time series representing cybersecurity alerts collected by system Warden. This paper shows an analysis according to which the MAE and MASE loss functions give better results than MSE. Also, we can state that neural networks are more accurate for network security situation awareness forecasting.
Introduction
Nowadays, the number of new cybersecurity threats and cybersecurity incidents is on the rise. The main goal of organisations’ security teams is to prevent cybersecurity incidents or minimise their impact. For example, the organisations’ network administrators or security teams may prevent these incidents by disallowing the specific network protocols or updating systems to address security vulnerabilities. In this respect, we observe a trend of transition from reactive activities to proactive activities [1].
An important element in ensuring the proactive activities of the organisation is the maintenance of the operational picture of the organisation, which integrates all relevant information for identifying attacks and selecting appropriate countermeasures [2]. This operational picture can be defined as network security situation awareness (NSSA). Bass et al. introduced the origin, concept, target and characteristics of NSSA in more detail in [3].
According to a different perception of an object, NSSA can be divided into the network security situation assessment and network security situation forecasting [4]. Forecasting the security situation is an essential part of the NSSA and allows anticipating cybersecurity attacks and cybersecurity threats. It provides network administrators and security teams time to make adequate decisions on their next steps. Overall, this allows better analysing security threats and management of cybersecurity incidents.
Researchers have proposed and used various approaches to forecast network security situation awareness in recent years, such as statistical methods, game theory methods or neural networks. In the following section, we focus on state of the art in statistical methods and neural networks in more detail. At the same time, there are some problems in these methods, such as the loss of network data information caused by situation assessment and the low forecasting accuracy of the neural network model used for the NSSA forecasting [5]. To improve the accuracy of the NSSA forecasting, this paper aims to (I) analyse the influence of loss function in neural networks on the NSSA forecasting and (II) compare statistical methods and neural networks in NSSA forecasting.
This paper is based on previous research [6, 7]. Within this paper, we assume the fact that in the NSSA forecasting, there is a lot of time series forecasting with neural networks that look like naive forecasting with drift [8]. Definition of the mean absolute scaled error (MASE) shows that it compares forecasting with naive forecasting. Using MASE as a loss function, we can “punish” neural network when its forecasting looks like naive forecasting with drift.
This paper is organised into six sections. Section 2 reviews state of the art in network security awareness forecasting. Section 3 is devoted to research methodology and outlines the dataset and methods used for the analysis. Section 4 states the experimental evaluation. Section 5 discusses the results. The last section concludes the paper and discusses the challenges for future research.
