Towards a framework for the generation of enhanced attack/background network traffic for evaluation of network-based intrusion detection systems

Lo, O. C. W.; Graves, J. R.; Buchanan, W. J.

Abstract

There are a multitude of threats faced in computer networks such as viruses, worms, trojans, attempted user privilege gain, data theft and denial of service attacks. To combat such threats, multiple lines of defence are applied to a network including firewalls, malicious software scanners and intrusion detection systems (IDS). IDSs are generally considered a last line of defence for the detection of attacks; therefore, it is vital for users to assess how well an IDS will perform through means of testing. Although various methodologies have been proposed for the evaluation of IDSs in the past there is still no widely agreed upon standard.
A framework which is capable of carrying out an evaluation of network-based intrusion detection systems (NIDS) is presented in this paper. The paper shows that such a framework requires the need for both realistic real-time network traffic and meaningful metrics when carrying out an evaluation of IDSs. Automation of the testing process is also emphasised - which provides for ease-of-use and simplicity in repetition when carrying out an evaluation.
The framework is evaluated against the NIDS Snort in order to show its capabilities. Through the use of pre-existing programs and utilities, the aim of generating real-time attack traffic is achieved whilst benign background traffic is generated using static data sets. The metrics of efficiency, effectiveness, packet loss, CPU utilisation and memory usage are derived and, finally, the goal of automation is achieved by implementing the framework as a singular application. The results of the evaluation show that, whilst Snort is highly effective in the detection of attacks (true-positives), its main weakness is the dropping of network packets at higher CPU utilisations due to high traffic volume.
Finally, the conclusion to this paper illustrates that the main weakness with current IDS evaluation methodologies is in the approaches used in the generation of benign background traffic. Whilst using static data sets is viable, the main argument against such an approach is that an IDS under evaluation will not react to the traffic in a real-time manner. Furthermore, the use of synthetic traffic generators also has limitations due to the fact that such traffic may not accurately reflect traffic seen on a live network. This paper proposes that further research and development must be applied in the area of benign traffic generation in order to achieve the aim of providing real-time generation of background traffic which realistically mirrors real-life networks when carrying out an evaluation of IDSs.

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: European Conference on Information Warfare and Security
• Start Date: Jul 1, 2010
• End Date: Jul 2, 2010
• Acceptance Date: Jul 1, 2010
• Online Publication Date: Jul 2, 2010
• Publication Date: 2010
• Deposit Date: Aug 31, 2010
• Publicly Available Date: Aug 31, 2010
• Publisher: Academic Publishing Limited
• Peer Reviewed: Peer Reviewed
• Pages: 190-200
• Book Title: Proceedings of 9th European Conference on Information Warfare and Security
• ISBN: 9781906638672 9781622765355
• Keywords: Network traffic; Network-Based Intrusion Detection Systems; evaluation framework; attack traffic; background traffic; evaluation metrics
• Public URL: http://researchrepository.napier.ac.uk/id/eprint/3817
• Contract Date: Aug 31, 2010