IDS Testing Methodology

Nicolas J Puketza et al, "A methodology for testing intrusion detection systems"
In this paper, the authors try to test the performance objectives of the IDS systems
Precondition - IDS systems should be installed and configured appropriately.
They have considered the basic testing procedure as described below and brought in variations in it
1. create or select a set of test scripts
2. establish the desired conditions in the computing environment
3. start the IDS
4. run the test scripts
5. analyze the IDS's output
The different IDS Tests considered include the following

1. Intrusion Identification tests
- Basic detection tests
a. create a set of intrusion scripts
b. as much as possible, eliminate unrelated computing acitvity in testing environment
c. start the IDS
d. run the intrusion scripts
Another technique, test results include a number associting it to warning based on the level of abnormal behavior. This would assist the testers in better classification of the abnormal behavior.
- Normal Users test - Here instead of the intruder scripts, normal user scripts are introduced resulting in the understanding on how often normal scenarios are flagged as intrusions.

2. Resource Usage tests
This is done to understand the resource consumption of the IDS systems. The main test is the disk space. The procedure adopted is as follows:
1. eliminate unrelated activity in the test environment
2. start the IDS
3. run the test script for a measured period of time (e.g., one hour) and
4. calculate the total disk space used by the IDS to record the session associated with the script.


Evaluation: