Characterization and Solution to A Stateful IDS Evasion

Issam Aib, Tung Tran, and Raouf Boutaba
University of Waterloo, Waterloo, ON, Canada
(iaib, t3tran, rboutaba)@uwaterloo.ca
2009 29th IEEE International Conference on Distributed Computing Systems

In this paper, the authors discuss about the concept of signature evasion, where-in the attackers on purpose try to tamper with the flow stored as a regular flow to complete the attack.

The authors explain this concept through an FTP session intrusion. The steps or states involved in the flow of a normal and admin user authentication are coded as rule set and. Here, the authors say that a normal user by following his states, can innately access a file which he/ she is not authorized to view. This is possible without raising any alarm.

Among the set of rules that are defined for a specific operation, if a fake network packet (in the example scenario) can trigger a signature, then the signature is considered to be evasive (i.e. something which can be deceived easily)

Thus in order to address such a scenario, they have come up with an algorithm based on deterministic finite automate, i.e. using the states of the flow to raise the false positive alarms in the right time.

The set of rules that are defined for a specific flow of operation could contain both an evasive rule and a target rule (which assists in identifying the attack).

Understanding about Finite State Automata machines

Finite State Automata are helpful in depicting the states and their relationship from start to end of a system. FSM representation could be done either using a state chart or regular expressions.

In the case of intrusion detection systems, if we go by the raw definition of FSM, it is feasible for us to implement a FSM based detection technique. Now let us assume that the detection technique is implemented using an FSM. Then will the technique not be misuse. Is it possible for it to be anomalous in nature? In the misuse scenario, we normally store all the possible intrusions that we have come across and then try to match with the incoming intrusions and raise an alarm. In the case of anomalous, we do the opposite thing, where-in the system is trained with the expected behavior. The expected series of behavior could be classified using the FSM.

Now the question is, if we use FSM, is there a possibility of reducing the false alarm rates?

Above the advantages of FSM usage, the motive here in front of us is to understand the possibility of generating test cases from the IDS FSM.

IDS systems using FSM technique

http://www.springerlink.com/content/g25w816354413354/fulltext.pdf

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4285750 (A Memory-Efficient Reconfigurable Aho-Corasick FSM Implementation for Intrusion Detection Systems)

http://opensiuc.lib.siu.edu/cgi/viewcontent.cgi?article=1009&context=ece_articles (Self-Addressable Memory-Based FSM: A Scalable Intrusion Detection Engine)

Generation and use of test data sets in IDS testing

Vidar Evenrud Seeberg, "Generation and use of test data sets in IDS testing", September 16, 2005

When evaluating an IDS, the evaluator can choose mainly between four approaches in
generating and using test data sets:

The evaluator can base the test on an empty test data set (no background trac)
 The evaluator can generate test data by recording real network trac
 The evaluator can generate test data by sanitizing recorded real network trac
 The evaluator can generate test data using simulated traffic

Quoted reference to arrive at the approaches-

P Mell, V Hu, R Lippmann, J Haines, and M Zissman. An overview of issues in testing
intrusion detection systems. Technical Report NIST IR 7007, National Institute of
Standards and Technology, August 2003.

Intrusion Detection Testing and Benchmarking Methodologies

Nicholas Athanasides, Randal Abler et al, "Intrusion Detection Testing and Benchmarking Methodologies", Georgia Institute of Technology, Proceedings of the First IEEE International workshop on Information assurance (IWIA '03)

The authors discuss the existing tools and testing methodologies for performing benchmark testing of intrusion detection systems. Based on their study they propose the use of an open source environment to execute the testing.

Environments discussed include:

1. DARPA Environment
2. LARIAT environment

In addition the authors have also listed the test suite and the tools that could be used

1. Nidsbench and IDS Wakeup
2. IDSwakeup
3. Flame Thrower
4. WebAvalanche/ WebReflector
5. Tcpreplay
6. Fragrouter
7. Hping2
8. Iperf

- Issues in generating realistic evaluation environments are also discussed

- Examples of IDS Evaluation environments include:

1. DARPA Like Environment
2. Custom Software
3. Advanced security audit trail analysis on Unix
4. Vendor Independent testing lab
5. Trade magazine evaluation

A methodology of testing intrusion detection systems

Nicholas J.Puketza, et al, "A methodology of testing intrusion detection systems", University of California

- Elaborate information is provided on the testing methodology to assess the performance of IDS

- the authors have quoted that test data could be obtained from sources such as CERT advisories, periodicals such as PHRACK and 2600, USENET, analyzing the vulnerabilities detected by security tools such as COPS and TIGER

Possible articles that could be referred to related to obtaining intrusion data:

1. M. Bishop, \A Taxonomy of UNIX System and Network Vulnerabilities," Technical
Report CSE-95-10, University of California at Davis, September 1995.
2. D. Farmer and W. Venema, \Improving the Security of Your Site by Breaking Into
It," USENET posting, December 1993.
3. D. Farmer and E. H. Spaord, \The COPS Security Checker System," Proc., Summer
USENIX Conference, pp. 165-170, June 1990.
4. S. Kumar and E. H. Spaord, \A Pattern Matching Model for Misuse Intrusion Detec-
tion," Proc., 17th National Computer Security Conference, Baltimore, MD, pp. 11-21,
October 1994.
5. D. R. Saord, D. L. Schales, and D. K. Hess, \The TAMU Security Package: An
Ongoing Response to Internet Intruders in an Academic Environment," Proc., Fourth
USENIX UNIX Security Symposium, Santa Clara, CA, pp. 91-118, October, 1993.

- test case selection based on 3 different strategies is elaborated

The authors have not provided in depth information on how large volume of test data could be generated, but they have given guidance related to the direction which needs to be adopted to achieve it

Generation of test data for testing intrusion detection systems

Internet has information related to the generation of test data for testing IDS systems in assessing its efficiency. detection techniques are implemented based on finite state machines and descriptive languages for the systems in place. But there is no sufficient data available on how these systems are tested.

Following listing consists of the papers available in this area along with the amount of information it contains pertaining to test data generation and usage of test data generation tools if any.

- Nicholas J.Puketza, et al, "A methodology of testing intrusion detection systems", University of california

- Nicholas Athanasides, Randal Abler et al, "Intrusion Detection Testing and Benchmarking Methodologies", Georgia Institute of Technology, Proceedings of the First IEEE International workshop on Information assurance (IWIA '03)

- Ming Li and Wei Zhao, "A principle of a data synthesizer for Performance test of anti-DDOS Flood Attacks", International journal of computers, issue 3, volume 1, 2007

- Vidar Evenrud Seeberg, "Generation and use of test data sets in IDS testing", September 16, 2005

- John McHugh, "Testing Intrusion Detection Systems: A Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory"

- Darren Mutz et al, "An Experience Developing an IDS Simulator for the Black Box testing of Network Intrusion Detection Systems", Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC 2003)

- Roy A. Maxion et al, "Benchmarking Anomaly-based detection systems", 2000 IEEE

- Emilie Lundin Barse et al, "Synthesizing Test Data for Fraud Detection Systems"

- R Sekar, A Gupta et al, "Specification based anomaly detection : A new approach for detecting network intrusions", CCS'02 November 18-22, 2002, Washington DC, USA, ACM

Tools for testing IDS

1. RACOON - RACOON: Rapidly Generating User Command Data For Anomaly Detection From Customizable Templates (http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01377229)
2. fTester - (http://dev.inversepath.com/trac/ftester) Firewall and IDS Testing tool
3. Tcpreplay - network traffic testing (http://tcpreplay.synfin.net/trac/)
4. Nemesis - well suited for testing Network Intrusion Detection Systems, firewalls, IP stacks and a variety of other tasks (http://nemesis.sourceforge.net/)
5. IDSwakeup - IDSwakeup is to generate false attack that mimic well known ones, in order to see if NIDS detects them and generates false positives
6. Unix tool EXPECT - Simulation of “normal” and “intruder” behaviour.
Extends TCL interpreter to provide simulation scripts. (http://expect.nist.gov/)
7. Fragrouter: Routes network traffic such that it elude most NIDS. (http://www.securityfocus.com/tools/176)

Issues involved in anomaly based detection technique

1. Inadvertently including malicious activity as part of a profile is a common problem with anomaly-based IDPS (Intrusion Detection and Prevention systems) products
2. It can be very challenging to build profiles in some cases to make them accurate, because computing activity can be so complex.
3. It is often difficult for analysts to determine why a particular alert was generated and to validate that an alert is accurate and not a false positive, because of the complexity of events and number of events that may have caused the alert to be generated.

Source - http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

Evaluating Network Intrusion Detection Signatures

http://www.securityfocus.com/infocus/1623

Evaluating Network Intrusion Detection Signatures, Part One

http://www.securityfocus.com/infocus/1630

Evaluating Network Intrusion Detection Signatures, Part Two

IDS measurements which can be used to assess performance accuracy

source - TESTING INTRUSION
DETECTION SYSTEMS
Elizabeth B. Lennon, Editor
Information Technology Laboratory
National Institute of Standards and Technology

1. Coverage
2. Probability of False Alarms
3. Probability of Detection
4. Resistance to attacks directed at the IDS
5. Ability to Handle High Bandwidth Traffic
6. Ability to Correlate Events
7. Ability to Detect Never-Before-Seen Attacks
8. Ability to Identify an Attack
9. Ability to Determine Attack Success
10. Capacity Verification for NIDS

USim: A User Behavior Simulation Framework for Training and Testing IDSes in GUI Based Systems

by A. Garg, S.Vidyaraman, S. Upadhyaya et al,

Currently reading

Information to be looked at

Firewall and IDS Testing toolhttp://www.secguru.com/link/firewall_and_ids_testing_tool

IDS Testing info from NIST website
http://www.itl.nist.gov/lab/bulletns/

http://csrc.nist.gov/publications/nistbul/bulletin07-03.pdf
http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf
http://csrc.nist.gov/publications/nistbul/04-2004.pdf
http://csrc.nist.gov/publications/nistir/nistir-7007.pdf

An Experience Developing an IDS Stimulator for the Black-Box Testing of Network Intrusion Detection Systems (2003)

Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation (2000)

Testing Network Intrusion Detection Systems (2006) (http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.129.9810)

Data Collection Mechanisms for Intrusion Detection Systems (2000)

Intrusion Detection Systems - Technologies, . . . (2003)

Automated Audit Trail Analysis and Intrusion Detection: A Survey (1988)