Kinds of evasions

Now the next point to consider is ... do all IDS systems have common evasions?

When we read the paper on fixing the evasions through FSM, snort was the system under consideration ... there is no specific information provided regarding to what extent where the evasions present in the system were identified ... how can we confirm that if we have identified certain types of evasions we have addressed all that present in the specific IDS system.

Now concentrating on the basic evasion types as described in the case of SQL injection ... next question is ... if we use snort ... what kinds of intrusions in general it would address ...

http://en.wikipedia.org/wiki/Intrusion_detection_system_evasion_techniques ... talks about vulnerability scanners that also incorporate IDS evasion techniques

http://www.mail-archive.com/issforum@iss.net/msg02072.html --> "ADMmutate is a shellcode mutation engine, can evade NIDS

A way of breaking into a system. An exploit takes advantage of a weakness in a system in order to hack it. Exploits are the root of the hacker culture. Hackers gain fame by discovering an exploit. Others gain fame by writing scripts for it. Legions of script-kiddies apply the exploit to millions of systems, whether it makes sense or not. Since people make the same mistakes over-and-over, exploits for very different systems start to look very much like each other. Most exploits can be classified under major categories: buffer overflow, directory climbing, defaults, Denial of Service."

thoughts on evasion identification

Possible study that should be undertaken:

When we draw an FSM for a specific situation which gets translated to a ruleset , all that we can identify is that the evasive path ... what could a possible evasive path ? Is it not the state where-in the alarm trigger is expected and there is no alarm raised for an attack ... so in this scenario ... even if we are able to generate test cases from the FSM ... how far is this going to be towards our goal of identifying evasions in the IDS system from the users point of view ...

Now let us consider a situation ... where-in we are aware of the possible modes of evasion available for a kind of attack ... say sql-injection ... would it not be beneficial to identify a testing technique ... that mainly assess the IDS system if it is evasive ... and then maybe compare the systems for check which one performs better with respect to a given evasion ...

Is it not better to have a tool developed which would automatically test the IDS systems for the evasions and check what all evasions it is vulnerable to ? Coz for a developer before he/ she uses the IDS system they want to understand if it is prone to known evasions ... this could also be done from the IDS system developer point of view ... they need a tool to identify ... now the next question is how is this different from the AGENT tool developed ... as we understand the tool is mainly used to generate automated test data which can be run on the IDS system there-by identifying possible evasions by going through the code ... now what we want to avoid is reading through the code to understand the evasions ...

first point we need to be clear is who we are targetting this application ... if it for the IDS system developers ... then there is a possibility that there might be new evasions ... if we are targetting for the developers who are going to use the IDS system then ... we can assume that the evasions that needs to be checked are mostly known ... hence the tool should directly test the IDS system for those evasions ... ?

FSM for SQL injection

SQL Injection attack is found to be one of the most vulnerable in the case of websites. Identity thefts seem to be happening often due to the support of the attack.
Hence the white paper "The SQL Injection and Signature Evasion" discusses the possible evasion techniques that are prevalent and also quotes how the products of Impervia may suit protection against them.

Our focus here is mainly to understand the various evasion techniques available. This is done by using the SQL injection example quoted by them, drawing FSM for the situation and deriving test cases and checking if the specific evasion flow could be identified from the list of test cases drawn for the situation.

The example used in the paper is a SQL injection attack on a healthcare website
--> one of the module lists the SSN of all the family members based on the gender
If the query is genuine, then the web address looks like
http://www.superhealth.com/show_members.asp?gender=m

and the query looks like
selet SSN, Name from PATIENTS where FAMILY = XXX and gender='m'

This indicates that the query is done at particular family level and that the person trying to retrieve the information should have permission to perform the query. So based on this initial understanding we can try to come up with rule sets that the IDS system should be have been built upon to prevent a basic attack. Lets not worry about the evasion or any problem associated there-off.

r1 --> user login attempt with username
r2 --> user login attempt with password
r3 --> guest user / no login required / can access the website login page ... and browse through for information on the products that are available etc ... and plan for registration
r4 --> registered user logs in successful/ login attempt succeeded
r5 --> registered user login attempt failed
r6 --> registered user query database for SSN information for their family/ no alarm raised
r7 --> guest user query database for SSN information for their family / alarm should be raised
r8 --> server decodes the query to assess the quality and genuineness of it/ responds back

some tips on IPS testing from CISCO
http://www.cisco.com/web/about/security/intelligence/cwilliams-ips.html

Thoughts

The aim is to figure out the signature evasions in NIDS systems. Currently how does it work ?

There are rule sets (or could be considered as policy) defined which mimic the real series of event execution, based on which the NIDS identifies if the system is working fine of there are possible intrusions.

Now the existence of the NIDS system or any IDS is known to all. So intruders also come up with a counter attack so as to successful in their venture. The fact is however the IDS is designed to figure out the vulnerable , the rule sets cannot be perfect, there are possible loop holes and the intruders capitalize on these loop holes.

Research has been done to understand the possible signature evasion in the case of an NIDS system with deterministic FSM in place. Test data generator by name AGENT has been implemented using prologue to test the NIDS and identify the possible evasions. Now this test data is based on the rule set and it targets both black and white hat problems.

What we want to do is try to generate test data from the deterministic FSM defined based on the rule set. Research has also been done to identify algorithms which will fix the evasions by considering the FSM of the rule set defined.

Revisiting what we want to do
1. We want to figure out the evasions in the rule sets for the IDS. Rule sets direct the IDS with respect to their performance. If we can identify the evasion in the rule set, we can directly related that to the performance of the NIDS
2. So the rule set can be defined as FSM. We can identify possible evasions. Generate test data depicting those evasions and then use the test data to test the IDS systems. If the system can detect the intrusion then the system is well defined. If not then this would be considered as an evasion to the system.

Characterization and Solution to A Stateful IDS Evasion

Issam Aib, Tung Tran, and Raouf Boutaba
University of Waterloo, Waterloo, ON, Canada
(iaib, t3tran, rboutaba)@uwaterloo.ca
2009 29th IEEE International Conference on Distributed Computing Systems

In this paper, the authors discuss about the concept of signature evasion, where-in the attackers on purpose try to tamper with the flow stored as a regular flow to complete the attack.

The authors explain this concept through an FTP session intrusion. The steps or states involved in the flow of a normal and admin user authentication are coded as rule set and. Here, the authors say that a normal user by following his states, can innately access a file which he/ she is not authorized to view. This is possible without raising any alarm.

Among the set of rules that are defined for a specific operation, if a fake network packet (in the example scenario) can trigger a signature, then the signature is considered to be evasive (i.e. something which can be deceived easily)

Thus in order to address such a scenario, they have come up with an algorithm based on deterministic finite automate, i.e. using the states of the flow to raise the false positive alarms in the right time.

The set of rules that are defined for a specific flow of operation could contain both an evasive rule and a target rule (which assists in identifying the attack).

Understanding about Finite State Automata machines

Finite State Automata are helpful in depicting the states and their relationship from start to end of a system. FSM representation could be done either using a state chart or regular expressions.

In the case of intrusion detection systems, if we go by the raw definition of FSM, it is feasible for us to implement a FSM based detection technique. Now let us assume that the detection technique is implemented using an FSM. Then will the technique not be misuse. Is it possible for it to be anomalous in nature? In the misuse scenario, we normally store all the possible intrusions that we have come across and then try to match with the incoming intrusions and raise an alarm. In the case of anomalous, we do the opposite thing, where-in the system is trained with the expected behavior. The expected series of behavior could be classified using the FSM.

Now the question is, if we use FSM, is there a possibility of reducing the false alarm rates?

Above the advantages of FSM usage, the motive here in front of us is to understand the possibility of generating test cases from the IDS FSM.

IDS systems using FSM technique

http://www.springerlink.com/content/g25w816354413354/fulltext.pdf

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4285750 (A Memory-Efficient Reconfigurable Aho-Corasick FSM Implementation for Intrusion Detection Systems)

http://opensiuc.lib.siu.edu/cgi/viewcontent.cgi?article=1009&context=ece_articles (Self-Addressable Memory-Based FSM: A Scalable Intrusion Detection Engine)

Generation and use of test data sets in IDS testing

Vidar Evenrud Seeberg, "Generation and use of test data sets in IDS testing", September 16, 2005

When evaluating an IDS, the evaluator can choose mainly between four approaches in
generating and using test data sets:

The evaluator can base the test on an empty test data set (no background trac)
 The evaluator can generate test data by recording real network trac
 The evaluator can generate test data by sanitizing recorded real network trac
 The evaluator can generate test data using simulated traffic

Quoted reference to arrive at the approaches-

P Mell, V Hu, R Lippmann, J Haines, and M Zissman. An overview of issues in testing
intrusion detection systems. Technical Report NIST IR 7007, National Institute of
Standards and Technology, August 2003.

Intrusion Detection Testing and Benchmarking Methodologies

Nicholas Athanasides, Randal Abler et al, "Intrusion Detection Testing and Benchmarking Methodologies", Georgia Institute of Technology, Proceedings of the First IEEE International workshop on Information assurance (IWIA '03)

The authors discuss the existing tools and testing methodologies for performing benchmark testing of intrusion detection systems. Based on their study they propose the use of an open source environment to execute the testing.

Environments discussed include:

1. DARPA Environment
2. LARIAT environment

In addition the authors have also listed the test suite and the tools that could be used

1. Nidsbench and IDS Wakeup
2. IDSwakeup
3. Flame Thrower
4. WebAvalanche/ WebReflector
5. Tcpreplay
6. Fragrouter
7. Hping2
8. Iperf

- Issues in generating realistic evaluation environments are also discussed

- Examples of IDS Evaluation environments include:

1. DARPA Like Environment
2. Custom Software
3. Advanced security audit trail analysis on Unix
4. Vendor Independent testing lab
5. Trade magazine evaluation

A methodology of testing intrusion detection systems

Nicholas J.Puketza, et al, "A methodology of testing intrusion detection systems", University of California

- Elaborate information is provided on the testing methodology to assess the performance of IDS

- the authors have quoted that test data could be obtained from sources such as CERT advisories, periodicals such as PHRACK and 2600, USENET, analyzing the vulnerabilities detected by security tools such as COPS and TIGER

Possible articles that could be referred to related to obtaining intrusion data:

1. M. Bishop, \A Taxonomy of UNIX System and Network Vulnerabilities," Technical
Report CSE-95-10, University of California at Davis, September 1995.
2. D. Farmer and W. Venema, \Improving the Security of Your Site by Breaking Into
It," USENET posting, December 1993.
3. D. Farmer and E. H. Spaord, \The COPS Security Checker System," Proc., Summer
USENIX Conference, pp. 165-170, June 1990.
4. S. Kumar and E. H. Spaord, \A Pattern Matching Model for Misuse Intrusion Detec-
tion," Proc., 17th National Computer Security Conference, Baltimore, MD, pp. 11-21,
October 1994.
5. D. R. Saord, D. L. Schales, and D. K. Hess, \The TAMU Security Package: An
Ongoing Response to Internet Intruders in an Academic Environment," Proc., Fourth
USENIX UNIX Security Symposium, Santa Clara, CA, pp. 91-118, October, 1993.

- test case selection based on 3 different strategies is elaborated

The authors have not provided in depth information on how large volume of test data could be generated, but they have given guidance related to the direction which needs to be adopted to achieve it

Generation of test data for testing intrusion detection systems

Internet has information related to the generation of test data for testing IDS systems in assessing its efficiency. detection techniques are implemented based on finite state machines and descriptive languages for the systems in place. But there is no sufficient data available on how these systems are tested.

Following listing consists of the papers available in this area along with the amount of information it contains pertaining to test data generation and usage of test data generation tools if any.

- Nicholas J.Puketza, et al, "A methodology of testing intrusion detection systems", University of california

- Nicholas Athanasides, Randal Abler et al, "Intrusion Detection Testing and Benchmarking Methodologies", Georgia Institute of Technology, Proceedings of the First IEEE International workshop on Information assurance (IWIA '03)

- Ming Li and Wei Zhao, "A principle of a data synthesizer for Performance test of anti-DDOS Flood Attacks", International journal of computers, issue 3, volume 1, 2007

- Vidar Evenrud Seeberg, "Generation and use of test data sets in IDS testing", September 16, 2005

- John McHugh, "Testing Intrusion Detection Systems: A Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory"

- Darren Mutz et al, "An Experience Developing an IDS Simulator for the Black Box testing of Network Intrusion Detection Systems", Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC 2003)

- Roy A. Maxion et al, "Benchmarking Anomaly-based detection systems", 2000 IEEE

- Emilie Lundin Barse et al, "Synthesizing Test Data for Fraud Detection Systems"

- R Sekar, A Gupta et al, "Specification based anomaly detection : A new approach for detecting network intrusions", CCS'02 November 18-22, 2002, Washington DC, USA, ACM

Tools for testing IDS

1. RACOON - RACOON: Rapidly Generating User Command Data For Anomaly Detection From Customizable Templates (http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01377229)
2. fTester - (http://dev.inversepath.com/trac/ftester) Firewall and IDS Testing tool
3. Tcpreplay - network traffic testing (http://tcpreplay.synfin.net/trac/)
4. Nemesis - well suited for testing Network Intrusion Detection Systems, firewalls, IP stacks and a variety of other tasks (http://nemesis.sourceforge.net/)
5. IDSwakeup - IDSwakeup is to generate false attack that mimic well known ones, in order to see if NIDS detects them and generates false positives
6. Unix tool EXPECT - Simulation of “normal” and “intruder” behaviour.
Extends TCL interpreter to provide simulation scripts. (http://expect.nist.gov/)
7. Fragrouter: Routes network traffic such that it elude most NIDS. (http://www.securityfocus.com/tools/176)

Issues involved in anomaly based detection technique

1. Inadvertently including malicious activity as part of a profile is a common problem with anomaly-based IDPS (Intrusion Detection and Prevention systems) products
2. It can be very challenging to build profiles in some cases to make them accurate, because computing activity can be so complex.
3. It is often difficult for analysts to determine why a particular alert was generated and to validate that an alert is accurate and not a false positive, because of the complexity of events and number of events that may have caused the alert to be generated.

Source - http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

Evaluating Network Intrusion Detection Signatures

http://www.securityfocus.com/infocus/1623

Evaluating Network Intrusion Detection Signatures, Part One

http://www.securityfocus.com/infocus/1630

Evaluating Network Intrusion Detection Signatures, Part Two

IDS measurements which can be used to assess performance accuracy

source - TESTING INTRUSION
DETECTION SYSTEMS
Elizabeth B. Lennon, Editor
Information Technology Laboratory
National Institute of Standards and Technology

1. Coverage
2. Probability of False Alarms
3. Probability of Detection
4. Resistance to attacks directed at the IDS
5. Ability to Handle High Bandwidth Traffic
6. Ability to Correlate Events
7. Ability to Detect Never-Before-Seen Attacks
8. Ability to Identify an Attack
9. Ability to Determine Attack Success
10. Capacity Verification for NIDS

USim: A User Behavior Simulation Framework for Training and Testing IDSes in GUI Based Systems

by A. Garg, S.Vidyaraman, S. Upadhyaya et al,

Currently reading

Information to be looked at

Firewall and IDS Testing toolhttp://www.secguru.com/link/firewall_and_ids_testing_tool

IDS Testing info from NIST website
http://www.itl.nist.gov/lab/bulletns/

http://csrc.nist.gov/publications/nistbul/bulletin07-03.pdf
http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf
http://csrc.nist.gov/publications/nistbul/04-2004.pdf
http://csrc.nist.gov/publications/nistir/nistir-7007.pdf

An Experience Developing an IDS Stimulator for the Black-Box Testing of Network Intrusion Detection Systems (2003)

Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation (2000)

Testing Network Intrusion Detection Systems (2006) (http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.129.9810)

Data Collection Mechanisms for Intrusion Detection Systems (2000)

Intrusion Detection Systems - Technologies, . . . (2003)

Automated Audit Trail Analysis and Intrusion Detection: A Survey (1988)