Anomaly Detection
-Misuse
-System Calls
-Adaptive
-Feature Selection
-Network Based
-Host Based
-Behavior Based
-Cooperative
-Cost Sensitive
The authors as quoted in my previous blog have collated relevant papers related to the classification mentioned above. This could act as a good starting point to understand more about the different anomaly based systems
Tuesday, December 15, 2009
Intrusion detection systems and models
Joseph S. Sherif, Tommy G. Dearmond, California Institute of Technology
Mahoney, M., “Computer Security: A Survey of Attacks and Defenses” Hyperlink docshow.net/ids.htm, 2000.
http://cs.fit.edu/~mmahoney/ids.html
According to Mohoney, there exists at least six types of attacks to the system
1. Worms - self replicating programs that spread across a network.
The three security flaws that the worms exploited were
a. a backdoor in the sendmail server program that allowed clients to execute remote commands on the server
b. a buffer overflow vulnerability in the fingerd server
c. weak passwords
2. Viruses - programs that replicate when a user performs some action such as running a program.
types discussed in this paper include the following
a. Boot sector infectors
b. File infectors
c. Email viruses
3. Server attacks - a client exploits a bug in the server to cause it to perform some unintended action.
Examples,
a. Microsoft IIS 4.0 web server
b. Cold Fusion web server
c. lpd attack
d. Sendmail buffer overflow
e. rpc.statd buffer overflow
4. Client attacks - a server exploits a bug in a client to cause it to perform some unintended action.
a. Microsoft Office 2000 UA Control
b. Microsoft Outlook
5. Network attacks (denial of service) - a remote attacker exploits a bug in the network software or weakness in the protocol to cause a server, router, or network to fail.
a. Winnuke
b. IRDP spoofing
c. Teardrop
d. Land
e. Ping of death
f. Smurf
g. Vengeance
h. SYN flood
i. Local network attacks
j. SNMP attacks
6. Root attacks - a user on a multiuser operating system obtains the priveliges of another user (usually root) by either
- Obtaining the other user's password, or
- Bypassing controls that restrict access.
NT attacks - NT password weaknesses
UNIX attacks - Shell script attacks, Dynamic library attacks, Directory tree escapes, Symbolic links in /tmp, Console attacks
Password Capture - Sniffer attacks, Weak encryption, Trojan attacks, Backdoor passwords, Stored passwords, Hardware key attacks
Mahoney, M., “Computer Security: A Survey of Attacks and Defenses” Hyperlink docshow.net/ids.htm, 2000.
http://cs.fit.edu/~mmahoney/ids.html
According to Mohoney, there exists at least six types of attacks to the system
1. Worms - self replicating programs that spread across a network.
The three security flaws that the worms exploited were
a. a backdoor in the sendmail server program that allowed clients to execute remote commands on the server
b. a buffer overflow vulnerability in the fingerd server
c. weak passwords
2. Viruses - programs that replicate when a user performs some action such as running a program.
types discussed in this paper include the following
a. Boot sector infectors
b. File infectors
c. Email viruses
3. Server attacks - a client exploits a bug in the server to cause it to perform some unintended action.
Examples,
a. Microsoft IIS 4.0 web server
b. Cold Fusion web server
c. lpd attack
d. Sendmail buffer overflow
e. rpc.statd buffer overflow
4. Client attacks - a server exploits a bug in a client to cause it to perform some unintended action.
a. Microsoft Office 2000 UA Control
b. Microsoft Outlook
5. Network attacks (denial of service) - a remote attacker exploits a bug in the network software or weakness in the protocol to cause a server, router, or network to fail.
a. Winnuke
b. IRDP spoofing
c. Teardrop
d. Land
e. Ping of death
f. Smurf
g. Vengeance
h. SYN flood
i. Local network attacks
j. SNMP attacks
6. Root attacks - a user on a multiuser operating system obtains the priveliges of another user (usually root) by either
- Obtaining the other user's password, or
- Bypassing controls that restrict access.
NT attacks - NT password weaknesses
UNIX attacks - Shell script attacks, Dynamic library attacks, Directory tree escapes, Symbolic links in /tmp, Console attacks
Password Capture - Sniffer attacks, Weak encryption, Trojan attacks, Backdoor passwords, Stored passwords, Hardware key attacks
Monday, December 14, 2009
Techniques to detect anomalies (IDS)
Susheela Sarasamma, Julie Huff, "Anomaly based techniques for intrusion detection systems", Northrop Grumman Mission Systems
In this presentation, the authors provide information on the various anomaly based techniques which can be used for the IDS.
General anomaly based techniques include, statistical, neural network and machine learning.
The authors have specifically discussed a couple of anomaly detection techniques for an IDS which include the following
1. A novel anomaly detection technique using Kohonen network
2. Conclusions on the test conducted on multi-level k map
Kinds of anomalies to be considered as outliers include the following
1. outlier detection
2. novelty detection
3. noice detection
4. deviation detection
5. exception mining
* Now suppose we have to come up with efficient testing techniques for anomaly based intrusion detection systems there are set of things which we might want to understand
- study the different anomaly based intrusion detection systems. As the test methodology might vary depending on the technique which has been adopted.
- research on the platform which might be required for performing the testing for such a system.
- in addition we might have to come to a decision or rather the focus of the research
- we might also want to understand the typical attacks in the case of the anomaly based intrusions
- audit trails how are they being used for testing purposes
- if we need to compose of a testing methodology, then how should the evaluation be done for it
In this presentation, the authors provide information on the various anomaly based techniques which can be used for the IDS.
General anomaly based techniques include, statistical, neural network and machine learning.
The authors have specifically discussed a couple of anomaly detection techniques for an IDS which include the following
1. A novel anomaly detection technique using Kohonen network
2. Conclusions on the test conducted on multi-level k map
Kinds of anomalies to be considered as outliers include the following
1. outlier detection
2. novelty detection
3. noice detection
4. deviation detection
5. exception mining
* Now suppose we have to come up with efficient testing techniques for anomaly based intrusion detection systems there are set of things which we might want to understand
- study the different anomaly based intrusion detection systems. As the test methodology might vary depending on the technique which has been adopted.
- research on the platform which might be required for performing the testing for such a system.
- in addition we might have to come to a decision or rather the focus of the research
- we might also want to understand the typical attacks in the case of the anomaly based intrusions
- audit trails how are they being used for testing purposes
- if we need to compose of a testing methodology, then how should the evaluation be done for it
Friday, December 11, 2009
Road Map
In the last quarter report, there were two potential streams identify for research
1. Testing of anomaly based intrusion detection systems
2. Modeling intrusion detection systems
Proceeding ahead with the testing of anomaly based IDS, we intended to do the following
1. Understand the anomaly based IDS models
2. How currently the IDS systems are tested. i.e. testing methodology, test scripts generation etc
3. How audit records are used to understand any intrusions in the system etc.
Currently, based on the information obtained. There are not many research papers supporting the testing of IDS systems.
When we are not getting sufficient information, the thought on what could have led to scenario is a question that comes up. In addition most of the testing related research have been carried out a few years ago, implying there is no currently existing research based on the literature review that has been obtained.
Given the situation what could be the steps ahead.
1. Irrespective of the availability of specific information, understanding the basic model of anomaly based IDS would help us in deriving better testing techniques.
2. Improvement of the test scripts could be an option which need to be worked on.
3. In addition, there was also some work related to the software platform for testing IDS undertaken by Prof. Biswanath Mukherjee and his associates.
4. The model itself could be looked at. One fact that should be considered is that anomaly based IDS does have the ability to raise an alarm for all possible situations which it was not trained before, hence there is greater probability of false alarms as well. Hence we could spend some effort here to understand how this could be avoided.
Though we have signature and anomaly based IDS. Current industry preference is signature since it is trained to identify fixed anomalies and it does most of the time. While in the case of anomaly based IDS, it is dynamic, but there is a probability if the system is trained to accept an intrusion as a normal behavior it cannot differentiate.
1. Testing of anomaly based intrusion detection systems
2. Modeling intrusion detection systems
Proceeding ahead with the testing of anomaly based IDS, we intended to do the following
1. Understand the anomaly based IDS models
2. How currently the IDS systems are tested. i.e. testing methodology, test scripts generation etc
3. How audit records are used to understand any intrusions in the system etc.
Currently, based on the information obtained. There are not many research papers supporting the testing of IDS systems.
When we are not getting sufficient information, the thought on what could have led to scenario is a question that comes up. In addition most of the testing related research have been carried out a few years ago, implying there is no currently existing research based on the literature review that has been obtained.
Given the situation what could be the steps ahead.
1. Irrespective of the availability of specific information, understanding the basic model of anomaly based IDS would help us in deriving better testing techniques.
2. Improvement of the test scripts could be an option which need to be worked on.
3. In addition, there was also some work related to the software platform for testing IDS undertaken by Prof. Biswanath Mukherjee and his associates.
4. The model itself could be looked at. One fact that should be considered is that anomaly based IDS does have the ability to raise an alarm for all possible situations which it was not trained before, hence there is greater probability of false alarms as well. Hence we could spend some effort here to understand how this could be avoided.
Though we have signature and anomaly based IDS. Current industry preference is signature since it is trained to identify fixed anomalies and it does most of the time. While in the case of anomaly based IDS, it is dynamic, but there is a probability if the system is trained to accept an intrusion as a normal behavior it cannot differentiate.
Wednesday, December 9, 2009
General definition of anomaly based intrusion detection
An Anomaly-Based Intrusion Detection System, is a system for detecting computer intrusions and misuse by monitoring system activity and classifying it as either normal or anomalous. The classification is based on heuristics or rules, rather than patterns or signatures, and will detect any type of misuse that falls out of normal system operation. This is as opposed to signature based systems which can only detect attacks for which a signature has previously been created.
In order to determine what is attack traffic, the system must be taught to recognise normal system activity. This can be accomplished in several ways, most often with artificial intelligence type techniques. Systems using neural networks have been used to great effect. Another method is to define what normal usage of the system comprises using a strict mathematical model, and flag any deviation from this as an attack. This is known as strict anomaly detection.
Source : - http://en.wikipedia.org/wiki/Anomaly-based_intrusion_detection_system
In order to determine what is attack traffic, the system must be taught to recognise normal system activity. This can be accomplished in several ways, most often with artificial intelligence type techniques. Systems using neural networks have been used to great effect. Another method is to define what normal usage of the system comprises using a strict mathematical model, and flag any deviation from this as an attack. This is known as strict anomaly detection.
Source : - http://en.wikipedia.org/wiki/Anomaly-based_intrusion_detection_system
IDS Testing Methodology
Nicolas J Puketza et al, "A methodology for testing intrusion detection systems"
In this paper, the authors try to test the performance objectives of the IDS systems
Precondition - IDS systems should be installed and configured appropriately.
They have considered the basic testing procedure as described below and brought in variations in it
1. create or select a set of test scripts
2. establish the desired conditions in the computing environment
3. start the IDS
4. run the test scripts
5. analyze the IDS's output
The different IDS Tests considered include the following
1. Intrusion Identification tests
- Basic detection tests
a. create a set of intrusion scripts
b. as much as possible, eliminate unrelated computing acitvity in testing environment
c. start the IDS
d. run the intrusion scripts
Another technique, test results include a number associting it to warning based on the level of abnormal behavior. This would assist the testers in better classification of the abnormal behavior.
- Normal Users test - Here instead of the intruder scripts, normal user scripts are introduced resulting in the understanding on how often normal scenarios are flagged as intrusions.
2. Resource Usage tests
This is done to understand the resource consumption of the IDS systems. The main test is the disk space. The procedure adopted is as follows:
1. eliminate unrelated activity in the test environment
2. start the IDS
3. run the test script for a measured period of time (e.g., one hour) and
4. calculate the total disk space used by the IDS to record the session associated with the script.
Evaluation:
In this paper, the authors try to test the performance objectives of the IDS systems
Precondition - IDS systems should be installed and configured appropriately.
They have considered the basic testing procedure as described below and brought in variations in it
1. create or select a set of test scripts
2. establish the desired conditions in the computing environment
3. start the IDS
4. run the test scripts
5. analyze the IDS's output
The different IDS Tests considered include the following
1. Intrusion Identification tests
- Basic detection tests
a. create a set of intrusion scripts
b. as much as possible, eliminate unrelated computing acitvity in testing environment
c. start the IDS
d. run the intrusion scripts
Another technique, test results include a number associting it to warning based on the level of abnormal behavior. This would assist the testers in better classification of the abnormal behavior.
- Normal Users test - Here instead of the intruder scripts, normal user scripts are introduced resulting in the understanding on how often normal scenarios are flagged as intrusions.
2. Resource Usage tests
This is done to understand the resource consumption of the IDS systems. The main test is the disk space. The procedure adopted is as follows:
1. eliminate unrelated activity in the test environment
2. start the IDS
3. run the test script for a measured period of time (e.g., one hour) and
4. calculate the total disk space used by the IDS to record the session associated with the script.
Evaluation:
Friday, December 4, 2009
An intrusion detection model
Denning, D. E. 1987. An intrusion detection model. IEEE Transactions of Software Engineer-
ing 13, 2, 222 - 232.
The model denning arrived at was for a generic purpose which could cater to any kind of environment and system vulnerability and the type of intrusion. He believed that most of the intrusions could be identified from the audit records that is created by the intrusion detection system. He proposed a model in lines of the IDES and claimed it to work well.
The model proposed by him was expected to cater to the following list of intrusions
1. attempted break-in
2. masquerading or successful break-in
3. penetration by legitimate user
4. leakage by legitimate user
5. Inference by legitimate user
6. Trojan horse
7. Virus
8. Denial of service
Model proposed by him:
Since his model was a general purpose one, he called it as an intrusion-detection expert system (IDES). The six main components of his model includes the following:
subjects
objects
audit records
profiles
anomaly records
activity rules
Denning also describes various metrics and statistical models using the audit records could be analyzed for detecting anomalies
The metrics defined are:
1. event counter - number of audit records satisfying some property occurring during a period
2. interval timer - length of time between two related events
3. resource measure - quantity of resources consumed by some action during a period as specified in the Resource-Usage field of the audit records.
Statistical models proposed include the following:
1. Operational model
2. Mean and standard deviation model
3. Multivariate model
4. Markov Process model
5. Time series model
In this paper, in addition to defining metrics and models, Denning also elaborates on the various profiles of data that could be generated for different sessions.
Thus this data could be used for designing intrusion detection system test cases
ing 13, 2, 222 - 232.
The model denning arrived at was for a generic purpose which could cater to any kind of environment and system vulnerability and the type of intrusion. He believed that most of the intrusions could be identified from the audit records that is created by the intrusion detection system. He proposed a model in lines of the IDES and claimed it to work well.
The model proposed by him was expected to cater to the following list of intrusions
1. attempted break-in
2. masquerading or successful break-in
3. penetration by legitimate user
4. leakage by legitimate user
5. Inference by legitimate user
6. Trojan horse
7. Virus
8. Denial of service
Model proposed by him:
Since his model was a general purpose one, he called it as an intrusion-detection expert system (IDES). The six main components of his model includes the following:
subjects
objects
audit records
profiles
anomaly records
activity rules
Denning also describes various metrics and statistical models using the audit records could be analyzed for detecting anomalies
The metrics defined are:
1. event counter - number of audit records satisfying some property occurring during a period
2. interval timer - length of time between two related events
3. resource measure - quantity of resources consumed by some action during a period as specified in the Resource-Usage field of the audit records.
Statistical models proposed include the following:
1. Operational model
2. Mean and standard deviation model
3. Multivariate model
4. Markov Process model
5. Time series model
In this paper, in addition to defining metrics and models, Denning also elaborates on the various profiles of data that could be generated for different sessions.
Thus this data could be used for designing intrusion detection system test cases
Follow - up survey - focus
The authors from the University of Minnesotta as explained in my previous blog had conducted an extensive study on the anomaly detection in different domains. The domain of interest to us are the intrusion detection systems, where-in they try to focus on the intrusions based on anomaly's in computer systems.
According to the authors the complexity with respect to anomaly detection in computer intrusion systems is the volume of data. In general labeled data corresponding to the expected behavior is available while those beyond expected are large in number and difficult to collate. In addition we understand that behavior beyond the normal and expected are considered to be an anomaly.
Denning in his paper classifies the types of intrusion detection systems to be host and network based.
(Denning, D. E. 1987. An intrusion detection model. IEEE Transactions of Software Engineer-
ing 13, 2, 222{232.)
According to the authors the complexity with respect to anomaly detection in computer intrusion systems is the volume of data. In general labeled data corresponding to the expected behavior is available while those beyond expected are large in number and difficult to collate. In addition we understand that behavior beyond the normal and expected are considered to be an anomaly.
Denning in his paper classifies the types of intrusion detection systems to be host and network based.
(Denning, D. E. 1987. An intrusion detection model. IEEE Transactions of Software Engineer-
ing 13, 2, 222{232.)
Anamoly detection
This is based on the technical report consisting of a survey conducted by Varun Chandola, Arindam Banerjee and Vipin Kumar from University of Minnessota.
They talk in detail on what anamolies are and how they could be identified by using different techniques.
They discuss about anomaly detection in terms of intrusions from the following perspective
1. Host based intrusion detection systems
2. Network intrusion detection systems
Apart from the intrusion detection systems, they also delve into fraud detections like credit card, mobile, insurance claim frauds and insider trading detections.
The different techniques that they discuss on to detect anomaly include the following
1. Neural network based
2. Bayesian network based
3. Support vector machine based
4. rule based
The authors also talk about the additional anomaly detection techniques from a mathematical perspective as well.
Among the research objectives that we have in place, one of them is to identify the testing technique that could be adopted to detect anomaly based intrusions.
Thus the technical report defined by the authors could act as an input to the way test data is being prepared for identifying intrusions. Effort needs to be added in order to achieve this. At the same time, this does sound to be a source to be taken into consideration.
The authors process of writing down the advantages and disadvantages of each technique they describe which could be used for anomaly detection could also add into the understanding.
One drawback which the authors quote is that, they believe the data considered for describing the various techniques in the report is not standard. It seems to be varying across. So the results that they have achieved might not give us a consistent understanding. Hence one of their future work would be to unify all the assumptions that they have considered.
They talk in detail on what anamolies are and how they could be identified by using different techniques.
They discuss about anomaly detection in terms of intrusions from the following perspective
1. Host based intrusion detection systems
2. Network intrusion detection systems
Apart from the intrusion detection systems, they also delve into fraud detections like credit card, mobile, insurance claim frauds and insider trading detections.
The different techniques that they discuss on to detect anomaly include the following
1. Neural network based
2. Bayesian network based
3. Support vector machine based
4. rule based
The authors also talk about the additional anomaly detection techniques from a mathematical perspective as well.
Among the research objectives that we have in place, one of them is to identify the testing technique that could be adopted to detect anomaly based intrusions.
Thus the technical report defined by the authors could act as an input to the way test data is being prepared for identifying intrusions. Effort needs to be added in order to achieve this. At the same time, this does sound to be a source to be taken into consideration.
The authors process of writing down the advantages and disadvantages of each technique they describe which could be used for anomaly detection could also add into the understanding.
One drawback which the authors quote is that, they believe the data considered for describing the various techniques in the report is not standard. It seems to be varying across. So the results that they have achieved might not give us a consistent understanding. Hence one of their future work would be to unify all the assumptions that they have considered.
Friday, November 6, 2009
New tool for detecting intrusions in the web
Authors Sean McAllister, Engin Kirda and Christopher Kruegel in their paper "Leveraging User Interactions for In-Depth Testing of Web Applications" talk about a new scanner (automated testing tool) which they have put in place for detecting web application vulnerabilities as the reflected and stored cross site scripting.
Their claim is that the new tool deleveloped would be able to identify more defects comparatively. This is due to the fact that more number of test cases could be generated leading to more test scenario coverage for the web applications.
In the process of their evaluation, they have tested three applications with 3 other existing scanners along with their new developed one. Burp Spider, w3af spider and Acunetix Web Vulnerability are the others. They had concentrated on blogging applications, Forum application and online shopping application.
The authors feel that black box testing might not be very efficient as it lacks the capability to enter to all the requisite pages to perform the testing. While the tool they have developed has the feature to store and use the values keyed in so as to enter into the pages of the application and ensure there is no possible vulnerability injected in.
In order to ensure that they application does a deeper testing, the techniques adopted by them include guided fuzzing and stateful fuzzing.
In the case of guided fuzzing, it stores the previous valid values entered by the user and tries to reevaluate the page and compares the existence of any possible intrusion. They also use the extended fuzzing concept where-in they try to extensively use the data and dig even deeper into the application. Next is the stateful fuzzing which was built to ensure that the use cases are not repeated often as a result of the requirement of guided fuzzing in performing the testing of the applications. They call this as the undesirable side-effects.
They seem to be happy with what they have produced and talk about no extention to their current work.
Their claim is that the new tool deleveloped would be able to identify more defects comparatively. This is due to the fact that more number of test cases could be generated leading to more test scenario coverage for the web applications.
In the process of their evaluation, they have tested three applications with 3 other existing scanners along with their new developed one. Burp Spider, w3af spider and Acunetix Web Vulnerability are the others. They had concentrated on blogging applications, Forum application and online shopping application.
The authors feel that black box testing might not be very efficient as it lacks the capability to enter to all the requisite pages to perform the testing. While the tool they have developed has the feature to store and use the values keyed in so as to enter into the pages of the application and ensure there is no possible vulnerability injected in.
In order to ensure that they application does a deeper testing, the techniques adopted by them include guided fuzzing and stateful fuzzing.
In the case of guided fuzzing, it stores the previous valid values entered by the user and tries to reevaluate the page and compares the existence of any possible intrusion. They also use the extended fuzzing concept where-in they try to extensively use the data and dig even deeper into the application. Next is the stateful fuzzing which was built to ensure that the use cases are not repeated often as a result of the requirement of guided fuzzing in performing the testing of the applications. They call this as the undesirable side-effects.
They seem to be happy with what they have produced and talk about no extention to their current work.
Swarm attacks
swarm attacks against networks are in trend these days, where-in the aim of the attacker is not to win his bid immediately but to win in increments there-by resulting in the performance degradation of the IDS systems.
Though the IDS systems are being tested for misuse patterns of the system, such attacks go beyond the scanner. One reason could be a repeated performance without violation could train the IDS system to accept that as a basic behavior or the network its protecting.
Hence the authors Simon P Chung and Aloysius K. Mok in their paper "Swarm Attacks against Network-level Emulation/ Analysis" (11th international symposium, RAID 2008 Cambridge, MA, USA, September 2008 Proceedings) try to analyze the possibility of breaking through the network by attacking the system incrementally. Their work is to understand if the IDS in place will defer to detect such an attack. They have implemented an attack similar to this for 8 different systems to see if the behavior will differ.
Now the question that arises is, we have different testing methodologies that concentrate on misuse and anamoly based intrusion detection. If the swarm attack is to fall under the category of repeating the same attack more number of times but performing limited violations, it is not a misuse patter of intrusion detection.
How can the IDS be trained such that it is able to understand such swarm attacks.
Further the authors quote that due to time gap between two attacks of the swarm attack, there is higher probability of even the Intrusion prevention systems not identifying the trend thereby evading the identification of the attack that is existing.
Future work as quoted by the authors is that they want to extend their research extensively on both types of systems to understand what the real time behavior
RAID - Recent Advances in Intrusion Detection
Though the IDS systems are being tested for misuse patterns of the system, such attacks go beyond the scanner. One reason could be a repeated performance without violation could train the IDS system to accept that as a basic behavior or the network its protecting.
Hence the authors Simon P Chung and Aloysius K. Mok in their paper "Swarm Attacks against Network-level Emulation/ Analysis" (11th international symposium, RAID 2008 Cambridge, MA, USA, September 2008 Proceedings) try to analyze the possibility of breaking through the network by attacking the system incrementally. Their work is to understand if the IDS in place will defer to detect such an attack. They have implemented an attack similar to this for 8 different systems to see if the behavior will differ.
Now the question that arises is, we have different testing methodologies that concentrate on misuse and anamoly based intrusion detection. If the swarm attack is to fall under the category of repeating the same attack more number of times but performing limited violations, it is not a misuse patter of intrusion detection.
How can the IDS be trained such that it is able to understand such swarm attacks.
Further the authors quote that due to time gap between two attacks of the swarm attack, there is higher probability of even the Intrusion prevention systems not identifying the trend thereby evading the identification of the attack that is existing.
Future work as quoted by the authors is that they want to extend their research extensively on both types of systems to understand what the real time behavior
RAID - Recent Advances in Intrusion Detection
An overview of testing issues in IDS
The paper gives a list of possible measurements which could be obtained as a result of testing the IDS.
Centres that are currently spending efforts on testing the IDS include the following:
1. University of California at DAVIS
2. IBM Zurich
3. MIT Lincoln Laboratory
4. Air Force Research Laboratory
5. MITRE
6. Neohapsis/ Network-Computing
7. The NSS Group
8. Network World Fusion
List of challenges as quoted include the following
1. Difficulties in Collecting Attack Scripts and Victim Software
2. Differing Requirements for Testing Signature Based vs. Anomaly Based
IDSs
3. Differing Requirements for Testing Network Based vs. Host Based IDSs
4. Four Approaches to Using Background Traffic in IDS Tests
- Testing using no background traffic/logs
- Testing using real traffic/logs
- Testing using sanitized traffic/logs
- Testing by generating traffic on a testbed network
Recommendations
1. Shared Datasets
2. Attack Traces
3. Cleansing Real Data
4. Sensor and Detector Alert Datasets
5. Real-life performance metrics
6. New Technologies
Centres that are currently spending efforts on testing the IDS include the following:
1. University of California at DAVIS
2. IBM Zurich
3. MIT Lincoln Laboratory
4. Air Force Research Laboratory
5. MITRE
6. Neohapsis/ Network-Computing
7. The NSS Group
8. Network World Fusion
List of challenges as quoted include the following
1. Difficulties in Collecting Attack Scripts and Victim Software
2. Differing Requirements for Testing Signature Based vs. Anomaly Based
IDSs
3. Differing Requirements for Testing Network Based vs. Host Based IDSs
4. Four Approaches to Using Background Traffic in IDS Tests
- Testing using no background traffic/logs
- Testing using real traffic/logs
- Testing using sanitized traffic/logs
- Testing by generating traffic on a testbed network
Recommendations
1. Shared Datasets
2. Attack Traces
3. Cleansing Real Data
4. Sensor and Detector Alert Datasets
5. Real-life performance metrics
6. New Technologies
Tuesday, October 20, 2009
Synthesizing Test Data for Fraud Detection Systems
Emilie Lundin Barse, Håkan Kvarnström, Erland Jonsson: Synthesizing Test Data for Fraud Detection Systems. ACSAC 2003: 384-395
Tuesday, October 13, 2009
Day 2 Research
Following the paper on Intrusion Detection System testing, the authors came up with a following paper:
"A Software Platform for Testing Intrusion Detection Systems"
(Its a post script hence I have provided a link to its conversion)
In Prof. B. Mukherjee's web page, I could get details on related work, but nothing beyond 1997 on the testing of Intrusion Detection Systems. I do not have access to the following 3 papers which I got from this page. (http://networks.cs.ucdavis.edu/~mukherje/bio/pubsa.html)
G. Dias, K. Levitt, and B. Mukherjee,
``Modeling Attacks on Computer Systems: Evaluating Vulnerabilities and Forming a Basis for Intrusion Detection,''
Proc., CERT Workshop, Pleasanton, CA, June 1990.
N. Puketza, B. Mukherjee, R. A. Olsson, K. Zhang,
``Testing Intrusion Detection Systems: Design Methodologies and Results from an Early Prototype,''
Proc., National Computer Security Conference (NCSC), Baltimore, MD, pp. 1-10, Oct. 1994.
M. Chung, K. Zhang, N. Puketza, R. A. Olsson, and B. Mukherjee,
``Simulating Concurrent Intrusions for Testing Intrusion Detection Systems--Parallelizing Intrusions,''
Proc., 18th National Information Systems Security Conference, Baltimore, MD, pp. 173-183, Oct. 1995.
Following 2 papers are beyond the references obtained from the web page of the authors of "A Methodology of testing Intrusion Detection Systems"
An Overview of Issues in Testing Intrusion Detection Systems
by Peter Mell , Vincent Hu , Richard Lippmann , Josh Haines , Marc Zissman
Synthesizing Test Data for Fraud Detection Systems (2003)
by Emilie Lundin Barse , Emilie Lundin , Kvarnström Erl
"A Software Platform for Testing Intrusion Detection Systems"
(Its a post script hence I have provided a link to its conversion)
In Prof. B. Mukherjee's web page, I could get details on related work, but nothing beyond 1997 on the testing of Intrusion Detection Systems. I do not have access to the following 3 papers which I got from this page. (http://networks.cs.ucdavis.edu/~mukherje/bio/pubsa.html)
G. Dias, K. Levitt, and B. Mukherjee,
``Modeling Attacks on Computer Systems: Evaluating Vulnerabilities and Forming a Basis for Intrusion Detection,''
Proc., CERT Workshop, Pleasanton, CA, June 1990.
N. Puketza, B. Mukherjee, R. A. Olsson, K. Zhang,
``Testing Intrusion Detection Systems: Design Methodologies and Results from an Early Prototype,''
Proc., National Computer Security Conference (NCSC), Baltimore, MD, pp. 1-10, Oct. 1994.
M. Chung, K. Zhang, N. Puketza, R. A. Olsson, and B. Mukherjee,
``Simulating Concurrent Intrusions for Testing Intrusion Detection Systems--Parallelizing Intrusions,''
Proc., 18th National Information Systems Security Conference, Baltimore, MD, pp. 173-183, Oct. 1995.
Following 2 papers are beyond the references obtained from the web page of the authors of "A Methodology of testing Intrusion Detection Systems"
An Overview of Issues in Testing Intrusion Detection Systems
by Peter Mell , Vincent Hu , Richard Lippmann , Josh Haines , Marc Zissman
Synthesizing Test Data for Fraud Detection Systems (2003)
by Emilie Lundin Barse , Emilie Lundin , Kvarnström Erl
Saturday, October 10, 2009
IEEE Paper 1
A Methodology of Testing Intrusion Detection Systems:
Introduction:
Software Testing concepts have been used as a basis for performing testing of the Intrusion detection systems.
Testing goals concentrated are the performance measures of the intrusion detection systems. The test case selection and procedures have been filtered to satisfy this goal.
Issues in evaluating an IDS
1. It might be difficult to identify all possible intrusions in a website where the IDS could be employed so as to arrive at the parameters for testing
2. Depending on the usage of the system, the IDS may at times miss out on identifying and tracking specific attacks
Types of Intrustions in focus:
1. Single Intruder Single Terminal (SIST)
2. Single Intruder Multiple Terminal (SIMT)
3. Multiple Intruder Multiple Terminal (MIMT)
Performance Objectives:
Broad Detection Range
Economy in Resource Usage
Resilience to Stress
Proposed Methodology:
Intrusion Identification tests
Resource Usage tests
Stress tests - Smoke screen noise, background noise, High Volume Sessions, Intensity and Load
Future Work:
1. Careful development of a suite of Intrusion test cases for basic detection system
2. Identification of additional performance objectives based on the information obtained from testing of other systems
3. Another task is to fine tune the testing procedures and develop suitable metrics to create a benchmark suite of IDSS, similar in spirit to the well established benchmarks such as SPECmarks, Livermore Loops and Dhrystone , which are used to test the performance of various computer architectures.
4. Testing techniques arrived at in this paper could be looked to adopt to testing other systems as well
Introduction:
Software Testing concepts have been used as a basis for performing testing of the Intrusion detection systems.
Testing goals concentrated are the performance measures of the intrusion detection systems. The test case selection and procedures have been filtered to satisfy this goal.
Issues in evaluating an IDS
1. It might be difficult to identify all possible intrusions in a website where the IDS could be employed so as to arrive at the parameters for testing
2. Depending on the usage of the system, the IDS may at times miss out on identifying and tracking specific attacks
Types of Intrustions in focus:
1. Single Intruder Single Terminal (SIST)
2. Single Intruder Multiple Terminal (SIMT)
3. Multiple Intruder Multiple Terminal (MIMT)
Performance Objectives:
Broad Detection Range
Economy in Resource Usage
Resilience to Stress
Proposed Methodology:
Intrusion Identification tests
Resource Usage tests
Stress tests - Smoke screen noise, background noise, High Volume Sessions, Intensity and Load
Future Work:
1. Careful development of a suite of Intrusion test cases for basic detection system
2. Identification of additional performance objectives based on the information obtained from testing of other systems
3. Another task is to fine tune the testing procedures and develop suitable metrics to create a benchmark suite of IDSS, similar in spirit to the well established benchmarks such as SPECmarks, Livermore Loops and Dhrystone , which are used to test the performance of various computer architectures.
4. Testing techniques arrived at in this paper could be looked to adopt to testing other systems as well
Subscribe to:
Posts (Atom)
