Synthesizing Test Data for Fraud Detection Systems

Emilie Lundin Barse, Håkan Kvarnström, Erland Jonsson: Synthesizing Test Data for Fraud Detection Systems. ACSAC 2003: 384-395

Day 2 Research

Following the paper on Intrusion Detection System testing, the authors came up with a following paper:

"A Software Platform for Testing Intrusion Detection Systems"

(Its a post script hence I have provided a link to its conversion)

In Prof. B. Mukherjee's web page, I could get details on related work, but nothing beyond 1997 on the testing of Intrusion Detection Systems. I do not have access to the following 3 papers which I got from this page. (http://networks.cs.ucdavis.edu/~mukherje/bio/pubsa.html)

G. Dias, K. Levitt, and B. Mukherjee,
``Modeling Attacks on Computer Systems: Evaluating Vulnerabilities and Forming a Basis for Intrusion Detection,''
Proc., CERT Workshop, Pleasanton, CA, June 1990.

N. Puketza, B. Mukherjee, R. A. Olsson, K. Zhang,
``Testing Intrusion Detection Systems: Design Methodologies and Results from an Early Prototype,''
Proc., National Computer Security Conference (NCSC), Baltimore, MD, pp. 1-10, Oct. 1994.

M. Chung, K. Zhang, N. Puketza, R. A. Olsson, and B. Mukherjee,
``Simulating Concurrent Intrusions for Testing Intrusion Detection Systems--Parallelizing Intrusions,''
Proc., 18th National Information Systems Security Conference, Baltimore, MD, pp. 173-183, Oct. 1995.

Following 2 papers are beyond the references obtained from the web page of the authors of "A Methodology of testing Intrusion Detection Systems"

An Overview of Issues in Testing Intrusion Detection Systems
by Peter Mell , Vincent Hu , Richard Lippmann , Josh Haines , Marc Zissman

Synthesizing Test Data for Fraud Detection Systems (2003)
by Emilie Lundin Barse , Emilie Lundin , Kvarnström Erl

IEEE Paper 1

A Methodology of Testing Intrusion Detection Systems:

Introduction:
Software Testing concepts have been used as a basis for performing testing of the Intrusion detection systems.
Testing goals concentrated are the performance measures of the intrusion detection systems. The test case selection and procedures have been filtered to satisfy this goal.

Issues in evaluating an IDS
1. It might be difficult to identify all possible intrusions in a website where the IDS could be employed so as to arrive at the parameters for testing
2. Depending on the usage of the system, the IDS may at times miss out on identifying and tracking specific attacks

Types of Intrustions in focus:

1. Single Intruder Single Terminal (SIST)
2. Single Intruder Multiple Terminal (SIMT)
3. Multiple Intruder Multiple Terminal (MIMT)

Performance Objectives:

Broad Detection Range
Economy in Resource Usage
Resilience to Stress

Proposed Methodology:
Intrusion Identification tests
Resource Usage tests
Stress tests - Smoke screen noise, background noise, High Volume Sessions, Intensity and Load

Future Work:
1. Careful development of a suite of Intrusion test cases for basic detection system
2. Identification of additional performance objectives based on the information obtained from testing of other systems
3. Another task is to fine tune the testing procedures and develop suitable metrics to create a benchmark suite of IDSS, similar in spirit to the well established benchmarks such as SPECmarks, Livermore Loops and Dhrystone , which are used to test the performance of various computer architectures.
4. Testing techniques arrived at in this paper could be looked to adopt to testing other systems as well