Tzi-cker Chiueh, Program Semantics Aware Intrusion Detection

Published: May 28, 2006, 12:34 a.m.

One of the most dangerous cybersecurity threats is ``control hijacking"" attacks, which hijack the control of a victim application, and execute arbitrary system calls assuming the identity of the victim program's effective user.

These types of attacks are viperous because they do not require any special set-up and because production-mode programs with such vulnerabilities appear to be wide spread. System call monitoring has been touted as an effective defense against control hijacking attacks because it could prevent remote attackers from inflicting damage upon a victim system even if they can successfully compromise certain applications running on the system.

However, the Achilles" heel of the system call monitoring approach is the construction of accurate system call behavior model that minimizes false positives and negatives. This presentation describes the design, implementation, and evaluation of a Program semantics-Aware Intrusion Detection system called PAID, which automatically derives an application-specific system call behavior model from the application's source code, and checks the application's run-time system call pattern against this model to thwart any control hijacking attacks. The per-application behavior model is in the form of the sites and ordering of system calls made in the application, as well as its partial control flow.

Experiments on a fully working PAID prototype show that PAID can indeed stop attacks that exploit non-standard security holes, such as format string attacks that modify function pointers, and that the run-time latency and throughput penalty of PAID are under 11.66% and 10.44%, respectively, for a set of production-mode network server applications including Apache, Sendmail, Ftp daemon, etc.

Dr. Tzi-cker Chiueh is currently an Associate Professor in Computer Science Department of Stony Brook University, and the Chief Scientist of Rether Networks Inc. He received his B.S. in Electrical Engineering from National Taiwan University, M.S. in Computer Science from Stanford University, and Ph.D. in Computer Science from University of California at Berkeley in 1984, 1988, and 1992, respectively. He received an NSF CAREER award in 1995. Dr. Chiueh's research interest is on computer security, network/storage QoS, and wireless networking. Dr. Chiueh's group developed the world's fastest array bound checking compiler that incurs less than 10% run-time overhead than programs without checking under Gcc, and built the world's fastest disk-based logging system, which accomplishes a single-sector disk write operation within 450 micro-seconds.