0x1A Great Papers in Computer SecurityAfter All Else FailsWhat Should Be Detected?Intrusion Detection Systems (IDS)Intrusion Detection TechniquesMisuse Detection (Signature-Based)Extracting Misuse SignaturesAnomaly DetectionStatistical Anomaly Detection“Self-Immunology” ApproachLevel of MonitoringSystem Call InterpositionJanusPolicy DesignProblems and PitfallsIncorrectly Mirroring OS StateTOCTTOU in Syscall InterpositionR. Watson Exploiting Concurrency Vulnerabilities in System Call Wrappers (WOOT 2007)Exploiting TOCTTOU ConditionsTOCTTOU via a Page FaultTOCTTOU on SysjailMitigating TOCTTOUD. Wagner, D. Dean Intrusion Detection via Static Analysis (Oakland 2001)Interposition + Static AnalysisTrivial “Bag-O’Calls” ModelCallgraph ModelNFA ExampleAnother NFA ExampleNFA Permits Impossible PathsNFA: Modeling TradeoffsAbstract Stack ModelPDA ExampleAnother PDA ExamplePDA: Modeling TradeoffsDyck ModelExample of Dyck ModelCFG Extraction IssuesSystem Call Processing ComplexityDyck: Runtime OverheadsPersistent Interposition Attacksslide 10x1A Great Papers inComputer SecurityVitaly ShmatikovCS 380Shttp://www.cs.utexas.edu/~shmat/courses/cs380s/slide 2After All Else FailsIntrusion prevention•Find buffer overflows and remove them•Use firewall to filter out malicious network trafficIntrusion detection is what you do after prevention has failed•Detect attack in progress•Discover telltale system modificationsslide 3What Should Be Detected?Attempted and successful break-insAttacks by legitimate users•Illegitimate use of root privileges, unauthorized access to resources and data …Malware•Trojan horses, rootkits, viruses, worms …Denial of service attacksslide 4Intrusion Detection Systems (IDS)Host-based•Monitor activity on a single host•Advantage: better visibility into behavior of OS and individual applications running on the hostNetwork-based (NIDS)•Often placed on a router, firewall, or network gateway•Monitor traffic, examine packet headers and payloads•Advantage: single NIDS can protect many hosts and look for global patternsslide 5Intrusion Detection TechniquesMisuse detection•Use attack “signatures” (need a model of the attack)–Sequences of system calls, patterns of network traffic, etc.•Must know in advance what attacker will do (how?)•Can only detect known attacksAnomaly detection•Using a model of normal system behavior, try to detect deviations and abnormalities•Can potentially detect unknown (zero-day) attacksWhich is harder to do?slide 6Misuse Detection (Signature-Based)Set of rules defining a behavioral signature likely to be associated with attack of a certain type•Example: buffer overflow –A setuid program spawns a shell with certain arguments–A network packet has lots of NOPs in it–Very long argument to a string function•Example: denial of service via SYN flooding–Large number of SYN packets without ACKs coming back …or is this simply a poor network connection?Attack signatures are usually very specific and may miss variants of known attacks•Why not make signatures more general?slide 7Extracting Misuse SignaturesUse invariant characteristics of known attacks•Bodies of known viruses and worms, RET addresses of memory exploits, port numbers of applications with known vulnerabilities •Hard to handle mutations–Polymorphic viruses: each copy has a different bodyBig research challenge: fast, automatic extraction of signatures of new attacksslide 8Anomaly DetectionDefine a profile describing “normal” behavior•Works best for “small”, well-defined systems (single program rather than huge multi-user OS)Profile may be statistical•Build it manually (this is hard)•Use machine learning and data mining techniques–Log system activities for a while, then “train” IDS to recognize normal and abnormal patterns•Risk: attacker trains IDS to accept his activity as normal–Daily low-volume port scan may train IDS to accept port scansIDS flags deviations from the “normal” profileslide 9Statistical Anomaly DetectionCompute statistics of certain system activitiesReport an alert if statistics outside rangeExample: IDES (Denning, mid-1980s)•For each user, store daily count of certain activities–For example, fraction of hours spent reading email•Maintain list of counts for several days•Report anomaly if count is outside weighted normProblem: the most unpredictable user is the most importantslide 10“Self-Immunology” Approach Normal profile: short sequences of system calls•Use strace on UNIX… open,read,write,mmap,mmap,getrlimit,open,close …open,read,write,mmapread,write,mmap,mmap …write,mmap,mmap,getrlimitmmap,mmap,getrlimit,open…remember last K eventsCompute % of traces that have been seen before.Is it above the threshold?YNnormalabnormalRaise alarm if a high fraction ofsystem call sequences haven’tbeen observed before[Forrest]slide 11Level of MonitoringWhich types of events to monitor?•OS system calls•Command line•Network data (e.g., from routers and firewalls)•Keystrokes•File and device accesses•Memory accessesAuditing / monitoring should be scalableslide 12System Call InterpositionObservation: all sensitive system resources are accessed via OS system call interface•Files, sockets, etc.Idea: monitor all system calls and block those that violate security policy•Inline reference monitors•Language-level: Java runtime environment inspects stack of the function attempting to access a sensitive resource to check whether it is permitted to do so•Common OS-level approach: system call wrapper–Want to do this without modifying OS kernel (why?)slide 13Janus [Berkeley project, 1996]slide 14Policy DesignDesigning a good system call policy is not easyWhen should a system call be permitted and when should it be denied?Example: ghostscript•Needs to open X windows•Needs to make X windows calls•But what if ghostscript reads characters you type in another X window?slide 16Problems and PitfallsIncorrectly mirroring OS stateOverlooking indirect paths to resources•Inter-process sockets, core dumpsRace conditions (TOCTTOU)•Symbolic links, relative paths, shared thread meta-dataUnintended consequences of denying OS calls•Process dropped privileges using setuid but didn’t check value returned by setuid… and monitor denied the callBugs in reference monitors and
View Full Document