Systems and Internet Infrastructure Security Network and Security Research Center Department of Computer Science and Engineering Pennsylvania State University University Park PA Advanced Systems Security Virtual Machine Systems Trent Jaeger Systems and Internet Infrastructure Security SIIS Lab Computer Science and Engineering Department Pennsylvania State University Systems and Internet Infrastructure Security SIIS Laboratory March 18 2010 Page 1 Two Directions OS Security from Reference Monitor perspective Mediation LSM Tamperproof Linux and TCB Simple enough to verify Correct code Correct policy Systems and Internet Infrastructure Security SIIS Laboratory Page 2 Basis for OS Security Isolation A protection domain defines a boundary of isolation Based on Rings Address spaces Access control policy Do these work in modern OSes Systems and Internet Infrastructure Security SIIS Laboratory Page 3 Virtual Machine Systems Protection domain is extended to operating systems on one physical platform Invented for resource utilization But also provide a potential security benefit due to default ISOLATION How does VM isolation differ from OS isolation Systems and Internet Infrastructure Security SIIS Laboratory Page 4 Systems and Internet Infrastructure Security SIIS Laboratory Page 5 Systems and Internet Infrastructure Security SIIS Laboratory Page 6 Systems and Internet Infrastructure Security SIIS Laboratory Page 7 VM Systems and Ref Monitor How does a VM System improve ability to achieve reference monitor guarantees Mediation Mediation between VM interactions Tamperproof Protection boundaries between OS Simple Enough to Verify Code that needs to be correct Policy Systems and Internet Infrastructure Security SIIS Laboratory Page 8 VAX VMM A1 assured VMM system Carefully crafted VMM Mediation VM interaction Tamperproof Minimal TCB Simple enough to verify Code assurance Policy assurance MLS policy Biba policy privileges Systems and Internet Infrastructure Security SIIS Laboratory Page 9 VAX VMM Design Systems and Internet Infrastructure Security SIIS Laboratory Page 10 VAX VMM Reference Monitor Key design tasks Virtualize processor Make all sensitive instructions privileged More rings Need a new ring for the VMM I O emulation Self virtualizable What components constitute the VAX VMM reference monitor Systems and Internet Infrastructure Security SIIS Laboratory Page 11 VAX VMM Policy MLS Control secrecy Biba Control integrity Privileges Exceptional accesses Audited There are more of these than meets the eye How is the protection state modified Systems and Internet Infrastructure Security SIIS Laboratory Page 12 VAX VMM Evaluation Mediation ensure all security sensitive operations are mediated Virtualizing instructions I O emulation VM level operations Privileges Mediation mediate all resources VMM level Mediation verify complete mediation A1 assured at VMM level Systems and Internet Infrastructure Security SIIS Laboratory Page 13 VAX VMM Evaluation Tamperproof protect VMM Similar to Multics no gatekeepers but some kind of filters authentication in VMM protection system ops in VMM fixed system Tamperproof protect TCB All trusted code at ring 0 trusted path from VMs for admin Verification verify code A1 assured at VMM level Verification verify policy MLS and Biba express goals and policy Privileges are ad hoc Systems and Internet Infrastructure Security SIIS Laboratory Page 14 VAX VMM Tasks Despite A1 assurance still several challenges in VAX VMM system Device driver management no network Amount of assembler code Covert channel countermeasures Implications of privileges Nonetheless interesting mechanisms Trusted path administration Architecture of VMM Virtualization for security Systems and Internet Infrastructure Security SIIS Laboratory Page 15 Modern VM Systems The development of a virtual machine monitor for x86 systems unleashed VMs on the masses Why did this take so long VMware Xen KVM NetTop Everyone is a virtual machine monitor now How do we implement a reference validation mechanism for these systems What granularity of control Systems and Internet Infrastructure Security SIIS Laboratory Page 16 Isolation and Network VMware and NetTop assume that the VMM and privileged VM will isolate guest VMs Then the problem is to control inter VM communication Only other communication is via the network VMware uses firewall NetTop is built on VMware where only VMs of the same label may communicate Systems and Internet Infrastructure Security SIIS Laboratory Page 17 VMs as Processes Type II VM systems can treat VMs as processes KVM uses SELinux to control access of VMs as if they are a process VMs are processes to the host OS VMs can access host OS resources files Uses SELinux to control VM access Systems and Internet Infrastructure Security SIIS Laboratory Page 18 Control of VMM Resources There are many virtual machine monitor resources that may be used to communicate Memory devices IPC sHype adds reference monitor for some objects IPC and the privileged VM uses for networking Xen Security Modules XSM adds reference validation on the Xen hypervisor s distribution of these resources Less trust in privileged VMs so finer grained policy results Minimizing TCB versus simplicity Systems and Internet Infrastructure Security SIIS Laboratory Page 19 Xen as a Reference Monitor Reference Monitor XSM in Xen Scope includes dom0 Linux and user level Mediation XSM to control VMM operations SELinux in dom0 use network to communicate Tamperproof Xen has a much larger TCB and more flexible Verification Code lots Policy SELinux style Systems and Internet Infrastructure Security SIIS Laboratory Page 20 Trusted VMs VMware and NetTop assume that the privileged VM there is only one in these systems prevents information flow like a kernel Thus the only information flows between VMs are via networking Privileged VM controls inter VM communication via networking sHype controls IPC and networking at hypervisor level Privileged VM uses hypervisor as policy store Systems and Internet Infrastructure Security SIIS Laboratory Page 21 Take Away VM Systems provide isolation Between OSes apps that may be untrusted VM Systems enable a small TCB Type 1 VMMs A1 Assured like VAX VMM VM Systems can mediate inter VM actions Virtualized operations Inter VM operations Systems and Internet Infrastructure Security SIIS Laboratory Page 22