Systems and Internet Infrastructure Security Network and Security Research Center Department of Computer Science and Engineering Pennsylvania State University University Park PA Advanced Systems Security VMs and Untrusted OSes Trent Jaeger Systems and Internet Infrastructure Security SIIS Lab Computer Science and Engineering Department Pennsylvania State University Systems and Internet Infrastructure Security SIIS Laboratory March 23 2010 Page 1 Conventional OS vs VM System Conventional OS Broken easily and often VM system Coarser control based on isolation If we trust the VM system and don t trust the OS what can we do Systems and Internet Infrastructure Security SIIS Laboratory Page 2 Untrusted OS Don t trust OS but need its services Run programs on a specialized trusted system But use conventional OS like an untrusted network Run programs directly on VMM But use conventional OS like an untrusted network must use more OS How do we accomplish these options Systems and Internet Infrastructure Security SIIS Laboratory Page 3 Options Microkernels Reduce code running in kernel mode But need the same services These are just as trusted running in user space SELinux AppArmor Trusted Solaris What do you think Isolate in VM systems e g Terra Can deploy an application on a custom OS Still have to trust all services used though Systems and Internet Infrastructure Security SIIS Laboratory Page 4 Options Hardware XOM Still need to configure software services Secure co processors IBM 4758 Extremely specialized and limited devices Systems and Internet Infrastructure Security SIIS Laboratory Page 5 Splitting Interfaces Systems and Internet Infrastructure Security SIIS Laboratory Page 6 Splitting Interfaces Systems and Internet Infrastructure Security SIIS Laboratory Page 7 Proxos Architecture Systems and Internet Infrastructure Security SIIS Laboratory Page 8 Proxos Guarantees Systems and Internet Infrastructure Security SIIS Laboratory Page 9 Proxos Routing Language Systems and Internet Infrastructure Security SIIS Laboratory Page 10 Proxos Implementation Systems and Internet Infrastructure Security SIIS Laboratory Page 11 Proxos Implementation Systems and Internet Infrastructure Security SIIS Laboratory Page 12 Proxos SSH Server Systems and Internet Infrastructure Security SIIS Laboratory Page 13 Compare to Privilege Separation Systems and Internet Infrastructure Security SIIS Laboratory Page 14 Implementation Effort Systems and Internet Infrastructure Security SIIS Laboratory Page 15 Performance Systems and Internet Infrastructure Security SIIS Laboratory Page 16 Remaining Problem Deploying a custom OS is painful Building a special kernel is non trivial And it may not be secure itself Still need a methodology to determine code correctness and tamperproofing What if you want to eliminate trust in the OS altogether Systems and Internet Infrastructure Security SIIS Laboratory Page 17 A solution should Ease Adoption It is usable Support Diverse Applications to a lot of people Have an Incremental Path to Higher Assurance also Systems and Internet Infrastructure Security SIIS Laboratory Page 18 Insight Shadowing Memory VMMs need to manage physical to virtual mapping of memory This is done with a shadow page table Multi shadowing give context aware views of this memory Use encryption instead Systems and Internet Infrastructure Security SIIS Laboratory Page 19 Memory Cloaking Not new idea XOM LT Leverage the awesome power of VMMs Encrypt the pages in memory IV H meta data This is used for writes to disk too How do we store the metadata Systems and Internet Infrastructure Security SIIS Laboratory Page 20 Tasks of the Overshadow Context Identification Secure Control Transfer System Call Adaptation Mapping Cloaked Resources Managing Protection Metadata Systems and Internet Infrastructure Security SIIS Laboratory Page 21 Shim baby Shim The key to overshadow is the Shim Manages transitions to and from VMM via a hypercall Shim Memory protects application CTC protects control registers Uncloaked Shim Neutral ground Trampoline Systems and Internet Infrastructure Security SIIS Laboratory Page 22 Loading Applications The Shim uses a Loader program Sets up the cloaked memory with a hypercall The loader shim must be trusted Metadata on the CTC checks for compromise Here is the meat of the problem Is it even used Propagate shims to spawned applications Systems and Internet Infrastructure Security SIIS Laboratory Page 23 Its not that easy Lot of OS interfaces that must be handled Faults Interrupts System Calls Pass control to the VMM The shim catches this and stores registers Clear the registers to prevent side channels Systems and Internet Infrastructure Security SIIS Laboratory Page 24 Complex Syscalls Some syscalls are easy No side effects Nice getpid sync Others less so Pipe r w Zero data Clone Fork Signal Handling Systems and Internet Infrastructure Security SIIS Laboratory Page 25 Performance Microbenchmarks Not so hot Application Benchmarks SPEC isn t so bad High bandwidth hits some bottlenecks Why Systems and Internet Infrastructure Security SIIS Laboratory Page 26 Take Away VM Systems provide isolation At OS granularity some can be untrusted OS provides services used by applications Access to devices demultiplexed among VMs Can we use VM isolation to prevent compromise of applications by OS compromise Proxos use a trusted OS and redirect service requests Overshadow use OS as untrusted communication media Systems and Internet Infrastructure Security SIIS Laboratory Page 27