A Retrospective on the VAX VMMSecurity KernelPaul A. Karger, Mary Ellen Zurko, Douglas W. Bonin,Andrew H. Mason and Clifford E. KahnYogesh Raju Sreenivasan & Todd ArnoldOutline• Background• Related Work• Design/Methodology– Virtualizing VAX– Security– Layered Design– Engineering Issues– Human Interface• ConclusionBackground• Discretionary Access Controls– Access rights at the discretion of the owner.– ACL & Capability based systems– Covert Channels• Storage Channels• Timing Channels• Mandatory Access Controls– System defines access policy.– Lattice Security modelsBackground• NCSC Evaluation Criteria (4 Major Categories)• D-Minimal Protection• C-Discretionary Protection– C1-Discretionary Security Protection– C2-Controlled Access Protection• B-Mandatory Protection– B1-Labeled Security Protection– B2-Structured Protection– B3-Security Domains• A-Verified Protection– A1-Verified Design– Beyond A1http://en.wikipedia.org/wiki/TCSECBackground• Virtual Machine– “Efficient, isolated duplicate of the real machine” – Popek &Goldberg– A software program that emulates a hardware system.• Virtual Machine Monitor (or Hypervisor)– Control program that implements virtual machines.– Multiplexes multiple virtual resources onto a single physicalresource.– Properties of a VMM• Efficient• Control of Resources• Equivalence or TransparencyBackgroundVirtual Machine based approach for Security– Provides high degree of isolation between users.– All services and applications can be provided without extensivemodifications those Operating Systems.– More ReliableSource: http://www.intel.comRelated Work• IBM’s KVM/370• sHype – Security Architecture of Xen Hypervisor• Microsoft Virtual Server 2005• VMWare ESX ServerIntroduction• VAX Security Kernel is a virtual-machine monitor.• It provides an interface of the VAX Architecture andsupports VMS & ULTRIX-32 in virtual machines• 5 major goals for VAX VMM Security Kernel– Meet A1 security requirements.– Run on commercial hardware.– Provide software compatibility for applications.– Provide acceptable performance.– Commercial software product.Methodology• Virtualizing the VAXa) Sensitive Instructions:– “An architecture will support virtual machines if the set ofsensitive instructions is a subset of privileged instructions. “– Goldberg– Sensitive Instructions: those that read or modify privileged systemstate– Privileged Instructions: those that trap when executed from a non-privileged mode.– VAX architecture has some unprivileged but sensitiveinstructions (MOVPSL, PROBEx, REI)– Extensions to VAX Architecture– VM bit is added to the PSLMethodology• Virtualizing the VAXb) Ring Compression:– 4 protection rings.– User, Supervisor,Executive & Kernel- Real ring numbers areconcealed using1) VM Bit in PSL2) VMPSL3) Modify all instructions that reveal ring numbers- Change Memory protection of VM Pages.- No boundary between virtual kernel and executive mode!!- In both VMS & ULTRIX-32 Executive modes not used properlyVM Access ModesReal Access ModesMethodology• Virtualizing the VAXc) I/O Emulation:a) In VAX I/O devices are programmed by reading and writing CSRb) VAX Security Kernel I/O is a specialized kernel call mechanismoptimized for performance.c) VM stores I/O parameters in its I/O spaced) Real I/O takes place on MTPRd) Self-Virtualization:a) “Ability of the VMM to run on its own VM”b) VAX VMM Support self-virtualizationc) Useful for debugging.Methodology• Subjectsa) Users & Virtual Machinesb) Trusted path betweenUser and Server (trusted kernel process)a) VMs are untrusted subjects(can also be treated as objects).• Objectsa) Real devices, Volumes & Primary Memoryb) Disk volumesa) Exchangeable volumes and Security Kernel volumesc) Security Kernel filesa) System Databases & logsb) Virtual DisksUser VMMLOGINServerUser CONNECTUser SESSIONVM1)2)3)TRUSTED CONNECTIONMethodology• Access Classesa) VAX Kernel supports both secrecy (Bell LaPadula) andintegrity (Biba) models.b) Each kernel subject & kernel object has an access class (asecrecy class and an integrity class)c) Read Access: SAC dom OACd) Write Access: OAC dom SAC• Privilegesa) Analogous to Roles in RBACb) Restrict Access beyond DAC & MACc) User Privileges & Virtual-Machine privilegesLayered Design• Levels of abstraction• Based heavily on Multic’s• Layers add specificfunctionalityLayered Design (cont.)• Hardware-interrupt• Low-level scheduler• I/O Service• Virtual Machine PhysicalSpace Layer• Virtual Machine Virtual SpaceManagerHLS and VP InteractionVirtual Machines (VM)Level 1 Virtual Processes (VP1)Level 2 Virtual Processes (VP2)DedicatedVP1 - DeviceDriversBindable VP1 AddressableVP1DedicatedVP2 -ServerProcessesBindable VP2VAXLayered Design (cont.)• Audit trail• Files-11Files• Volumes• Virtual terminals• Virtual PrintersLayered Design (cont.)• Kernel Interface• Virtual Vax• Secure Server• Virtual machine OS• UsersSoftware Engineering Issues• Used multiple languages– Pascal (16.5%)– PL/I (60.0%)– MACRO (23.5%)• Memory Strategies– Sections of memory separated by no-access locations– Unused memory set to all zeroesSoftware Engineering Issues (cont.)• Defensive coding– Each layer protects against higher levels– Lower levels cannot call on higher levelsHuman Interface• Required to meet needs of commercial users• Two command sets– Secure Server Commands– SECURE Commands• User SECURE• VM SECURE• Secure Utilities– Reclassification• Design issuesAssurance• Code Design• Testing– Layered Tests– KCALL– DTM• Formal Methods• Covert Channel AnalysesProduction-Quality Kernels• Required tools– Quality compiler and debugger• RobustnessConclusion• VAX Security Kernel is a working production level kernel• Effectively deals with covert-channels• Successfully demonstrates work required to build A1level kernel• Obtaining performance is difficult, but not sufficient• Discipline required for A1 certification improves overallsoftware quality and
View Full Document
Unlocking...