1Edgar GabrielCOSC 6385 Computer Architecture Virtualizing Compute ResourcesEdgar GabrielFall 2009COSC 6385 – Computer ArchitectureEdgar GabrielReferences[1] J. L. Hennessy, D. A. Patterson“Computer Architecture – A Quantitative Approach”Chapter 5.4[2] G. Neiger, A. Santoni, F. Leung, D. Rodgers, R. Uhlig:“Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization”,Intel Technology Journal., Vol. 10, No. 3. (August 2006), pp. 167-177.ftp://download.intel.com/technology/itj/2006/v10i3/v10-i3-art01.pdf2COSC 6385 – Computer ArchitectureEdgar GabrielMotivation• Server consolidation– What are you doing with hundreds of cores in a single server?• Sandboxing– Have the web server and the database server on the same physical machine but run in different OS– If the web server is hacked, the data base server is not necessarily affected by that• Multiple execution environments– Service a runs best on Windows, Service b runs best on LinuxCOSC 6385 – Computer ArchitectureEdgar GabrielVirtualization• Goal:– Run multiple instances of different OS on the same hardware– Present a transparent view of one or more environments (M-to-N mapping of M “real” resources, N “virtual” resources)• Challenges:– Have to split all resources (processor, memory, hard drive, graphics card, networking card etc.) among the different OS -> virtualize the resources– The OS can not be aware that it is using virtual resources instead of real resources3COSC 6385 – Computer ArchitectureEdgar GabrielCOSC 6385 – Computer ArchitectureEdgar GabrielVirtualization - concepts• Two components when using virtualization:– Virtual Machine Monitor (VMM)– Virtual Machine(s) (VM)• Para-virtualization:– Operating System has been modified in order to run as a VM• ‘Fully‘ Virtualized:– No modification required of an OS to run as a VM4COSC 6385 – Computer ArchitectureEdgar GabrielVirtual Machine Monitor• Also called ‘hypervisor’• Isolates the state of each guest OS from each other• Protects itself from guest software• Determines how to map virtual resources to physical resources– Access to privileged state– Address translation– I/O– Exceptions and interrupts• Relatively small code ( compared to an OS)• VMM must run in a higher privilege mode than guest OSCOSC 6385 – Computer ArchitectureEdgar GabrielManaging Virtual Memory (I)• Virtual memory offers many of the features required for hardware virtualization– Separates the physical memory onto multiple processes– Each process ‘thinks’ it has a linear address space of full size– Processor holds a page table translating virtual addresses used by a process and the according physical memory– Additional information restricts processes from • Reading a page of on another process or• Allow reading but not modifying a memory page or• Do not allow to interpret data in the memory page as instructions and do not try to execute them5COSC 6385 – Computer ArchitectureEdgar GabrielManaging Virtual Memory (II)• Virtual Memory management thus requires– Mechanisms to limit memory access to protected memory– At least two modes of execution for instructions• Privileged mode: an instruction is allowed to do what it whatever it wants -> kernel mode for OS• Non-privileged mode: user-level processes• Intel x86 Architecture: processor supports four levels– Level 0 used by OS– Level 3 used by regular applications– Provide mechanisms to go from non-privileged mode to privileged mode -> system callCOSC 6385 – Computer ArchitectureEdgar GabrielManaging Virtual Memory (III)– Provide a portion of processor state that a user process can read but not modify• E.g. memory protection information6COSC 6385 – Computer ArchitectureEdgar GabrielManaging Virtual Memory (IV)• Each guest OS maintains its page tables to do the mapping from virtual address to physical address• Most simple solution: VMM holds an additional table which maps the physical address of a guest OS onto the ‘machine address’– Introduces a third level of redirection for every memory access• Alternative solution: VMM maintains a shadow page table of each guest OS– Copy of the page table of the OS– Page tables still works with regular physical addresses– Only modifications to the page table are intercepted by the VMMCOSC 6385 – Computer ArchitectureEdgar GabrielISA related problems when running a guest OS• Ring Aliasing: arises when software runs at a privilege level other than which it has been written for– E.g. OS wants to modify page table• Address Space Compression: OS expects to have access to the processors full virtual address– VMM needs a portion of the virtual address space• Non-faulting access to privileged State– E.g. read-access to certain state registers• Adverse Impact on Guest System Calls– A system call on a guest OS should not change to the privilege level 0, but to the level of the guest OS7COSC 6385 – Computer ArchitectureEdgar GabrielISA related problems when running a guest OS (II)• Interrupt Virtualization– IA32 architecture provides the ability to mask external interrupts if the OS is not ready to handle it– OS frequently uses this features– VMM will have to handle external interrupts (e.g. hard drive, network), since it has to determine which guest OS needs to be ‘informed’ about that• Access to Hidden State– E.g. IA32 has hidden descriptor caches for segment registers– No mechanism available saving a restoring hidden components of a guest context when changing VMCOSC 6385 – Computer ArchitectureEdgar GabrielIntel Virtualization Architecture for IA32• Two new forms of CPU operations:– VMX root operations: intended for use by a VMM– VMX non-root operations: intended for use by a VM– Orthogonal concept to the privileges, i.e. both forms support all four privilege levels– VM entry: operation to enter VMX non-root mode– VM exit: Operation to enter VMX root mode• Instructions and events can cause VM exits8COSC 6385 – Computer ArchitectureEdgar Gabriel• Virtual Machine Control Structure (VMCS): – manages VM entries and VM exits. – Contains a guest-state area and a host-state area.– VM entry loads processor state from the guest-state area– VM exit saves processor state to the guest-state area and loads processor state from the host-state area.COSC 6385 – Computer ArchitectureEdgar