VIRTUALIZING PHYSICAL MEMORY IN A VIRTUAL MACHINE SYSTEM
A processor including a virtualization system of the processor with a memory virtualization support system to map a reference to guest-physical memory made by guest software executable on a virtual machine which in turn is executable on a host machine in which the processor is operable to a reference to host-physical memory of the host machine.
Virtualization enables a single host machine with hardware and software support for virtualization to present multiple abstractions of the host, such that the underlying hardware of the host machine appears as one or more independently operating virtual machines. Each virtual machine may therefore function as a self-contained platform. Often, virtualization technology is used to allow multiple guest operating systems and/or other guest software to coexist and execute apparently simultaneously and apparently independently on multiple virtual machines while actually physically executing on the same hardware platform. A virtual machine may mimic the hardware of the host machine or alternatively present a different hardware abstraction altogether.
Virtualization systems may include a virtual machine monitor (VMM) which controls the host machine. The VMM provides guest software operating in a virtual machine with a set of resources (e.g., processors, memory, IO devices). The VMM may map some or all of the components of a physical host machine into the virtual machine, and may create fully virtual components, emulated in software in the VMM, which are included in the virtual machine (e.g., virtual IO devices). The VMM may thus be said to provide a “virtual bare machine” interface to guest software. The VMM uses facilities in a hardware virtualization architecture to provide services to a virtual machine and to provide protection from and between multiple virtual machines executing on the host machine.
As guest software executes in a virtual machine, certain instructions executed by the guest software (e.g., instructions accessing peripheral devices) would normally directly access hardware, were the guest software executing directly on a hardware platform. In a virtualization system supported by a VMM, these instructions may cause a transition to the VMM, referred to herein as a virtual machine exit. The VMM handles these instructions in software in a manner suitable for the host machine hardware and host machine peripheral devices consistent with the virtual machines on which the guest software is executing. Similarly, certain interrupts and exceptions generated in the host machine may need to be intercepted and managed by the VMM or adapted for the guest software by the VMM before being passed on to the guest software for servicing. The VMM then transitions control to the guest software and the virtual machine resumes operation. The transition from the VMM to the guest software is referred to herein as a virtual machine Entry.
As is well known, a process executing on a machine on most operating systems may use a virtual address space, which is an abstraction of the underlying physical memory system. As is known in the art, the term virtual when used in the context of memory management e.g. “virtual address,” “virtual address space,” “virtual memory address” or “virtual memory space,” refers to the well known technique of a processor based system, generally in conjunction with an operating system, presenting an abstraction of underlying physical memory to a process executing on a processor-based system. For example, a process may access a virtual, contiguous and linearized address space abstraction which is mapped to non-linear and non-contiguous physical memory by the underlying operating system. This use of virtual is distinguishable from the use of the same term used in the context virtualization, where virtual generally refers to an abstraction that simulates a physical machine e.g. “virtual machine,” “virtual bare machine,” “virtual hardware,” “virtual processor” or “virtual network interface.” The intended meaning of the term will be clear to one in the art based on the context in which it is used herein.
Memory is customarily divided into pages, each page containing a known amount of data, varying across implementations, e.g. a page may contain 4096 bytes of memory. As memory locations are referenced by the executing process, they are translated into page references. In a typical machine, memory management maps a reference to a page in process virtual memory to a page in machine physical memory. In general, memory management may use a page table to specify the physical page location corresponding to a process space page location.
One aspect of managing guest software in a virtual machine environment is the management of memory. Handling memory management actions taken by the guest software executing in a virtual machine creates complexity for a controlling system such as a virtual machine monitor. Consider for example a system in which two virtual machines execute via virtualization on a host machine implemented on a 32-bit IA-32 Intel® Architecture platform (IA-32), which is described in the IA-32 Intel® Architecture Software Developer's Manual (IA-32 documentation). The IA-32 platform may include IA-32 page tables implemented as part of an IA-32 processor. Further, assume that each virtual machine itself presents an abstraction of an IA-32 machine to the guest software executing thereon. Guest software executing on each virtual machine may make references to a guest process virtual memory address, which in turn is translated by the guest machine's memory management system to a guest-physical memory address. However, guest-physical memory itself may be implemented by a further mapping in host-physical memory through a VMM and the virtualization subsystem in hardware on the host processor. Thus, references to guest memory by guest processes or the guest operating system, including for example references to guest IA-32 page table control registers, must then be intercepted by the VMM because they cannot be directly passed on to the host machine's IA-32 page table without further reprocessing, as the guest-physical memory does not, in fact, correspond directly to host-physical memory but is rather further remapped through the virtualization system of the host machine.
As shown in
The virtual machines and memory mapping shown in
A processor-based system that is presented as a virtual machine in a system such as that depicted in
To this end, the VMM must trap a variety of events surrounding the use of the paging mechanism by the guest software. This includes writes to control registers such as control registers of the IA-32 memory management system (e.g., CR0, CR3 and C4), accesses to model-specific registers (MSRs) associated with paging and memory access (e.g., memory-type range registers (MTRRs)), handling certain exceptions (e.g., page faults), as described in the IA-32 documentation. This use of the IA-32 page tables to virtualize physical memory is complex and exacts a significant performance overhead.
The platform hardware 316 may be a personal computer (PC), mainframe, handheld device such as a personal digital assistant (PDA) or “smart” mobile phone, portable computer, set top box, or another processor-based system. The platform hardware 316 includes at least a processor 318 and memory 320. Processor 318 may be any type of processor capable of executing programs, such as a microprocessor, digital signal processor, microcontroller, or the like. The processor may include microcode, programmable logic or hard coded logic for execution in embodiments. Although
The VMM 312 presents to guest software an abstraction of one or more virtual machines, which may provide the same or different abstractions to the various guests.
In one embodiment, the processor 318 controls the operation of the virtual machines 302 and 314 in accordance with data stored in a virtual machine control structure (VMCS) 324. The VMCS 324 is a structure that may contain state of guest software 303 and 313, state of the VMM 312, execution control information indicating how the VMM 312 wishes to control operation of guest software 303 and 313, information controlling transitions between the VMM 312 and a virtual machine, etc. The processor 318 reads information from the VMCS 324 to determine the execution environment of the virtual machine and to constrain its behavior. In one embodiment, the VMCS 324 is stored in memory 320. In some embodiments, multiple VMCS structures are used to support multiple virtual machines.
The VMM 312 may need to manage the physical memory accessible by guest software running in the virtual machines 302 and 314. To support physical memory management in one embodiment, the processor 318 provides an extended page table (EPT) mechanism. In the embodiment, the VMM 312 may include a physical memory management module 326 that provides values for fields associated with physical memory virtualization that may need to be provided before transition of control to the virtual machine 302 or 314. These fields are collectively referred to as EPT controls. EPT controls may include, for example, an EPT enable indicator specifying whether the EPT mechanism should be enabled and one or more EPT table configuration controls indicating the form and semantics of the physical memory virtualization mechanism. These will be discussed in detail below. Additionally, in one embodiment, EPT tables 328 indicate the physical address translation and protection semantics which the VMM 312 may place on guest software 303 and 313.
In one embodiment, the EPT controls are stored in the VMCS 324. Alternatively, the EPT controls may reside in a processor 318, a combination of the memory 320 and the processor 318, or in any other storage location or locations. In one embodiment, separate EPT controls are maintained for each of the virtual machines 302 and 314. Alternatively, the same EPT controls are maintained for both virtual machines and are updated by the VMM 312 before each virtual machine entry.
In one embodiment, the EPT tables 328 are stored in memory 320. Alternatively, the EPT tables 328 may reside in the processor 318, a combination of the memory 320 and the processor 318, or in any other storage location or locations. In one embodiment, separate EPT tables 328 are maintained for each of the virtual machines 302 and 314. Alternatively, the same EPT tables 328 are maintained for both virtual machines 302 and 314 and are updated by the VMM 312 before each virtual machine entry.
In one embodiment, the processor 318 includes EPT access logic 322 that is responsible for determining whether the EPT mechanism is enabled based on the EPT enable indicator. If the EPT mechanism is enabled, the processor translates guest-physical addresses to host-physical addresses-based on the EPT controls and EPT tables 328.
In one embodiment, in which the system 300 includes multiple processors or multi-threaded processors, each of the logical processors is associated with a separate EPT access logic 322, and the VMM 312 configures the EPT tables 328 and EPT controls for each of the logical processors.
Resources that can be accessed by guest software (e.g., 303, including guest OS 304 and application 308) may either be classified as “privileged” or “non-privileged.” For privileged resources, the VMM 312 facilitates functionality desired by guest software while retaining ultimate control over these privileged resources. Further, each guest software 303 and 313 expects to handle various platform events such as exceptions (e.g., page faults, general protection faults, etc.), interrupts (e.g., hardware interrupts, software interrupts), and platform events (e.g., initialization (INIT) and system management interrupts (SMIs)). Some of these platform events are “privileged” because they must be handled by the VMM 312 to ensure proper operation of virtual machines 302 and 314 and for protection from and among guest software. Both guest operating system and guest applications may attempt to access privileged resources and both may cause or experience privileged events. Privileged platform events and access attempts to privileged resources are collectively referred to as “privileged events' or ‘virtualization events” herein.
At 440, guest software 303 accesses a non-privileged resource 442. Non-privileged resources do not need to be controlled by the VMM 312 and can be accessed directly by guest software which continues without invoking the VMM 312, allowing the guest to continue operation at 445 after accessing the non-privileged resource 442. A non-privileged platform event would likewise be handled without the intervention of the VMM 312 (this is not shown in
At 405, the guest software 303 attempts to access a privileged resource, and/or experiences a privileged platform event. When such a privileged event occurs as at 405, control may be transferred 407 to the VMM 312. The transfer of control 407 from guest software to the VMM 312 is referred to herein as a virtual machine exit. After facilitating the resource access or otherwise handling the privileged event appropriately, the VMM 312 may return control to guest software as at 432 which then resumes operation, 435. The transfer of control 432 from the VMM 312 to guest software is referred to as a virtual machine entry. In one embodiment, the VMM 312 initiates a virtual machine entry by executing an instruction specially designed to trigger the transition, 430, referred to herein as a virtual machine entry instruction.
In one embodiment, when a virtual machine exit occurs, components of the processor state used by guest software are saved, 410, components of the processor state required by the VMM 312 are loaded, and the execution resumes in the VMM 312 at 420. In one embodiment, the components of the processor state used by guest software are stored in a guest-state area of VMCS 324 and the components of the processor state required by the VMM 312 are stored in a monitor-state area of VMCS 324. In one embodiment, when a transition from the VMM 312 to guest software occurs, components of the processor state that were saved at the virtual machine exit (and may have been modified by the VMM 312 while processing the virtual machine exit) are restored 425 and control is returned to the virtual machine 302 or 314 at 430.
The components of the VM environment in the embodiment depicted in
The EPT mechanism itself may be configured by the VMM 312 which configures the EPT tables 328 and the EPT controls which may be stored in the VMCS 324. In this embodiment, the configuration of the EPT mechanism may be done by the VMM 312 as part of the operation of the physical memory management module 326 following the processing of a privileged event 405 in the VMM 312 and prior to VM entry 430. In configuring the EPT mechanism, the VMM 312 may update the EPT tables 328 and EPT controls, in order to enable, disable or otherwise control the EPT mechanism, 460.
Of course, many other forms of processing are possible for the use of extended paging tables in conjunction with a VM environment, for example, different locations for the EPT controls and EPT tables 328 as discussed earlier with reference to
In this example, the appropriate bits 502 in the CR3 register 520 point to the base of the guest's page directory table 560 in guest-physical memory. This value 502 is combined with the upper bits from the guest virtual address 510 (appropriately adjusted, according to IA-32 semantics by multiplying by 4 because, in this example, the entries in the tables are 4 bytes each) to form the guest-physical address 512 of the page directory entry (PDE) in the guest's PD table 560. This value 512 is translated through the EPT tables 555 to form the host-physical address 504 of the page directory entry. The processor accesses the page directory entry using this host-physical address 504.
Information from the PDE includes the base address 522 of the guest's page table 570. This guest-physical address 522 is combined with bits 21:12 of the guest virtual address 510 appropriately adjusted to form the guest-physical address 532 of the page table entry in the guest's page table 570. This guest-physical address 532 is translated through the EPT tables 565 to form the host-physical address 514 of the guest's page table entry (PTE). The processor accesses the PTE using this host-physical address 514.
Information from the PTE includes the base address 542 of the page in guest-physical memory being accessed. This value is combined with the low-order bits (11:0) of the guest virtual address 510 to form the guest-physical address 552 of the memory being accessed. This value 552 is translated through the EPT tables 575 to form the host-physical address 524 of the memory being accessed.
Each time the EPT tables are used to translate a guest-physical address to a host-physical address, the processor also validates that the access is permitted according to controls in the EPT tables, as will be described below. Additionally, it must be understood that the EPT tables 555, 565, and 575, though indicated as distinct in
Part of the EPT table entry 651 is the base address 612 of the next-level EPT table 660. A second adjusted address component 613 is formed from bits 29:21 (611) of the guest-physical address 610. This adjusted value 613 is combined with (added to) the base address 612 to form the host-physical address 614 of an EPT table entry 661 in the next level EPT table 660. The processor accesses the EPT table entry 661 using this host-physical address 614.
Information from the EPT table entry 661 includes the base address 622 of the final EPT table 670. This base address 622 is combined with adjusted bits 20:12 (623) of the guest virtual address 610 after appropriately adjusting to form the address 624 of the EPT table entry 671 in the final EPT table 670. The processor accesses the EPT table entry using this host-physical address 624.
Information from the EPT table entry 671 includes the base address 632 of the page being accessed in host-physical memory 690. This page address value 690 is combined with the low-order bits (11:0) of the guest-physical address 610 to form the final host-physical address 634 of the memory being accessed.
In the exemplary embodiment shown in
The hierarchical configuration depicted in the figure shows three levels of hierarchy, with two of the EPT tables 650 and 660 serving as indices to lower level EPT tables 660 and 670 respectively. In other embodiments, there may be fewer, e.g. two levels, or more e.g. four or more, levels of hierarchy in such a hierarchical table. In general the number of levels of hierarchy may vary depending at least in part on one or more of the number of bits in the guest-physical address, the size of each table, and the number of bytes in each table entry. The guest-physical address in the example in
In the embodiment shown in
The EPT tables may be in a variety of different formats. For example, they may be implemented as shown in
The EPT tables may support one or more sizes of pages in host-physical memory. In one embodiment, each entry in each EPT table includes a super page bit which indicates that the walk of the EPT tables should stop at that point and the host-physical address formed using the address information in the EPT table entry and the remaining bits in the guest-physical address. In the example shown in
In some embodiments, the extended paging tables and EPT address translation mechanism may be enabled by virtual machine entry and disabled by virtual machine exit. Thus, as a consequence, the EPT mechanism may not be available for use by either guest software or VMM software to manage its own address translation. Furthermore, in such embodiments, the EPT mechanism may be distinct from and independent of other conventional memory page management mechanisms available to guest or host software, such as for example the IA-32 paging tables in an IA-32 embodiment, though EPT operations may utilize features of the conventional page management mechanism. Thus, the organization and operation of the EPT tables may be completely distinct from other page translation facilities provided by the processor for conventional execution of programs and operations directly on the host machine, as opposed to the execution of guest software that utilizes the virtualization and EPT mechanisms of the host machine. In one embodiment, the EPT mechanism may utilize tables in the same format as that used by a conventional page management mechanism of the embodiment, available to guest and VMM software. However, the tables controlling the EPT mechanism may still be distinct from those controlling either guest-virtual address to guest-physical address translation and from those controlling host-virtual address to host-physical address translation.
Although the examples provided may describe providing support for physical memory virtualization in a virtual machine system in the context of execution units and logic circuits, other embodiments can be accomplished by way of software. Some embodiments may be provided as a software program product or software which may include a machine or machine-readable medium having stored thereon instructions which when accessed by the machine perform a process of the embodiment. In other embodiments, processes might be performed by specific hardware components that contain hardwired logic for performing the processes, or by any combination of programmed components and custom hardware components.
In the preceding description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the described embodiments, however, one skilled in the art will appreciate that many other embodiments may be practiced without these specific details.
Some portions of the detailed description above is presented in terms of algorithms and symbolic representations of operations on data bits within a processor-based system. These algorithmic descriptions and representations are the means used by those skilled in the art to most effectively convey the substance of their work to others in the art. The operations are those requiring physical manipulations of physical quantities. These quantities may take the form of electrical, magnetic, optical or other physical signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the description, terms such as “executing” or “processing” or “computing” or “calculating” or “determining” or the like, may refer to the action and processes of a processor-based system, or similar electronic computing device, that manipulates and transforms data represented as physical quantities within the processor-based system's storage into other data similarly represented or other such information storage, transmission or display devices.
In the description of the embodiments, reference may be made to accompanying drawings. In the drawings, like numerals describe substantially similar components throughout the several views. Other embodiments may be utilized and structural, logical, and electrical changes may be made. Moreover, it is to be understood that the various embodiments, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described in one embodiment may be included within other embodiments.
Further, a design of an embodiment that is implemented in a processor may go through various stages, from creation to simulation to fabrication. Data representing a design may represent the design in a number of manners. First, as is useful in simulations, the hardware may be represented using a hardware description language or another functional description language. Additionally, a circuit level model with logic and/or transistor gates may be produced at some stages of the design process. Furthermore, most designs, at some stage, reach a level of data representing the physical placement of various devices in the hardware model. In the case where conventional semiconductor fabrication techniques are used, data representing a hardware model may be the data specifying the presence or absence of various features on different mask layers for masks used to produce the integrated circuit. In any representation of the design, the data may be stored in any form of a machine-readable medium. An optical or electrical wave modulated or otherwise generated to transmit such information, a memory, or a magnetic or optical storage such as a disc may be the machine readable medium. Any of these mediums may “carry” or “indicate” the design or software information. When an electrical carrier wave indicating or carrying the code or design is transmitted, to the extent that copying, buffering, or re-transmission of the electrical signal is performed, a new copy is made. Thus, a communication provider or a network provider may make copies of an article (a carrier wave) that constitute or represent an embodiment.
Embodiments may be provided as a program product that may include a machine-readable medium having stored thereon data which when accessed by a machine may cause the machine to perform a process according to the claimed subject matter. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, DVD-ROM disks, DVD-RAM disks, DVD-RW disks, DVD+RW disks, CD-R disks, CD-RW disks, CD-ROM disks, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing electronic instructions. Moreover, embodiments may also be downloaded as a program product, wherein the program may be transferred from a remote data source to a requesting device by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
Many of the methods are described in their most basic form but steps can be added to or deleted from any of the methods and information can be added or subtracted from any of the described messages without departing from the basic scope of the claimed subject matter. It will be apparent to those skilled in the art that many further modifications and adaptations can be made. The particular embodiments are not provided to limit the claimed subject matter but to illustrate it. The scope of the claimed subject matter is not to be determined by the specific examples provided above but only by the claims below.
14. A processor comprising:
- a virtualization system of the processor, the virtualization system comprising a memory virtualization support system including an extended paging table to map a reference to guest-physical memory made by guest software executable on a virtual machine which in turn is executable on a host machine in which the processor is operable, to a reference to host-physical memory of the host machine.
15. The processor of claim 14 wherein the memory virtualization support system is further to specify restrictions on access and to control access to a location of the guest-physical memory in accordance with the restrictions.
16. The processor of claim 14 wherein the memory virtualization support system is further to partition the host-physical memory of the host machine.
17. The processor of claim 15 wherein the virtualization support system transitions to a virtual machine monitor in response to an attempt to access memory in violation of the restrictions.
18. The processor of claim 14 wherein the extended paging table is further to map a reference to a page of guest-physical memory to a reference to a page of host-physical memory.
19. The processor of claim 18 wherein the extended paging table specifies an access restriction for a page of guest-physical memory.
20. The processor of claim 19 wherein the access restriction for a page of guest-physical memory comprises at least one of the following:
- read access control information,
- write access control information, and
- execute access control information.
21. The processor of claim 19 wherein the virtualization system transitions to a virtual machine monitor in response to a violation of the access restriction for a page of guest-physical memory.
22. The processor of claim 18 wherein the extended paging table comprises a multi-level extended paging table.
23. The processor of claim 14 wherein the memory virtualization support system is activated in response to the activation of the virtualization system of the processor only if a flag of the virtualization system is set to a predetermined value.
24. A method comprising:
- mapping a reference to guest-physical memory made by guest software executable on a virtual machine which in turn is executable on a processor-based host machine, to a reference to host-physical memory of the host machine using an extended paging table of the host machine.
25. The method of claim 24 further comprising specifying restrictions on access and controlling access to a location of the guest-physical memory in accordance with the restrictions.
26. The method of claim 24 further comprising partitioning the host-physical memory of the host machine.
27. The method of claim 25 further comprising the virtualization support system transitioning to a virtual machine monitor in response to an attempt to access memory in violation of the restrictions.
28. The method of claim 24 further comprising mapping a reference to a page of guest-physical memory to a reference to a page of host-physical memory.
29. A processor-based system comprising:
- a virtualization system of a processor, the virtualization system comprising a memory virtualization support system including an extended paging table to map a reference to guest-physical memory made by guest software executable on a virtual machine which in turn is executable on a host machine in which the processor is operable, to a reference to host-physical memory of the host machine.
30. The system of claim 29 wherein the memory virtualization support system is further to specify restrictions on access and control access to a location of the guest-physical memory in accordance with the restrictions.
31. The system of claim 29 wherein the memory virtualization support system is further to partition the host-physical memory of the host machine.
32. The system of claim 30 wherein the virtualization support system transitions to a virtual machine monitor in response to an attempt to access memory in violation of the restrictions.
33. The system of claim 29 wherein the extended paging table is further to map a reference to a page of guest-physical memory to a reference to a page of host-physical memory.
Filed: Sep 18, 2018
Publication Date: Apr 25, 2019
Inventors: Steven M. Bennett (Hillsboro, OR), Andrew V. Anderson (Forest Grove, OR), Gilbert Neiger (Portland, OR), Rajesh Sankaran Madukkarumukumana (Portland, OR), Richard Uhlig (Hillsboro, OR), Lawrence Smith, III (Beaverton, OR), Scott D. Rodgers (Hillsboro, OR)
Application Number: 16/134,809