Life of Interrupts: Remapping

Introduction


In early versions of Hyper-V, we used to start HV after NT kernel in the host is already loaded. In this case APIC was already initialized by NT and serving interrupts to NT kernel. Before we started HV, we disabled interrupts, copied APIC state to the virtual APIC and then started the hypervisor. From this point onwards, HV became owner of the physical APIC and NT kernel in host OS managed only virtual APIC state. In this case, HV needed to provide identity mapping for the interrupts as NT had already programmed various devices to generate specific interrupts and HV had no way to reprogram those devices. So even though hypervisor would intercept the interrupt, it would simply deliver the interrupt to NT. For its own use, HV relied upon NT to reserve interrupt vectors or use NMIs. For example, we used NMIs for inter-processor synchronization. HV kept track of whether NMI was requested by it or not, and if a phantom NMI was received, it was delivered to NT kernel for processing. This ensured that if NMIs are delivered due to a fault or other critical event that doesn’t belong to HV, NT received the NMI and handled it correctly. (more…)

Read More

Life of Interrupts: Virtualization

Introduction


Interrupt handling is one of the critical subsystem in the hypervisor (HV). It is critical both from correctness and performance perspective. Hyper-V supports multi-processor systems and uses many optimizations to improve performance for interrupt virtualization.

Interrupt handling is done via APIC emulation. For each vCPU, a vAPIC is created in the hypervisor. These vAPIC behaves similar to APIC in a physical system and provide interrupt support to virtual machines. To support operating systems that are not APIC aware, PIC emulation is provided using virtual wire mode as specified in Intel multiprocessor specifications.

In this post, I would talk about how interrupts are virtualized in Hyper-V environment and discuss some of the performance optimizations. (more…)

Read More

Life of Interrupts: Background

In this series of posts, I am going to talk about interrupt handling in a virtualized environment (specifically Hyper-V). This discussion would also include interrupt handling on systems that have IOMMU based interrupt remapping support. Interrupt remapping is required in Hyper-V for supporting SR-IOV enabled devices and device assignment to virtual machines.

The series is mostly written from software perspective and hardware details are only provided where necessary for understanding of the concepts. The series is focused on x86/x64 based architectures though the concepts described here can be applied to other architectures. (more…)

Read More