Hyper-V Nested Virtualization Setup
In our adventures for knowledge and understanding, at some point you have to start a lab environment to test and verify things before performing them in a production network. For carrier networking, this can be especially challenging as everything involved with service provider networking is proprietary for the vendor chassis choices and the associated instructions with the images are normally related to niche, ideal environments. So I decided to spend some time working out how to do things as ‘easy’ as possible. And in a way that does not take a large out of pocket expense.
System Specifics:
The following was performed on an Intel i7 13700KF processor that has 12 traditional and 8 efficiency cores, with VIT-D and hyper-threading enabled. Which means 40 vCPU cores for a virtual environment. I will be protecting 2 physical cores and 12GB of RAM, for my host system. And the rest of my 18 physical cores and 20GB of RAM are being dedicated to the hyper-v virtual environment.
- CPU: Intel i7 13700KF “Core” (No integrated GPU in the CPU)
- RAM: 32GB
- SSD: WD Black 2TB M2
- External Media: 32GB SSD Thumb Drive
- Host OS: Windows 11 HOME (Hyper-V Manager and functions must be added to this OS)
- Guest OS: Ubuntu 22.04.3 LTS
- Nested VM: vJunos-router-23.2R1.15qcow2 (lite-mode developer licensed image. No license requirement)
- Prerequisites: I am not covering how to enable the motherboard/cpu virtualization functions. It is fairly easy and straight forward. But is Vendor/Bios specific. So please follow your hardware vendor instructions for enabling virtualization at the hardware level. And I will hopefully help the software manipulation easier from there.
Hyper-V Enablement and Configuration
The first step to functional virtualization on my Windows 11 Home installation, is to install the Hyper-V manager module. As it is not part of the base version of the operating system. And many resources will tell you that hyper-v is not available or possible in a home version. But that is not the case, it is a functionality that can be added to any version of Windows 10/11 beyond the windows 10 anniversary edition. I can’t remember the exact revision number off of the top of my head, but it’s somewhere around 20H2.
Moving right along, the point of this section is enablement and configuration of hyper-v. So let’s get to it! To execute the following command, you will need to open a ‘PowerShell’ or regular command shell with ADMINISTRATOR PRIVILEGE, and run the following command
DISM /online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V
This process will take a couple of minutes to complete. Once it has completed, you will need to reboot your PC. And then if you hit the WINDOWS key and type ‘hyper’, the ‘Hyper-V Manager’ should pop up as an option. When we get to the point of opening the “Hype-V Manager” application. Make sure that you do so with ADMINISTRATOR PRIVILEGES. Otherwise you will have issues with file access and creation when working within the virtualization tools.
Before you start up Hyper-V Manager for the first time, lets do one more thing that is crucial, for being able to run QEMU images natively within your windows environment. We must enable the ability for a ‘Guest OS’ or ‘Virtual-Machine’ The ability to control it’s own CPU resources that you give it access to. In this example, I will be passing the default of 10 (half) of my physical cores to the Ubuntu installation that will be later in this post. The following command string requires the Powershell instance with ADMINISTRATOR PRIVILEGES. And is required for running nested virtualized environments. Such as hosting QEMU VMs on our VM installation of Ubuntu.
Set-VMProcessor -VMName "Ubuntu22" -ExposeVirtualizationExtensions $true
#To remove nested virtualization from a named VM instance, just change $true to $false:
Set-VMProcessor -VMName "Ubuntu22" -ExposeVirtualizationExtensions $false
In order to run this command successfully, you must have created the virtual-machine instance already, which we haven’t covered yet. And I will refer back to this section when we have completed the VM installation process and this command can be executed successfully.
Spinning Up a Nested Virtualization Enabled Ubuntu Instance
Okay, to recap where we should be at so far. You should have your hardware/bios firmware configured for virtualization. You should have the ability to open an instance of “Hyper-V Manager” with administrator privileges. And have the OS image, that you want to run in the virtualization. I am using Ubuntu 22.04.3 in this example. But you could just as easy spin up an Alma/Rocky Linux, esxi or other preferred flavor of environment. Shooters choice.
Now that we have opened “Hyper-V Manager” with administrator privileges, lets go to the top-right menu and select > NEW > Virtual-Machine. This will open the “New Virtual Machine Wizard” and you can read the disclaimer, and check the box to never see it again, then click ‘NEXT’. Or you can just click ‘NEXT’ and leave the reminder.
The next step is to give the virtual-machine a system-wide unique name, that should make it easy to identify. And if you want the VM tree to be in a non-default save location, then this is also the page where you can point the VM settings to that location.
The next step after naming the virtualization instance will be specifying the ‘Generation’ of the Hyper-V environment. Unless you have a specific use case requirement, I would suggest using the Generation-II virtual-machines. Especially if you are going to utilize any UEFI or secure boot options. (Which are requirements for most routing virtual machines)
Next we will assign the amount of RAM we want dedicated to the virtual-machine instance. As I intend to perform nested virtualization with this, I am going to give the guest OS all of the RAM I consider “spare”. Which is anything over 12GB, which I reserve for my host system. the additional 20GB of RAM is dedicated to the VM. And the default for physical CPU cores assigned is 50% of your existing physical core count. Which in the case of this setup, is 10 physical cores and 20 vCPU when using hyper-threading. If you want internet access of your QEMU environment, please make sure that you attach the virtual machine to the ‘default switch’ in the next page. It uses a class B IP space, which is not configurable. Any VM you attach to the ‘default-switch’ will vend a DHCP address with the class ‘b’ 172.16.0.0/10 scope. And it will be NAT’d to the existing host internet connection. Providing that the host system does have a current active internet connection.
You can always modify the assigned CPU cores configuration after creating the virtual-machine. As long as the virtual-machine is in a shutdown state and not actively running. To my knowledge, only the VM generation setting is immutable. Everything else is configurable. With the exception of the default-switch DHCP pool.
Windows Server Hyper-V and Virtualization – Training | Microsoft Learn
Now that the networking is taken care of, we just have the virtual-hard-disk and OS image to line out now! You will need to have downloaded your target guest OS already at this point and you can associate it with the instance to create your QEMU environment in your hypervisor.
Please be sure to have all of your qcow2/qemu images that you want to access from the guest OS, put onto a thumb drive, external drive. Or any drive where you have only the files you NEED access to, to create a new disk for the virtual-machine, so that it can access the files as a secondary mounted disk.
Beginners Guide: Shared Folders in Hyper-V – BDRSuite
Create the hard drive name and specify the location that you want to store it. It’s best if you set it explicitly from my experience. So that you don’t have to hunt for it in the system folders down the road when you need to manipulate the files.
Next, you will need to point to the QEMU capable OS that you are going to run in the virtual machine and map the VM to the installation disk image.
!!! NOTE !!! – If you enable ‘secure boot’ and/or TPM functionality, you will need to delete the attached DVD, and create a new one, so that the drive will generate a hash compatible with the increased security capabilities. If you forget, you will have issues getting the OS image to boot. Then come back and read this note again! 😉 !!! NOTE !!!
Now we can complete the virtual-machine setup and we will see it show up in our list of virtual machines in the Hyper-V Manager (That you should have opened as an administrator.)
Just click ‘Finish’ and you will see the virtual machine listed in your Hyper-V Manager window, under “Virtual Machines”. Before we spin it up for the first time, lets run the powershell command to enable nested virtualization on the VM when we spin it up.
Set-VMProcessor -VMName "Ubuntu223" -ExposeVirtualizationExtensions $true
Spinning up and Configuring your QEMU Host!
Now lets ‘START’ the virtual-machine and get the operating system loaded! Providing everything is as it should be, then when you connect to the started virtual machine, you should end up at the installation screen for your operating system off choice.
I was going to cover setting up Ubuntu for virtualization as well, but this post is getting pretty long as it is. I will start working on the Ubuntu VM configuration to spin up a developer ‘lite-mode’ vjunos image in the nested virtualization environment tomorrow. So expect it soon!
Resource Links Related to the Material:
- How To Use PowerShell To Fix Hyper-V Virtual Disk Chains — Redmondmag.com
- Beginners Guide: Shared Folders in Hyper-V – BDRSuite
- How to Install Hyper-V on Windows 11 Home (makeuseof.com)
- System requirements for Hyper-V on Windows Server | Microsoft Learn
I am a huge fan of spinning something up and fiddling with it to get more comfortable or learn chassis, protocols and services better. And the ability to do QEMU within the Windows environment that I have entrenched myself in so heavily, opens a lot of options that I didn’t have previously. I hope that you can think of uses for it as well!
One thought on “Hyper-V Nested Virtualization Setup”