Like VMware Workstation, VirtualBox is a type-2 hypervisor, which requires a host OS as a base. This type is particularly suitable for desktop use because of its user-friendly nature and tight integration with the underlying operating system. Such tools support copying files to and from the host via drag and drop and can display individual applications on the host desktop via Seamless Mode.
Using VirtualBox on the Server
However, VirtualBox is also suitable in scenarios typical for a type-1 hypervisor like ESXi or Hyper-V. Such a system usually runs on a server that is managed remotely, requiring a remote display protocol in the VM. In these cases, Oracle VirtualBox is operated in headless mode because the GUI of the guest OS is usually not needed on the server.
Let’s first get the names with their UUID of the VMs because they are required for the various commands to control the VM:
VBoxManage list vms
To start a VM named Windows Server 2022 without displaying it locally, enter the following command:
VBoxHeadless -s "Windows Server 2022"
Alternatively, you can also use the following command:
VBoxManage startvm "Windows Server 2022" --type headless
The same result can be achieved in the graphical console via Start > Start without GUI.
Install RDP Server
To transfer the guest operating system console from the remote VM to the client device, Oracle provides the VirtualBox Remote Display Protocol (VRDP) and the VirtualBox Remote Display Extension (VRDE). You have to download and install them separately.
The RDP server offers the advantage of accessing remote VMs without enabling the Remote Desktop feature in the guest OS. It doesn't even need to be supported by the operating system. This mechanism is comparable to vmconnect in Hyper-V. VRDP is compatible with most RDP clients, including mstsc.exe on Windows.
Activate VRDE and configure ports
To activate the VRDP extension, switch to the installation directory of VirtualBox and call up the command-line tool VBoxManage.exe, which, in the following example, enables RDP access to a VM named Windows Server 2022:
VboxManage modifyvm "Windows Server 2022" --vrde on
The VRDP server can be turned off for the same VM with this command:
VBoxHeadless --startvm "Windows Server 2022" --vrde off
By default, the VRDP server in VirtualBox listens for incoming connections on TCP port 3389. You can change this as follows when the VM is not running:
VBoxManage modifyvm "Windows Server 2022" --vrde-port 5000,5010-5012
This is necessary, for example, if RDP is already activated on the server, as TCP 3389 is also the default here. The above command specifies that the VRDE server uses one of the ports from the list, in this example, 5000 or a port between 5010 and 5012.
The settings of the VRDP server can also be adjusted on the Remote Control tab in the VM settings under Display.
Change the IP address of the VRDP Server
To display information about the started VM, including those about the VRDP server, use the following command:
VBoxManage showvminfo "VM Name"
The IP address and the port used by the VRDP server can be found in the results under the VRDE property. To change its IP address, enter the following command:
VBoxManage modifyvm "VM Name" --vrdeaddress 10.0.20.10
Connecting to the VRDP Server, redirecting USB devices
In RDP clients, you do not use the IP address of the guest operating system but that of the VRDP server. If you have changed the port, you must specify it accordingly here.
The VRDP server can also access the USB devices of the client computer so that they are available to the user in the guest OS. The client being used must, of course, support this redirection.
Unlike a direct RDP connection with the guest OS, the VRDP server allows multiple simultaneous Windows VM access.
VirtualBox is not only usable on a workstation where users interact directly with the console of the guest operating system. It is also suitable for server deployment.
Subscribe to 4sysops newsletter!
In this case, you typically don’t want to display the screen of the guest OS locally but rather remotely on a client device. VirtualBox offers the headless mode of VMs for these usage scenarios, which allows remote access via the integrated VRDP server. The main advantage is that the operating systems in the VMs don’t have to be configured for remote access because the hypervisor manages the connection.