Virtualization Security Tips: Preventing Hyper Jumping
Just when you thought your virtualized environment was secure, here comes a very real threat. Virtual machine jumping, also know as hyper jumping, can be used to gain access to other hosts. Here's the lowdown on VM jumping and how to protect your virtual infrastructure against it.
What is VM Jumping or Hyper Jumping?
Virtual machine jumping, also know as hyper jumping, is an attack that is especially suited to virtualized environments due to the density of machines on a host or cluster. In a similar way to how human viruses jump from host to host, hyper jumping exploits a weakness in a virtual machine to allow it to be used as a platform to launch attacks against other machines on the same infrastructure. It can be protected against, but at a cost of significant management overhead.
Almost every application in use today can exist on a virtualized platform. This can lead to huge efficiency savings in terms of power, cooling and floor space. But virtualization's downside is that it effectively puts an administrator's eggs into one huge basket. N+1 hardware design and high availability may save the administrator from prolonged hardware failures, but what about failures in security resulting from and surrounding the hypervisor and guests?
Hypervisors will become a very attractive target for hackers due to the number of potential guest machines that can be compromised and also the sheer power of virtualization servers. In some configurations it is typical to see upwards of 60 cores coupled with half a terabyte of RAM per host, and a cluster with more than 10 hosts. These are exceptionally powerful and valuable servers. Read: Virtualization Admin's Guide to Using Both Hyper-V and VMware
There are two general scenarios for virtualization compromise, one proven, and one theoretical. In 2006 researcher Joanna Rutkowska developed a proof of concept attack called Hyperjacking, and it was designed to compromise the hypervisor itself. It was initially an exploit that appeared in the Pacifica, and later Vanderpool chipsets and it was essentially designed to virtualize the hypervisor. The attack was named blue pill, and the cure was called red pill, after scenes from The Matrix (read more about Blue Pill).
However, after much investigation and discussion this allegedly undetectable conceptual attack was proven to be pretty much unusable and not invulnerable to detection. The conceptual threat has since raised its head from time to time, but there is no known exploit based on it.
The second series of exploits involves the use of "virtual machine jumping." VM jumping exploits are designed, as the name would infer, to allow a guest virtual machine to be compromised and exploited to access other virtual machines.
Virtual Machine (VM) Jumping aka Hyper Jumping
Insecure operating systems are the biggest part of the issue. Previous versions of Windows are a security nightmare due to the lack of modern security features such as memory address layout randomization, poison cookies and a hardened stack. This makes them quite vulnerable to various attack methodologies.
Once an exploit has been developed and executed, it gains access to the virtual network. At this point it could be used to launch an exploit to attack other machines. Jason Joel previously blogged about this on the VMware blog, albeit only in a workstation environment. To make the attack vector even more powerful, a compromised guest can exploit a poorly secured virtual environment and gain access to a large collection of networks. Matt Conovor from Symantec also covered this subject at Black Hat 2009 with a presentation on inserting exploits into virtual machines (see PDF of the presentation here).
In most virtualization platforms, all the traffic from virtual machines flows to and from the external network using a layer two bridge. This means that all traffic flows over the same set of NICs and the traffic is not separated by anything other than VLAN tagging, if used. There have been a number of exploits that have allowed VLAN jumping. One typical attack methodology was to overload the attached switch. When overloaded, the switch would push all packets out on all ports to preserve performance, effectively turning it into a dumb hub and losing any security the switching afforded.
Quite a few environments run VLAN TRUNK ALL on the uplinks for convenience. In simple terms, this means all VLANS are sent down the network, regardless of whether they are actually needed; it is just laziness, and administrators are needlessly exposing a lot more of their estate to a possible compromise.
Some of the rudimentary management on virtual switches will allow you to drop forged packets and MAC address changes, but an ounce of prevention is worth a pound of cure as the old saying goes.
Prevent Hyper Jumping: Tips to Secure Your Virtual Network
There are additional ways to help prevent VM/hyper jumping. The easiest and most common way is to use separate groups of physical uplinks for groups of similar VLANs. This essentially means keeping web facing traffic separate from the database traffic, for example. In other words, keep the traffic in physically separate network tiers so that the database server will never be able to directly talk to the internal network.
If an administrator wishes to take the security aspect further, they could use Private VLANs. PVLANs are like VLANS but they have a unique feature: they allow the guest to only talk to the gateway, in effect hiding the rest of the machines from seeing each other. It is a somewhat cumbersome feature but one that is very useful.
On the host, you can use a separate management network. By default most virtualization products install the management network and the "Virtual Machines" port group on the same uplink. This is an extremely bad default setting because you're allowing your management traffic and VM traffic to intermingle.
The same principle applies to IP based storage networks as well. Using VLANs will, to a degree, help mitigate the risk. Having separate, redundant uplinks for management, storage and VMs is strongly advised and will mean that you are not exposing your management infrastructure to all the virtual machine traffic. The other gain is that you can apply more restrictive rules to the networks as required.
On the hypervisor front we strongly advise you to use the built-in firewall (most hypervisor platforms have them) to restrict access to the management network and also keep the uplinks separate using appropriate techniques (routing and firewalls).
If you are interested in additional changes that can be made to your VMware farm, review the VMware security hardening guides. Read what each setting does before applying it. Some of these changes will remove functionality and should be used with care.