network bridge for libvirt host with a single live IP

The default network config for libvirt is simple and works for most basic use cases but there’s a number of use cases where you need more complex config like Adam outlined for a local bridged config.

I run a number of VMs on a hosted server on the internet and I’ve had on my ToDo list for some time to add a IPSEC site to site VPN but the default network doesn’t make that easy because libvirtd deals with the iptables networking including the NAT automagically.

The network config looks like this:

                    *-----*
192.168.100.0/24  --| Hyp |-(eth0)- internet
 (br0) VM net       *-----*

Create non routed network bridge
Initially create a basic network bridge and disable STP (spanning tree protocol). Note we don’t bind it with eth0 which is a public internet facing interface.

nmcli con add type bridge ifname br0
nmcli con modify bridge-br0 bridge.stp no

I then edited the /etc/sysconfig/network-scripts/ifcfg-bridge-br0 file and to add IP network config and adjust a few bits to get the following:

DEVICE=br0
STP=no
TYPE=Bridge
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPADDR=192.168.100.254
NETMASK=255.255.255.0
IPV6INIT=no
IPV6_FAILURE_FATAL=no
NAME=bridge-br0
UUID=
ONBOOT=yes

Once we’ve done that we can bring the bridge online and check the config looks OK:

ifup br0
nmcli c show
nmcli -f bridge con show bridge-br0
ip addr

Now we have a network bridge with an IP address you can now edit any VM configuration and reassign the virtual NICs to the new bridge and adjust the VM network config to the new subnet and assign static IPs to each VM, or configure dhcpd to assign IPs on the br0 interface. Once that’s done you should be able to ping the gateway (192.168.101.254) and have local network connectivity.

Once you’ve moved everything over you can delete the original libvirtd network config.

Outbound NATed networking
Using the traditional iptables.service (firewalld investigation is on my todo list) you can add a basic outbound NAT configuration which restores the last of the missing functionality with the following basic rule set which will NAT by masquerading the br0 network out through the public IP on eth0:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state NEW -p tcp -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -i eth0 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i br0 -o eth0 -j ACCEPT
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables-save > /etc/sysconfig/iptables
systemctl enable iptables.service
systemctl start iptables.service

With the network now under complete control of NetworkManager and a IP firewall/NAT configuration under control from a single point it now makes it easier to add things like IPSEC connections and IPv6 configuration both of which are next on the list.