Getting dnsmasq, virtual machines (libvirt), and vpn to inter-operate

Fixing Dnsmasq to Work With Libvirt

There's a bug written in the Ubuntu tracker (and in Fedora too, I think) about running dnsmasq and libvirt at the same time. Basically, libvirt runs an instance of dnsmasq, and if you install the system dnsmasq it tries to bind to all interfaces by default. The "easy" workaround is to use the interface and bind-interfaces configuration options in dnsmasq.conf. On my machine, I've got:

interface=eth0
interface=br0
bind-interfaces

This makes it so that the system dnsmasq only listens on eth0 and br0, and does not bind to the wildcard address. Now you can run two instances of dnsmasq: the system instance, and libvirt's instance.

Getting The System Dnsmasq to Answer for the VMs

I wish I knew the answer for this: I'd like for my system dnsmasq to respond to queries for "fedora16" and "winxp" (hostnames of two VMs I use). I tried a couple of things that didn't work, and settled for a subpar solution.

First, I thought I might be able to use a fake domain ".vm" and tell the system dnsmasq to forward queries for that domain to libvirt's dnsmasq. This may be a workable solution; it didn't work when I tried it, but I was fighting other issues with libvirt at the time and I haven't had a chance to go back and try it again. The problem I ran into is that libvirt's dnsmasq was receiving queries for "fedora16.vm" but it didn't have ".vm" configured as a local domain. You can't monkey directly with libvirt's dnsmasq config (at least not that I saw), or this would be an easy fix. I tried changing the names assigned in the libvirt XML file dealing with DHCP names and addresses, but didn't get anywhere. Again, more experimentation here might win the day. (Definitely let me know if you know how to solve this.)

I also flailed around with some other config options trying to get those names forwarded to libvirt's dnamasq but couldn't come up with anything clean.

What I finally settled on is adding the names of the VMs to my local host file:

# Virtual Machines; this should match up with the DHCP assignments in
# /etc/libvirt/qemu/networks/default.xml
192.168.122.22 fedora16.home.bstpierre.org fedora16
192.168.122.23 winxp1.home.bstpierre.org winxp1
# etc...

This file is referenced by dnsmasq.conf via addn-hosts=/etc/dnsmasq.hosts. It's not perfect, since I have to remember to enter the addresses in two places (default.xml and dnsmasq.hosts) when I create a new VM, but I don't do that often so it won't be a huge burden.

Getting Names from the VPN

When I connect to $DAYJOB VPN, I need to be able resolve names for that domain from the corporate (internal) DNS servers. Dnsmasq has a config option that allows you to tie a certain domain to a certain set of DNS servers:

server=/example.com/172.18.1.1
server=/example.com/172.18.12.1
address=/vpnserver.example.com/169.254.123.210

The first two lines set the DNS servers when I'm on the VPN. The last line makes it so that I have the public address for the VPN server when I'm not yet connected -- since the domain matches, dnsmasq will try to forward the request to the internal DNS servers, which is guaranteed not to work. This is slightly clumsy since the address is hardcoded, but it doesn't change in practice and I can always query it directly from upstream DNS (e.g. dig vpnserver.example.com @192.168.1.1 to get it from my router).

Finally, I don't want to be making requests to those 172.18 addresses when I'm not on the VPN, so I set up this interface definition in /etc/network/interfaces:

iface tun0 inet manual
pre-up ip route delete 172.18.0.0/16 || true
pre-up service openvpn start || true
post-down service openvpn stop
post-down ip route add blackhole 172.18.0.0/16 || true

After the VPN is taken down, this will add a blackhole route for 172.18.0.0/16, which is where those DNS servers live. Just before the VPN is started, it removes the blackhole route so that I can get to those servers once the VPN is ready.

Posted on 2012-05-15 by brian in network .
Comments on this post are closed. If you have something to share, please send me email.