Fixing Dnsmasq to Work With Libvirt
There's a bug written in the Ubuntu tracker (and in Fedora too, I think) about running dnsmasq and libvirt at the same time. Basically, libvirt runs an instance of dnsmasq, and if you install the system dnsmasq it tries to bind to all interfaces by default. The "easy" workaround is to use the interface and bind-interfaces configuration options in dnsmasq.conf. On my machine, I've got:
interface=eth0 interface=br0 bind-interfaces
This makes it so that the system dnsmasq only listens on eth0 and br0, and does not bind to the wildcard address. Now you can run two instances of dnsmasq: the system instance, and libvirt's instance.
Getting The System Dnsmasq to Answer for the VMs
I wish I knew the answer for this: I'd like for my system dnsmasq to respond to queries for "fedora16" and "winxp" (hostnames of two VMs I use). I tried a couple of things that didn't work, and settled for a subpar solution.
First, I thought I might be able to use a fake domain ".vm" and tell the system dnsmasq to forward queries for that domain to libvirt's dnsmasq. This may be a workable solution; it didn't work when I tried it, but I was fighting other issues with libvirt at the time and I haven't had a chance to go back and try it again. The problem I ran into is that libvirt's dnsmasq was receiving queries for "fedora16.vm" but it didn't have ".vm" configured as a local domain. You can't monkey directly with libvirt's dnsmasq config (at least not that I saw), or this would be an easy fix. I tried changing the names assigned in the libvirt XML file dealing with DHCP names and addresses, but didn't get anywhere. Again, more experimentation here might win the day. (Definitely let me know if you know how to solve this.)
I also flailed around with some other config options trying to get those names forwarded to libvirt's dnamasq but couldn't come up with anything clean.
What I finally settled on is adding the names of the VMs to my local host file:
# Virtual Machines; this should match up with the DHCP assignments in # /etc/libvirt/qemu/networks/default.xml 192.168.122.22 fedora16.home.bstpierre.org fedora16 192.168.122.23 winxp1.home.bstpierre.org winxp1 # etc...
This file is referenced by dnsmasq.conf via
addn-hosts=/etc/dnsmasq.hosts. It's not perfect, since I have to
remember to enter the addresses in two places (default.xml and
dnsmasq.hosts) when I create a new VM, but I don't do that often so it
won't be a huge burden.
Getting Names from the VPN
When I connect to $DAYJOB VPN, I need to be able resolve names for that domain from the corporate (internal) DNS servers. Dnsmasq has a config option that allows you to tie a certain domain to a certain set of DNS servers:
server=/example.com/172.18.1.1 server=/example.com/172.18.12.1 address=/vpnserver.example.com/169.254.123.210
The first two lines set the DNS servers when I'm on the VPN. The last
line makes it so that I have the public address for the VPN server when
I'm not yet connected -- since the domain matches, dnsmasq will try to
forward the request to the internal DNS servers, which is guaranteed not
to work. This is slightly clumsy since the address is hardcoded, but it
doesn't change in practice and I can always query it directly from
upstream DNS (e.g. dig vpnserver.example.com @192.168.1.1 to get it
from my router).
Finally, I don't want to be making requests to those 172.18 addresses when I'm not on the VPN, so I set up this interface definition in /etc/network/interfaces:
iface tun0 inet manual pre-up ip route delete 172.18.0.0/16 || true pre-up service openvpn start || true post-down service openvpn stop post-down ip route add blackhole 172.18.0.0/16 || true
After the VPN is taken down, this will add a blackhole route for 172.18.0.0/16, which is where those DNS servers live. Just before the VPN is started, it removes the blackhole route so that I can get to those servers once the VPN is ready.