KVM VM traffic over host's Tinc VPN

Eric Wiggins ewiggi2 at lsu.edu
Wed Aug 15 22:12:37 CEST 2012


Hello Tinc list!

I'm trying to set up a Tinc VPN between two KVM host machines so that a 
VM on one host can communicate with a VM on the other host. While I do 
have a good bit of experience with virtualization, I'm not a 
particularly savvy network guy, so this is proving to be a pretty big 
challenge.

Requirements:

* ALL VM network traffic must be secure.
* VMs on one host must be able to communicate with VMs on other hosts.
* As I'm using another group's images for the VMs and will have no 
control over the VMs once they're up and running, all configuration 
needs to happen on the hosts and be invisible to the VMs.

My test setup:

* Two RHEL6.2 hosts, each running KVM with one VM set up on each host.
* Tinc set up on both hosts.

Configurations for each host:

=====
=Host1=
=====

tinc.conf:

         Name = host1
         ConnectTo = host2

tinc-up:

         #!/bin/sh
         ifconfig $INTERFACE 10.90.41.241 netmask 255.255.252.0

hosts/host1:

         Address = host1.my.domain
         Subnet = 10.90.41.241
         Port = 655

         -----BEGIN RSA PUBLIC KEY-----
                     keygibberrish
         -----END RSA PUBLIC KEY-----

hosts/host2:

         Address = host2.my.domain
         Subnet = 10.90.42.242/32
         Port = 655

         -----BEGIN RSA PUBLIC KEY-----
                     keygibberrish
         -----END RSA PUBLIC KEY-----

ifconfig results for VPN:

test    Link encap:UNSPEC  HWaddr 
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
           inet addr:10.90.41.241  P-t-P:10.90.41.241  Mask:255.255.252.0
           UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
           RX packets:5 errors:0 dropped:0 overruns:0 frame:0
           TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:420 (420.0 b)  TX bytes:420 (420.0 b)



=====
=Host2=
=====

tinc.conf:

         Name = host1

tinc-up:

         #!/bin/sh
         ifconfig $INTERFACE 10.90.42.242 netmask 255.255.252.0

hosts/host1:

         Address = host1.my.domain
         Subnet = 10.90.41.241
         Port = 655

         -----BEGIN RSA PUBLIC KEY-----
                     keygibberish
         -----END RSA PUBLIC KEY-----


hosts/host2:

         Address = host2.my.domain
         Subnet = 10.90.42.242/32
         Port = 655

         -----BEGIN RSA PUBLIC KEY-----
                     keygibberrish
         -----END RSA PUBLIC KEY-----

ifconfig results for VPN:

test    Link encap:UNSPEC  HWaddr 
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
           inet addr:10.90.42.242  P-t-P:10.90.42.242  Mask:255.255.252.0
           UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
           RX packets:5 errors:0 dropped:0 overruns:0 frame:0
           TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:420 (420.0 b)  TX bytes:420 (420.0 b)


At this point, Tinc seems to work. Pings from host1 to 10.90.42.242 are 
replied to, and pings from host2 to 10.90.41.241 are replied to.

Now to set up networking for the VMs...

My first thought was to simply bridge the VM connection to the VPN 
interface. So, in virt-manager, I went into the details tab of my VM on 
host1, selected the NIC, and chose "Host device test : macvtap" as the 
source device and "bridge" for the source mode. Cranked up the vm and 
got: "Error starting domain: error creating macvtap type of interface: 
Invalid argument."

So I tried to manually create the bridge and add the "test" device to it.

         [root at host1 test]# brctl addbr br0
         [root at host1 test]# brctl addif br0 test
         can't add test to bridge br0: Invalid argument

I did some more research on bridges, and decided that maybe I needed to 
specify the deviceType and interface in tinc.conf. So I changed my 
tinc.confs:

host1 tinc.conf:

         Name = host1
         DeviceType = tun
         Interface = tun0
         ConnectTo = host2

host2 tinc.conf:

         Name = host2
         DeviceType = tun
         Interface = tun0

Restarted Tincd on both hosts and tried my pings again. They worked, so 
I tried to bridge the new tun0 device.

         [root at host1 test]# brctl addif br0 tun0
         can't add tun0 to bridge br0: Invalid argument

No dice, again. So I tried to specify as a tap device in tinc.conf:

host1 tinc.conf:

         Name = host1
         DeviceType = tap
         Interface = tap0
         ConnectTo = host2

host2 tinc.conf:

         Name = host2
         DeviceType = tap
         Interface = tap0

Restarted tincd and it cried about /dev/tap0 not existing. So I made it:

         mknod /dev/tap0 c 36 16

Restarted tincd and tried my pings again. They went unanswered. =\ Ran a 
tracepath on the IP and got:

         [root at host1 test]# tracepath 10.90.42.242
          1:  10.90.41.241 (10.90.41.241)                            
0.123ms pmtu 1500
          1:  10.90.42.242 (10.90.42.242)                            
0.524ms pmtu 1445
          1:  no reply
          2:  no reply
          3:  no reply

So it looks like the ping is actually getting from host1 to host2, but 
host2 doesn't realize it's there. WTH?

Just for fun I tried to bridge the tap0 interface, and it worked:

         [root at host1 test]# brctl addif br0 tap0
         [root at host1 test]# brctl show
         bridge name    bridge id                        STP enabled    
interfaces
         br0                   8000.120ab67c44bd     no                
     tap0

I found that curious, so checked the ifconfig for tap0 and noticed that 
the Link encap was now defined as "Ethernet" where it was "UNSPEC" 
before. I can only assume that's why I wasn't able to bridge the VPN 
earlier.

tap0   Link encap:Ethernet  HWaddr 12:0A:B6:7C:44:BD
           inet addr:10.90.41.241  Bcast:10.90.43.255  Mask:255.255.252.0
           inet6 addr: fe80::100a:b6ff:fe7c:44bd/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:17 errors:0 dropped:0 overruns:0 frame:0
           TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:10905 (10.6 KiB)  TX bytes:1224 (1.1 KiB)

So that's where I'm at now. It seems that using a tap is getting me 95% 
of the way there. Google is not providing me with any more useful 
suggestions, so I come to you, the members of this list, with the 
following question:

Is there a way to configure Tinc to accomplish what I'm trying to do, or 
do I need to try to find some other solution?

Thanks for your time,

Eric



More information about the tinc mailing list