Tinc & moving VMs accross network

[Etienne Dechamps]

On 7 December 2015 at 17:20, Florent B <florent at coppint.com> wrote:
> I have a cluster of 5 nodes, running Proxmox 4, and Tinc as "virtual
> switch" for my nodes : on each node, a bridge "vmbr1" where Tinc is
> connected, provides me a secured network for my VMs (connected to that
> bridge).
>
> When I move (hot move) a VM from a host to another, I have to restart
> Tinc on node from where VM is coming to get access to that VM again.

I don't use tinc in Switch mode, but looking at the code (especially
the learn_mac() function) it looks like tinc is not really able to
migrate MAC addresses from one node to another in the way you'd
expect. It will add the MAC address to the new node, but will not
remove it from the other. Until the address expires on the old node
(or you restart it), I suspect packets will go to the first node in
lexicographical name order (because that's just what
subnet_compare_mac happens to use). After the MAC expires on the old
node (10 minutes by default), things will go back to normal.

You can verify that this is indeed what's causing your problem by
setting MACExpire to a very low value. If you set it to 5 seconds,
then it should only take 5 seconds for the MAC to move completely from
the old node to the new one. That could be a solution for you; in fact
I suspect we could change the MACExpire default, because 10 minutes
sure is a really long time. Even better would be to make tinc smarter
about migrating MAC addresses; for example by automatically overriding
the old subnet entry when the new one appears.

You can also use "tinc dump subnets" to see the current state of
tinc's internal routing table, that should help shed more light on the
issue.