Help needed with Tinc Setup on remote hosts and lots of ISPs / Failover Problems between ISPs

[Guus Sliepen]

On Sat, Mar 21, 2015 at 01:01:47PM +0100, Raimund Sacherer wrote:

> This is our setup which we are trying in a couple of our remote offices:
[...]
> FW-01 and FW-02 are Master/Slave firewalls (pfSense with Carp failover). We have currently 3 "remote offices" connected which have all basically the same setup, one office has more internet lines. 
> 
> Currently I have it configured this way:
> 
> "Remote Office 1", Tinc A: ConnectTo ISP-A and ConnectTo ISP-B over ISP-D, Tinc B: the same, ISP-E not used
> "Remote Office 2", Tinc A: ConnectTo ISP-A and ConnectTo ISP-B over ISP-D, Tinc B: the same, ISP-E not used
> "Remote Office 3", still not set up, here I have 5 Internet lines (as described below) and I am not sure how to set it up correctly.
> 
> Can you recommend me the correct or best-practice way to connect all those offices over their different connections to maintain a stable VPN system in which an outage or degradation of one ISP does not effect the network overall? Remote offices do not talk much between each other, the main communication is between remote office and head quarter, but that might change in the future. 

The main thing is that you don't need to have three ConnectTo's on Tinc
A and Tinc B, just one: ConnectTo headquarters. What you do want is to add three
Addresses in hosts/headquarters, assuming you got a different IP address from each
ISP at the headquarters. Then, when tinc A starts, it will try to
connect to the headquarters by selecting one of the given Addresses. If
that fails, it will go to the next one, and so on. If a working
connection suddenly fails, tinc will try to reconnect using a different
Address.

> A) How to connect correctly Tinc A and Tinc B with Tinc VM?
> Would it be a good design to interconnect like that: 
> 
> Tinc A via ISP-D with ConnectTo ISP-A, ISP-B, ISP-C
> Tinc B via ISP-E with ConnectTo ISP-A, ISP-B, ISP-C
> Tinc A having a higher priority than Tinc B

So this would be:

Tinc A via ISP-D with ConnectTo HQ, hosts/HQ contains Addresses for ISP-A, ISP-B and ISP-C
Tinc B via ISP-D with ConnectTo HQ, same as above.

If tinc A and tinc B are connected to the same internal network, and you
want to favor A over B, then, assuming you are using tinc in router
mode, you have the same Subnet(s) in hosts/A and hosts/B, but with a
lower weight for those in hosts/A than those in hosts/B.

> B) What if I have a remote office where I have (because of reliability problems) more Internet Connectivity?
> In Remote Office 3 I have 3 ADSL, 2 2Mbit SDSL and 1 FTTH connection, if I have the two tinc daemons on two firewalls, how would I interconnect those? I would like at least the FTTH, one SDSL and one ADSL line to be in the VPN setup. But I only have 2 tincd's on the two firewalls.

That's quite a complicated setup. I also assume you want to use the FTTH
connection whenever possible. One tinc daemon itself really isn't made
to select which path is best, it normally just tries connecting until
something works, and then sticks to it. You could do something if you
have one tinc daemon per ISP connection, but that is apparently not what
you want. If possible, see if you can have the firewall select the best
route to take, then tinc will just use that.

> C) I had a problem with ISP-A having a high packet loss in the direction from ISP-A to ISP-D. ISP-D to ISP-A was fine. Tinc did not switch from sending the VPN over ISP-A to ISP-B and I had communication problems until ISP-A had corrected their packet loss problem. 
> 
> Any Idea what went wrong? I tried to eliminate the ISP-A from the tinc config files in "Remote Office 1", restarted the tincd but it kept showing up and kept being elected as VPN Path. I later found out I had ISP-A also configured in "Remote Office 2" (which I thought I had not). 

Tinc does not check all possible communication paths for quality. It
tries available paths in succession until it finds one that works, and
sticks with it until it fails completely.

Tinc also learns IP addresses from other nodes. Or it could be that even
though tinc at ISP-D was sending packets to headquarters via ISP-B or
ISP-C, that headquarters was sending them back via ISP-A, causing the
tinc daemon at ISP-D to switch to ISP-A anyway.

If you want/need tighter control, you might have to run three tinc
daemons at the headquarters, one for each ISP you have there.

> I want to thank the developers and supporters of Tinc and I think it is a great piece of software and I am looking forward to expand it to be our only VPN Solution in the future as we are currently opening about 2-3 remote offices around the globe every year. 

You're welcome :) I hope the answers above help you.

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://www.tinc-vpn.org/pipermail/tinc/attachments/20150331/d8b099a1/attachment.sig>