Switch mode three-node routing problem

[Микола Дімура]

I've been investigating the problem with packet loss and duplication,
when tinc is in switch mode and there are more then 2 nodes attached.
I've stumbled upon a bug report on http://www.open-mesh.org/issues/170
. There person has very similar problem:
"received packet on ... with own address as source address" error when
more then two nodes are connected to network.
There the solution was a patch related to distributed_arp_table. Maybe
this will help with troubleshooting?

Once again about my configuration:
There are three nodes, each is a router (two openwrt, one asuswrt
Merlin). Nodes are connected to the Internet via different ISP each.
One node has real ipv4 address, two nodes are behind the NAT. On every
node I install tinc and for each node I specify "Mode = switch". I
also had to set "TCPonly = yes" on one of the nodes, because otherwise
the connection was unstable (bandwidth dropped down to 0 kb/s for an
interval of couple of seconds few times per minute). I bridge
tinc-interface to LAN on every router. Netmask is set to 255.255.0.0
for all nodes. I also tried to set "IndirectData = yes" on NAT-nodes.

When two nodes are on-line - everything works perfect. I tried
combinations of Node1 + Node2 or Node1 + Node3 - both combinations
work perfectly. However, when I turn on the third node - duplication
and packet loss appear!

Does anyone have successful experience setting up 3+ nodes in the
setup similar to mine? Could you please share?


Best regards


2014/1/12 Микола Дімура <burning.daylight at p-s.org.ua>:
> I did the following:
> 1) Started tcpdump -i pvpn > pvpn-no_roadwarrior.dump on
> Router2(192.168.0.1)
> 2) started ping 192.168.0.11 from Router1(192.168.10.1). No packet loss or
> duplication was observed. 192.168.0.11 is behind Router2.
> 3) killed tcpdump.
> 4) Repeated 1-3 with Roadwarrior up. There was no packet loss, but there
> were duplicated packets.
> 5) Started ping from Router2 to 192.168.10.100 (reverse direction):
> --- 192.168.10.100 ping statistics ---
> 22 packets transmitted, 15 packets received, 15 duplicates, 31% packet loss
> round-trip min/avg/max = 2.074/22.148/48.774 ms
> Dumps from 1) and 4) are attached.
>
> Pings to/from Roadwarrior show similar result: packet loss and duplication.
> Except from the case when I ping Router2(192.168.0.1) from Roadwarrior or
> vice versa. This particular pair of machines does not show packet loss or
> duplication.
>
> Router1, Router2 and Roadwarrior are all connected to the Internet via
> different ISP each. So no, Roadwarrior is not connected to LAN of Router1/2.
> Router2 and Roadwarrior do not have real ipv4 address, each of them is
> behind the NAT of its respective ISP.
> I've enabled TCPOnly on Router2 because elsewise traffic bettween Router1
> and Router2 is very intermittent, I've found this out empirically. I'm not
> sure, what is the reason for this.
> Best regards, Mykola Dimura.
>
>
>
> On 10 January 2014 23:42, Guus Sliepen <guus at tinc-vpn.org> wrote:
>> On Fri, Jan 10, 2014 at 05:50:57PM +0100, Микола Дімура wrote:
>>
>>> I am using tinc in switch mode. I have three nodes. Two nodes reside on
>>> routers, vpn-eth is bridged with internal lan, each router has several
>>> machines connected to it's internal lan. Third node is the roadwarrior -
>>> "endpoint" linux PC.
>>> When the roadwarrior is off - everything works perfectly, machines on
>>> both
>>> sides can communicate without a problem in any direction. When I turn on
>>> the roadwarrior - everything goes crazy: packet loss becomes huge (90%)
>>> even for machines, which worked perfectly before, ping shows duplicate
>>> packets. When I turn off the roadwarrior again - everything returns to
>>> the
>>> working state.
>>
>> That is strange indeed. Could you run tcpdump on the pvpn interface on one
>> of
>> the routers, then run ping both when the roadwarrior is online and not,
>> and
>> send me the results?
>>
>>> Router1 and Roadwarrior3 are using tinc 1.0.23 version. Router2 uses
>>> v1.0.19 because newer is not present in the repository :(.
>>
>> That should not be a problem.
>>
>>> $ cat hosts/karl_router
>>> TCPonly = yes
>>
>> Why did you enable TCPonly here?
>>
>>> Roadwarrior3: $ cat /etc/tinc/pvpn/tinc.conf
>>> Name = daniel_mpc
>>> Mode = switch
>>> ConnectTo = central_router
>>>
>>> Roadwarrior3: $ cat /etc/tinc/pvpn/tinc-up
>>> #!/bin/sh
>>> ifconfig $INTERFACE 192.168.3.150 netmask 255.255.0.0
>>> ifconfig $INTERFACE up
>>
>> Are you trying out the road warrior while it is connected to the LAN of
>> one of
>> the two routers?
>>
>> --
>> Met vriendelijke groet / with kind regards,
>>      Guus Sliepen <guus at tinc-vpn.org>
>>
>> _______________________________________________
>> tinc mailing list
>> tinc at tinc-vpn.org
>> http://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
> 2014/1/10 Guus Sliepen <guus at tinc-vpn.org>
>>
>> On Fri, Jan 10, 2014 at 05:50:57PM +0100, Микола Дімура wrote:
>>
>> > I am using tinc in switch mode. I have three nodes. Two nodes reside on
>> > routers, vpn-eth is bridged with internal lan, each router has several
>> > machines connected to it's internal lan. Third node is the roadwarrior -
>> > "endpoint" linux PC.
>> > When the roadwarrior is off - everything works perfectly, machines on
>> > both
>> > sides can communicate without a problem in any direction. When I turn on
>> > the roadwarrior - everything goes crazy: packet loss becomes huge (90%)
>> > even for machines, which worked perfectly before, ping shows duplicate
>> > packets. When I turn off the roadwarrior again - everything returns to
>> > the
>> > working state.
>>
>> That is strange indeed. Could you run tcpdump on the pvpn interface on one
>> of
>> the routers, then run ping both when the roadwarrior is online and not,
>> and
>> send me the results?
>>
>> > Router1 and Roadwarrior3 are using tinc 1.0.23 version. Router2 uses
>> > v1.0.19 because newer is not present in the repository :(.
>>
>> That should not be a problem.
>>
>> > $ cat hosts/karl_router
>> > TCPonly = yes
>>
>> Why did you enable TCPonly here?
>>
>> > Roadwarrior3: $ cat /etc/tinc/pvpn/tinc.conf
>> > Name = daniel_mpc
>> > Mode = switch
>> > ConnectTo = central_router
>> >
>> > Roadwarrior3: $ cat /etc/tinc/pvpn/tinc-up
>> > #!/bin/sh
>> > ifconfig $INTERFACE 192.168.3.150 netmask 255.255.0.0
>> > ifconfig $INTERFACE up
>>
>> Are you trying out the road warrior while it is connected to the LAN of
>> one of
>> the two routers?
>>
>> --
>> Met vriendelijke groet / with kind regards,
>>      Guus Sliepen <guus at tinc-vpn.org>
>>
>> _______________________________________________
>> tinc mailing list
>> tinc at tinc-vpn.org
>> http://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>>
>