tinc generating invalid packet checksums?

[Nathan Stratton Treadway]

On Thu, Sep 10, 2015 at 19:34:21 -0400, Nathan Stratton Treadway wrote:
> It seems that when the cksum is incorrect, it is always off by 1.
> 
[...]
> Am I correct in concluding that this cksum problem is a bug in Tinc? 

After investigating this further, I'm fairly certain that problem
originates in the following lines of the clamp_mss() function in
route.c:

     [...]
     csum ^= 0xffff;
     csum -= oldmss;
     csum += newmss;
     csum ^= 0xffff;
     packet->data[50] = csum >> 8;
     packet->data[51] = csum & 0xff;

Since the TCP checksum value needs to be computed using one's compliment
arithmetic, the above code generates new values that are off by one from
the correct checksum for the packet in cases where the calculations
wrap around zero in one direction but not the other....

According to Eqn 3 given in RFC 1624 (e.g. at
  https://tools.ietf.org/html/rfc1624#section-3
) and the UpdateTTL() example function given RFC 1141,
it looks like the calculation should instead be something like
     uint32_t csum = packet->data[50] << 8 | packet->data[51];
     [...]
     csum ^= 0xffff;
     csum += oldmss ^ 0xffff;
     csum += newmss;
     csum = (csum & 0xFFFF) + (csum >> 16);
     csum += (csum >> 16);
     csum ^= 0xffff;
     packet->data[50] = csum >> 8;
     packet->data[51] = csum & 0xff;


(I did some quick testing in Python which seems to confirm that this
algorithm works correctly [as in "generates the value tcpdump is
expecting"] for the packets I mentioned in my original email; I can
send the sample code if anyone is interested.)

Additional discussion of this topic can be found in the "Incremental
update of TCP Checksum" conversation at:
  https://lkml.org/lkml/2003/9/16/197


							Nathan


----------------------------------------------------------------------------
Nathan Stratton Treadway  -  nathanst at ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239