Problem with large pings with don't fragment set

All,

I have been having a problem when experimenting with sending large
pings with the DF bit set over a GRE tunnel.

I can send pings with a size of upto about 1450 over the tunnel, but
when they get any larger then I have problems. I send the packet and
I get the ICMP reply about not being able to send the send the packet
as it is too large and needs fragmenting. However I then try and send
a smaller packet and I still get the same reply, but now it is saying
the MTU on the tunnel interface is 552. This means that while I
should be able to send packets which are say 1000 bytes I can't until
this problem times out after a few minutes.

I suspect the problem is related to how PMTU discovery works. But I
can't work out if it supposed to work like this or there is some sort
of a bug. It seems strange that the MTU size would drop just because
one larger packet was sent.

I am doing the ping with the following: ping -s 1480 -M do w.x.y.z,
then I change the packet size to 1000 and it still doesn't work, until
this timeout happens. The inferrence from the PMTU discovery RFC is
that the maximum value to use is 576, so this is why I suspect this
could be the problem, but it seems to be a strange way of handling
this situation. I can't take the PMTU discovery off the tunnel as I
am tunnelling OSPF through the tunnel and need the ttl to be larger
otherwise the TTL is exceeded and changing the TTL is incompatable
with nopmtudisc for some reason.

Has anyone had this sort of problem before, and do they know a
workaround to either reduce the timeout or elimate it totally, as it
is causing problems with my experiments waiting for this timeout.

Many thanks

Pete

Pete <pacalvert@qinetiq.com> wrote:

> I have been having a problem when experimenting with sending large
> pings with the DF bit set over a GRE tunnel.

> I can send pings with a size of upto about 1450 over the tunnel, but
> when they get any larger then I have problems. I send the packet and

The ping message header is 8 bytes, so the size of data in the message
is limited to 1472, assuming an Ethernet interface to the Internet with
an MTU of 1500. I don't know about GRE tunnels but they very likely add
more overhead.

> I get the ICMP reply about not being able to send the send the packet
> as it is too large and needs fragmenting. However I then try and send
> a smaller packet and I still get the same reply, but now it is saying
> the MTU on the tunnel interface is 552. This means that while I

The size is limited by both the Ethernet and GRE interface MTUs.
It sounds to me like the first time it's the Ethernet interface MTU
that causes the PMTU Discovery reply and the second time it's the
GRE interface MTU.

> should be able to send packets which are say 1000 bytes I can't until
> this problem times out after a few minutes.

I can't parse this.

> I suspect the problem is related to how PMTU discovery works. But I
> can't work out if it supposed to work like this or there is some sort
> of a bug. It seems strange that the MTU size would drop just because
> one larger packet was sent.

If the smaller message (1000 octets) passed through the Internet Ethernet
interface and afterwards encountered the GRE interface with the 552 MTU
then that should explain it.

> I am doing the ping with the following: ping -s 1480 -M do w.x.y.z,
> then I change the packet size to 1000 and it still doesn't work, until
> this timeout happens. The inferrence from the PMTU discovery RFC is
> that the maximum value to use is 576, so this is why I suspect this
> could be the problem, but it seems to be a strange way of handling
> this situation. I can't take the PMTU discovery off the tunnel as I

Read section 7.1 of RFC 1191.

> am tunnelling OSPF through the tunnel and need the ttl to be larger
> otherwise the TTL is exceeded and changing the TTL is incompatable
> with nopmtudisc for some reason.

> Has anyone had this sort of problem before, and do they know a
> workaround to either reduce the timeout or elimate it totally, as it
> is causing problems with my experiments waiting for this timeout.

The maximum ICMP echo-request data size you will be able to send through
the GRE tunnel without fragmenting is 552 - 28 = 524 octets.

Setting the MTU for the Ethernet interface of the host originating
the ping echo-requests to 552 should cause large echo-requests to be
broken into packets of this size before going out that interface.
This repacketization doesn't involve PMTU Discovery, but likely does
mean that PMTU Discovery is then unnecessary.

I'm *not* an expert on networking protocols, so beware.

--
Clifford Kite Email: "echo xvgr_yvahk-ccc@ri1.arg|rot13"
PPP-Q&A links, downloads: http://ckite.no-ip.net/