SO_SNDBUF too big

Hi,

I'm trying to make a file transfer server that transfers files very fast.
The problem is that for some reason it takes very long for the server to
notice if the connection to the client is (fysically) down, mean while it
keeps sending those packages until the tcp stacks send buf fills. Sometimes
this might tak as long as 15 minutes. I have set the following:


int MAX_TX_BUFFER = 4096;
f( setsockopt( sockfd, SOL_SOCKET, SO_SNDBUF, &MAX_TX_BUFFER, sizeof( int ))
== -1)
{
perror("setsockopt");
exit(1);
}

Now I know if that if I tune the MAX_TX_BUFFER down the time (or actually
the amount of packet sent) needed to notice that the network connection is
down diminished. However since I'm trying to optimise the transfer speed
and reliability to multiple connections tuning the MAX_TX_BUFFER size down
isn't an option. Has anyone any clues?

I've also tried to set KEEPALIVE on but it seems linux doesn't send
keepalives too often.

============================
problems with windows: reboot,
problems with unix: be root.
============================

Teemu <teemu@easy.com> wrote:
> Hi,
>
> I'm trying to make a file transfer server that transfers files very fast.
> The problem is that for some reason it takes very long for the server to
> notice if the connection to the client is (fysically) down, mean while it
> keeps sending those packages until the tcp stacks send buf fills. Sometimes
> this might tak as long as 15 minutes. I have set the following:

That's standard behaivour for TCP. You will only get a fast notification
of failure if the service (not the host offering the server) is shut
down. You should then get something like a Broken Pipe message.

It seems that you want to be able to meddle with the timeouts, which you
shouldn't really do (they are engineered the way they are for a reason).

15 minutes does seem a bit excessive however.

If you really wanted (and I'm not sure this is the best solution
either), is to make a version of read/write that you impose a timeout
on. I'm not sure is recvmsg or sendmsg can deal with TCP.

The basics of this are the following (see Unix Network Programming, by
Stevens for a more complete rundown)

int timedread( int fd, void *buf, size_t count, unsigned int seconds )
{
int res;
alarm( seconds );
/* set SIGALARM handler to an empty function, its only purpose is to
* interrupt the read */
res = read( int buf, count );
/* reset SIGALARM to whatever it was */
}

Be a bit careful with this though, you really should read Stevens to see
the advice he gives on the matter wrt race conditions.

On second thought, it would be preferable (for me at least, if I were
doing this) to use select instead of signals. This is tidier, and with
less undesired side effects.

> I've also tried to set KEEPALIVE on but it seems linux doesn't send
> keepalives too often.

This is probably because its blocked in a read or write, and therefore
shouldn't be sending a KEEPALIVE.

> ==============================
> problems with windows: reboot,
> problems with unix: be root.
> ==============================

Nice sig, but preface it with a line saying "-- ", so client software
knows its a sig and can remove it.

--
Cameron Kerr
cameron.kerr@paradise.net.nz : http://nzgeeks.org/cameron/
Empowered by Perl!

Does the setting of the size of the socket buffer really affect the
speed at which a lost peer is noticed? I could see where it would
affect how long it would be before your application filled the socket
buffer, but that isn't really the same thing right?

Are you interpreting an EAGAIN or EWOULDBLOCK (which I believe you
would get when the socket buffer filled) as meaning the remote was
toast?

Seems to me that at the time the remote system dissapears of the face
of the net, there starts something of a race - a race between the TCP
stack's retransmission limits for the first segment sent after the
remote goes poof, and how long it is before your application fills the
local send socket buffer.

rick jones
--
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

Rick Jones wrote:

> Does the setting of the size of the socket buffer really affect the
> speed at which a lost peer is noticed? I could see where it would
> affect how long it would be before your application filled the socket
> buffer, but that isn't really the same thing right?

The behaviour seems to be different in different cases. If the the client
the connection breaks because software-hangs etc. The error is given right
away. However if the connection is lost because the network is down such as
in the case that I cut the cable etc. then it does seem that if I use for
instance 4k buffer size and keep sending pakcets it takes very long to get
anykind of notification that the connection is lost.

> Are you interpreting an EAGAIN or EWOULDBLOCK (which I believe you
> would get when the socket buffer filled) as meaning the remote was
> toast?

I'm hoping to get either one of those messages, but basically any error
would do. Also because of serious threading the errno is sometimes changed
before I have time to read it..

>
> Seems to me that at the time the remote system dissapears of the face
> of the net, there starts something of a race - a race between the TCP
> stack's retransmission limits for the first segment sent after the
> remote goes poof, and how long it is before your application fills the
> local send socket buffer.

This is what I suspect. The question is wheater or not I can do anything
about it?

- Teemu

============================
problems with windows: reboot,
problems with unix: be root.
============================

I'm not sure weather this kind of timed read would help much since I'm using
a nonblocking socket...


============================
problems with windows: reboot,
problems with unix: be root.
============================

Teemu <teemu@easy.com> wrote:
> The behaviour seems to be different in different cases. If the the
> client the connection breaks because software-hangs etc. The error
> is given right away. However if the connection is lost because the
> network is down such as in the case that I cut the cable etc. then
> it does seem that if I use for instance 4k buffer size and keep
> sending pakcets it takes very long to get anykind of notification
> that the connection is lost.

Indeed if the remote app goes toast, a send to it will elicit a RST as
the remote TCP is still running. The remote system goes toast, and
packets go into a black hole. No RST and you have to wait for the
retransmission timeouts.

However, I'd be very surprised if the retransmission timeouts were
affected by the settting of SO_SNDBUF.

They would be affected by system tuning - say via sysctl.

> I'm hoping to get either one of those messages, but basically any
> error would do. Also because of serious threading the errno is
> sometimes changed before I have time to read it..

Huh? I was under the impression a properly threaded situation would
have a thread-specific errno. Are you sure that you are running on
"proper" threads and have compiled "properly" for threads on yhour
platform?

rick jones
--
Process shall set you free from the need for rational thought.
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...