Problem with TCP connection over loopback interface

Hi,

I maintain a library at work which functions like RPC allowing function
call abstraction across a network. The user simply calls the function,
and the action takes place on a remote processor, or perhaps on the
same processor in a different process.

The network connection between the client and server is TCP/IP. The
client either connects over the loopback interface, or a HDLC/PPP
interface to the other processor.

The primary issue is that when it is a loopback connection, it snaps
when there is a LOT of traffic. This is really painful and the
application finds it nearly impossible to handle.

The other problem I've observed with the TCP connection over the
loopback is as follows.
I use non-blocking sockets with a select loop.
The server-side may not be up.
The client calls connect.
The connect should fail with the timeout and show up in the select, or
get connected.
If it failed, I cleanup, close the socket, open a new socket and try to
connect again.

Now, since I'm using a non-blocking socket call (connect), I should
ALWAYS get either a succesful connect, or EINPROGRESS (correct me if
I'm wrong). However, sometimes I get a connect failure other than
EINPROGRESS.
The problems are observed primarily in a 2.2 kernel. I tried to look
this up but I've not been able to find anyone with a similar issue.

I'd like to know if anyone can help me with this, or if the better
solution is to go with Unix sockets for the loopback special case.

My suspicion is that there is some 'optimization' for the loopback
connection that I'm hitting against.


Thanks,
Prasanna