epoll, and exceeding maximum sockets hangs process?

I'm having a problem with socket programming on Linux. I have many clients
connecting to a listen()ing socket. If I exceed the number of allowable
sockets for my process, accept() will still return a valid socket number.
The problem is that the entire process then stops receiving socket events.
The second a client disconnects and brings the server's socket count below
its maximum, the events start happening again.

The problem is that just exceeding the socket count hasn't been a reliable
indication for me - that is, I can't just hard-code this limit. First, it
may not be the same on every platform the application will run on. Second,
if a socket (a TCP client) is in any of the termination _WAIT states after
it has terminated, I'll be given the socket back but it still seems to
hang the server process. If I wait for them to expire it goes back to
normal.

What's going on, and is there some way I can identify either:
1) That I've hit the limit, so I can stop calling accept(), or
2) That I've EXCEEDED the limit and am now not getting events?

What I'd REALLY like to do is have the accept() call fail but that's not
happening. Also, why does socket 1027 affect socket 43? Shouldn't socket
43 continue operating normally, even if socket 1027 is past my limit?

Thanks,
Bill