Ufdjrw

The goal is to make an http request to a service that could take over an hour to respond.

I created a httpwebrequest and sent the request body fine,
then I used:

var ar = BeginGetResponse(null, null);
ar.AsyncWaitHandle.WaitOne();

This is in the belief that the async call does not respect timeouts so I expect the request to wait forever for the server to respond.

On the server when it eventually has its response ready,
when it tries to write I get the exception "the remote host has closed the connection"

It looks like the client is still waiting at the WaitOne() but the connection has been closed.

Finally sorted this I hope this saves others some time.
I think the critical change was activating settcpkeepalive
but I have mentioned other settings I changed just in case

In IIS, using location tags in the web.config I set the executetimeout for the specifc ashx to 2 hours (thanks to Jon Hanna and Tomas Varacek for pointing this out) Not sure if this is absolutely necessary but if you have a process that takes up to 2 hours then it seems to make sense that the executetimeout needs to be at least 2 hours.

The expect100continue in IIS is disabled.
On the client side, the default for servicepoint expect100continue is true.
I set it to false;

I set the servicepoint maxidletime to 2 hours.
Not sure if a connection waiting for a response is regarded as idle or not.
I think this refers to a service point which is not associated with a connection, so probably not relevant to my problem

I always use the begin/end pattern (async) versions where available
especially beginread and endread. This is to avoid having to decipher the many timeouts on request/stream

The default value of the service point settcpkeepalive is false
I set the service_point SetTcpKeepAlive(true, 1000 * 60, 1000);
I think this means that after 1 minute of idleness (waiting for the response) the client sends a keep alive "probe" to remind the server it is still waiting. If it does not receive a response to the probe it sends another after a second. Hopefully after it receives a response to the keep alive probe it returns to the 1 minute wait, but I am not sure. Note that there is a max of 10 probes which cannot be changed.

Note that of course the timeout interval must be less than the connectionTimout configured for the site in IIS site advanced setting connectionlimits or else the connection will timeout before the first keep alive packet is sent.

Another way I solved the problem was to accept that the system discourages open connections that are just waiting, so before kicking off the long running process, I start a task that does a response.Write every so often (1 minute).
I flush the write and buffering is off on the response and nagling is disabled for the site and even though the message is small (1 byte) it does get all the way back to the client 3000 miles away)
In this scenario:
IIS does not regard the connection as idle, the httpwebrequest does not regard the connection as idle and the request keeps waiting just as I want it to.
I have had this scheme running for several hours without problems

Of course yet another way would be to have the server kick off the process and return immediately and have some periodic follow up requests poll the server to see if it is finished. Some would say this is more in the tradition of the request/response http methodology. I may do still do that but I wanted to have this option as a fallback if needed