Scalability problem using web requests initiating asynch web service requests

We are experiencing some problems in production (IIS 7.5 integrated managed pipeline mode and .NET 4.5 application) when to may of our incoming IIS requests trigger asynchronous web service requests. There is a quite nice MSDN blog post here by Thomas Marquardt that gives a very good picture of the potential problems that can arise. I believe our problem is due to the ServicePointManager.DefaultConnectionLimit low default value and we are about to deploy this fix (requires code change as can be read about in the blog.

However, I have quite a few remaining questions from this blog post. I have posted these questions yesterday in the blog post (quite miserably since there was a cap on the word lenght...), but since I am guessing it may take some time before Thomas may response, I repost the questions here as well in hope that someone in the community knows some of the answers. You may need to read the blog post first to be in tune with what I write below, but I can promise it is good reading. Image may be NSFW.
Clik here to view. If someone knows one or several of the answers to the questions, please just refer to the what question it is in the answer (eg 1.c)

Here is my comment(s) in the blog post with nice WYSIWYG markup:

Thanks [Thomas Marquardt] for some great explanations and for taking the time to answer all these questions. It really clarifies things. In our case, we have a problem in production with an .NET 4.5 / IIS 7.5 web server running integrated [managed pipeline] mode when we get a high load of requests that utilises a high degree of outgoingasynchronous web service requests (stack trace reveals HttpWebRequest usage beneath the hood of the HttpClient class usage). Based on my reading here, it seems the solution is to set theServicePointManager.DefaultConnectionLimit programmatically in production code as you suggest (since we use autoConfig set to true). However before I try this in our production environment (initiating a redeploy) I have some remaining categorised questions with regards to what I’ve read (sublabeled (a), (b) etc to make your answering easier):

1. appRequestQueueLimit config:
   (a) Do I understand it correctly that the httpRuntime setting appRequestQueueLimit configuration only is applicable for IIS 6 and 7 when running in classic mode? I.e it sets the limit for the old application scoped queue (with “miserable performance” as you write)?
2. (questions a-f below) 
   When our servers gets over loaded and don’t scale (CPU only at 20%), we get 503 error code in response to the original request + that the async. web service requests ends up with quite a few exceptions: “SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full”. A problem with this type of error message hint is of course that you are not sure which queue may be full (if it is the queue). 
   
   (a) So how many queues exists on the server in the (total) request pipeline? From you description, in our scope, there is both a (1) process-wide native queue as well as the (2) CLR ThreadPool that naturally can be regarded as a queue. But you also mention the HTTP.sys kernal queue. 
   (b) Is the HTTP.sys kernal queue the same queue as (1) process-wide queue? 
   (c) If not, could you perhaps explain breifly what the purpose of this HTTP.sys queue is? 
   (d) Are there any other lower level queues that can become problematic? (eg network card driver) 
   
   However, in our case, if the problem is actually due to connectionManagement/MaxConnections, I guess the web service error message is quite logical. If there is no possibility for the web service to get access to the connection (keep-alive), the number of requests waiting to execute in the ThreadPool should increase of course until the maxium number of threads is hit. A poor solution would be to increase the number of maxThreads in the ThreadPool... 
   (e) Is there a performance monitor for showing the current number of threads in the ThreadPool? 
   
   Also, there seems to be a performance counter called [Web Service.Current Connections] that should really hit a max pegged value (12*core count) if this is the problem. 
   (f) Is this correct (below the (e) question)?
    
3. From you question response 8 Dec 2011, I understand it that “Requests Queued” is use both to measure the number of requests that are waiting as well as the number of executing requests in the ThreadPool + those requests that possible (unlikely if correctly configured) hit the native queue. It seems to me then, that it is quite difficult to distinguish the scenario when there actually are requests in the native queue. 
   (a) Would that be possible in .NET 4.5 / IIS 7 integrated mode? (how if so? :)
   
4. You mentioned in comment 22 april, 2010 and 28 oct, 2010 that if you run out of (ephemeral) ports it can be good to decrease the TcpTimedWaitDelay registry setting (as well as MaxUserPort). For us I don’t think this is a problem since we use TCP keep-alive (persisted) connection for the web service requests. 
   (a) However, in order to find out if this was a problem, would it not be enough to check the performance counter [TCPv4.Connection Established] and see if this value is close to the maximum amount of available ports (which should be quite high on Windows 2003/2008 by default). 
   
   Further more, I think decreasing TcpTimedWaitDelay is a good option although perhaps one need to consider other types of TCP reliant I/O high latency requests running on the server (eg web services and even database calls) if lowering this number. 
   (b) Do you agree?

 I can of course follow up on this thread with answers from our internal investigation to my questions.