More than 16,000 connections in Tomcat
The most important change is that HttpClient is used instead of standard HTTP connection. Moreover the servlet address is hard-coded, so a new String is not created every time. I do not believe that this change has big impact but the code generates less garbage so GC has easier job.
With this small enhancements, I was able to get more than 16,000 connections running
... Servlet no. 16354 called. Servlet no. 16355 called. Servlet no. 16356 called. Servlet no. 16357 called. May 6, 2010 1:21:54 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.SocketException) caught when processing request: Too many open files May 6, 2010 1:21:54 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Again, we get "To many open files". The limit is 32768 and we have two open connections per thread (incoming/outgoing). Theoretically, we could get even higher, but the heap was still filling really fast, so GC was quite slow. And again, threads are not the main issue, the problem is still somewhere else.
The source code is available here.