The preliminary benchmark started out simple enough. In the client app, I set up a for…loop with a start/end timer to see how many requests the server app could handle within a given time period. Surprised by the results, I decided to re-tool the server app into a proper HTTP web server that could handle a GET request and return an HTTP 200 OK with “hello world” to the client. The results didn’t change much from before (still very promising) so I decided to install Node.js in Windows 7 and run a more thorough benchmark using ApacheBench, comparing against Node.js and Apache.
This is a simulated benchmark which should not be taken as a conclusive real-world test. However, the results are fascinating and worth looking into further.
Each server was configured with an identical “hello world” page for testing. For the .NET Sockets and Node.js servers I coded the socket to send a “hello world” HTTP response to the client. For Apache I setup an index.html file with “hello world” as the content.
All tests were run using the “ab” command, short for ApacheBench, which is a utility program found in Apache’s bin directory.
Here are the final commands that were used:
- ab -n 10000 -c 100 http://127.0.0.1/
- ab -n 100000 -c 1000 http://127.0.0.1/
-n (requests) is the number of requests to perform for this benchmark session.
-c (concurrency) is the number of multiple requests to perform at a time.
The commands were run several times for each server to verify accuracy and eliminate the chance that the OS was too busy multitasking to provide useful results.
The test rig was an Intel Core 2 Quad Q9400, 4 GB RAM and Windows 7 SP1 64-bit.
.NET Sockets outperforms Node.js and Apache in these benchmarks. For whatever reason, I couldn’t push Apache to 1,000 concurrent requests, even after editing the WINNT section in the httpd-mpm.conf. I tried another benchmark utility called siege which is just like ApacheBench and the results confirm what I’ve shown above.
From these benchmarks alone, the .NET Sockets project would appear to have potential as a high performance HTTP web server. But to even begin making assumptions on the accuracy of these tests compared to real-world scenarios, and the viability of .NET Sockets as a production/enterprise web server, I’ll need to do some more benchmark sessions using other computers over the internet, tuning each server to serve a typical HTML page or CMS, and comparing resource usage.
Stay tuned for Part III of this series where I plan to show 1) how I was able to modify the .NET Sockets code to host a WordPress site in as little as 5 minutes, 2) how I was able to do the same in Node.js, 3) how C10k is definitely not a problem for .NET Sockets, 4) comprehensive benchmark results with ApacheBench, Siege and WeigHTTP, along with CPU / RAM usage, showing how each server performs when serving up a typical WordPress site, and 5) include IIS, nginx and vert.x in the line-up.
If you notice an error in these benchmarks, please let me know!