A Simple Webserver Performance Comparison Test

Last week, I had a long weekend (3 days off). It made me very bored. During that time I read PC Magazine. In it there is an article regarding several web performance tools. One of them is ab2 that is included in the Apache webserver.

For a long time I want to measure the performance of several webservers. That article inspired me to test the performance of several webservers.

The webservers I want to measure are Apache 2.0.55, Yaws, lighttpd and WEBrick (included with Ruby, actually it is not a webserver per se but a low-level web framework). My testing method may not be scientific enough, but at least it gives me a clue about the performance of the webservers I mentioned above.

Here are the methods I will use :

  • create a static webpage (it is a copy of http://tedi.heriyanto.net)
  • Apache will listen on port 80
  • Lighttpd will listen on port 80
  • Yaws will listen on port 8000
  • WEBrick will listen on port 8080
  • using ab2 request for a static webpage for 200 times from each webserver
  • using ab2 request for a static webpage for 2000 times from each webserver

Notes :
In this document, I will not give details about the installation of each webserver. You can consult with the appropriate documents for that.

Apache Setup

I setup the Apache so it can handle each user website from the URL : http://website/~user/

It’s a default setting in my SUSE Prof 9.3 Apache (version 2.0.55).

The directory is /home/tedi/public_html
There I put file
index.html to that directory

Lighttpd Setup

I just use the default Lighttpd configuration. The webpage is store in /srv/www/htdocs

Yaws Setup

I created a special directory for Yaws (test-yaws) and it contains static webpage (index.html). I configure Yaws (yaws.conf) with the following virtual server :

WEBrick Setup

For WEBrick, I had to write a simple webserver code like the following :

#!/usr/bin/env ruby
require ‘webrick’
include WEBrick

def start_webrick(config={})
config.update(:Port => 8080)
server = HTTPServer.new(config)

['INT', 'TERM'].each {|signal|
trap(signal) {server.shutdown}
}

server.start
end

start_webrick(:DocumentRoot => ‘/home/tedi/public_html’)


Starting Apache

To start apache just use the following command :

# rcapache start
Starting Lighttpd

To start lighttpd just type the following command :

# /etc/init.d/lighttpd start

Starting Yaws

To start Yaws, I use the following command :


$ bin/yaws -i
Erlang (BEAM) emulator version 5.4.12 [hipe]

Eshell V5.4.12 (abort with ^G)
1>
=INFO REPORT==== 10-Apr-2006::13:30:21 ===
Yaws: Using config file /home/tedi/yaws.conf
yaws:Add path “/home/tedi/software/erlang/yaws/scripts/../examples/ebin”
yaws:Add path “/home/tedi/software/erlang/yaws/examples/ebin”
yaws:Running with id=”default”
Running with debug checks turned on (slower server)
Logging to directory “/home/tedi/yaws_logs”

=INFO REPORT==== 10-Apr-2006::13:30:21 ===
Yaws: Listening to 0.0.0.0:8000 for servers
- http://suse:8000 under /home/tedi/software/erlang/yaws/scripts/../www
- http://localhost:8000 under /home/tedi/test-yaws
Starting WEBrick

To start WEBrick just type the following command :

$ ruby webserver.rb
[2006-04-10 13:26:52] INFO WEBrick 1.3.1
[2006-04-10 13:26:52] INFO ruby 1.8.2 (2004-12-25) [i686-linux]
[2006-04-10 13:26:52] WARN TCPServer Error: Address already in use – bind(2)
[2006-04-10 13:26:52] INFO WEBrick::HTTPServer#start: pid=6893 port=8080

Testing

Here are the commands I used to test Apache webserver performance :

/usr/sbin/ab2 -n 200 -c 10 http://localhost/~tedi/index.html
/usr/sbin/ab2 -n 2000 -c 10 http://localhost/~tedi/index.html

Here are the commands I used to test Lighttpd webserver performance :

/usr/sbin/ab2 -n 200 -c 10 http://localhost/index.html
/usr/sbin/ab2 -n 2000 -c 10 http://localhost/index.html

Here are the commands I used to test Yaws webserver performance :

/usr/sbin/ab2 -n 200 -c 10 http://localhost:8000/index.html
/usr/sbin/ab2 -n 2000 -c 10 http://localhost:8000/index.html

Here are the commands I used to test WEBrick webserver performance :

/usr/sbin/ab2 -n 200 -c 10 http://localhost:8080/index.html
/usr/sbin/ab2 -n 2000 -c 10 http://localhost:8080/index.html

Results

For 200 connections

Apache

Software: Apache/2.0.55
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 0.89339 seconds
Complete requests: 200
Total transferred: 1013400 bytes
HTML transferred: 956400 bytes
Requests per second: 2238.66 [#/sec] (mean)
Time per request: 4.467 [ms] (mean)
Time per request: 0.447 [ms] (mean, across all concurrent requests)
Transfer rate: 11070.19 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.9 0 4
Processing: 2 2 1.4 3 6
Waiting: 1 1 0.8 1 4
Total: 3 3 1.0 3 6

Percentage of the requests served within a certain time (ms)
50% 3
66% 3
75% 3
80% 3
90% 6
95% 6
98% 6
99% 6
100% 6 (longest request)

Yaws

Server Software: Yaws/1.57
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 0.171134 seconds
Complete requests: 200
Total transferred: 1026120 bytes
HTML transferred: 975528 bytes
Requests per second: 1168.67 [#/sec] (mean)
Time per request: 8.557 [ms] (mean)
Time per request: 0.856 [ms] (mean, across all concurrent requests)
Transfer rate: 5855.06 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.1 2 5
Processing: 1 2 1.0 3 6
Waiting: 0 1 1.0 1 4
Total: 4 4 0.8 4 7

Percentage of the requests served within a certain time (ms)
50% 4
66% 4
75% 4
80% 4
90% 5
95% 7
98% 7
99% 7
100% 7 (longest request)

Lighttpd

Server Software: lighttpd/1.4.11
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 0.53806 seconds
Complete requests: 200
Total transferred: 1003000 bytes
HTML transferred: 956400 bytes
Requests per second: 3717.06 [#/sec] (mean)
Time per request: 2.690 [ms] (mean)
Time per request: 0.269 [ms] (mean, across all concurrent requests)
Transfer rate: 18195.00 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.0 1 3
Processing: 1 1 0.8 1 4
Waiting: 0 0 0.7 0 3
Total: 2 2 0.6 2 4

Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 4
95% 4
98% 4
99% 4
100% 4 (longest request)

WEBrick

Server Software: WEBrick/1.3.1
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 1.435260 seconds
Complete requests: 200
Total transferred: 1009582 bytes
HTML transferred: 960496 bytes
Requests per second: 139.35 [#/sec] (mean)
Time per request: 71.763 [ms] (mean)
Time per request: 7.176 [ms] (mean, across all concurrent requests)
Transfer rate: 686.29 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 1
Processing: 35 55 23.6 54 277
Waiting: 20 38 23.4 37 258
Total: 35 55 23.7 54 278

Percentage of the requests served within a certain time (ms)
50% 54
66% 56
75% 58
80% 59
90% 66
95% 70
98% 77
99% 272
100% 278 (longest request)

For 2000 connections

Apache

Software: Apache/2.0.55
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 0.658329 seconds
Complete requests: 2000
Total transferred: 10134000 bytes
HTML transferred: 9564000 bytes
Requests per second: 3037.99 [#/sec] (mean)
Time per request: 3.292 [ms] (mean)
Time per request: 0.329 [ms] (mean, across all concurrent requests)
Transfer rate: 15032.00 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 4
Processing: 1 2 0.8 2 7
Waiting: 1 1 0.5 1 5
Total: 2 2 1.0 2 7

Percentage of the requests served within a certain time (ms)
50% 2
66% 3
75% 3
80% 3
90% 3
95% 4
98% 6
99% 6
100% 7 (longest request)

Lighttpd

Server Software: lighttpd/1.4.11
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 0.416057 seconds
Complete requests: 2000
Total transferred: 10060090 bytes
HTML transferred: 9592692 bytes
Requests per second: 4807.03 [#/sec] (mean)
Time per request: 2.080 [ms] (mean)
Time per request: 0.208 [ms] (mean, across all concurrent requests)
Transfer rate: 23612.15 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 2
Processing: 0 1 0.7 1 5
Waiting: 0 0 1.0 1 4
Total: 0 1 0.8 1 7

Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 2
80% 2
90% 2
95% 3
98% 3
99% 4
100% 7 (longest request)

Yaws

Server Software: Yaws/1.57
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 1.32380 seconds
Complete requests: 2000
Total transferred: 10060000 bytes
HTML transferred: 9564000 bytes
Requests per second: 1937.27 [#/sec] (mean)
Time per request: 5.162 [ms] (mean)
Time per request: 0.516 [ms] (mean, across all concurrent requests)
Transfer rate: 9515.88 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 3 18.1 2 605
Waiting: 0 3 18.1 2 605
Total: 0 3 18.1 2 605

Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 3
80% 3
90% 4
95% 4
98% 6
99% 7
100% 605 (longest request)

WEBrick

Server Software: WEBrick/1.3.1
Document Length: 4782 bytes

Concurrency Level: 10
Time taken for tests: 13.272744 seconds
Complete requests: 2000
Total transferred: 10054582 bytes
HTML transferred: 9568096 bytes
Requests per second: 150.68 [#/sec] (mean)
Time per request: 66.364 [ms] (mean)
Time per request: 6.636 [ms] (mean, across all concurrent requests)
Transfer rate: 739.71 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 95.0 0 2999
Processing: 4 61 15.2 66 251
Waiting: 0 43 13.4 48 236
Total: 27 65 95.4 66 3059

Percentage of the requests served within a certain time (ms)
50% 66
66% 67
75% 68
80% 69
90% 70
95% 71
98% 77
99% 87
100% 3059 (longest request)

Remarks

Here are the summary of the time taken to complete requests for each webserver :

200 connections

Yaws :

Time taken for tests: 0.171134 seconds
Requests per second: 1168.67 [#/sec] (mean)
Time per request: 8.557 [ms] (mean)
Time per request: 0.856 [ms] (mean, across all concurrent requests)
Transfer rate: 5855.06 [Kbytes/sec] received

Lighttpd :

Time taken for tests: 0.53806 seconds
Requests per second: 3717.06 [#/sec] (mean)
Time per request: 2.690 [ms] (mean)
Time per request: 0.269 [ms] (mean, across all concurrent requests)
Transfer rate: 18195.00 [Kbytes/sec] received

Apache :

Time taken for tests: 0.89339 seconds
Requests per second: 2238.66 [#/sec] (mean)
Time per request: 4.467 [ms] (mean)
Time per request: 0.447 [ms] (mean, across all concurrent requests)
Transfer rate: 11070.19 [Kbytes/sec] received

WEBrick :

Time taken for tests: 1.435260 seconds
Requests per second: 139.35 [#/sec] (mean)
Time per request: 71.763 [ms] (mean)
Time per request: 7.176 [ms] (mean, across all concurrent requests)
Transfer rate: 686.29 [Kbytes/sec] received

2000 connections

Lighttpd :

Time taken for tests: 0.416057 seconds
Requests per second: 4807.03 [#/sec] (mean)
Time per request: 2.080 [ms] (mean)
Time per request: 0.208 [ms] (mean, across all concurrent requests)
Transfer rate: 23612.15 [Kbytes/sec] received

Apache :

Time taken for tests: 0.658329 seconds
Requests per second: 3037.99 [#/sec] (mean)
Time per request: 3.292 [ms] (mean)
Time per request: 0.329 [ms] (mean, across all concurrent requests)
Transfer rate: 15032.00 [Kbytes/sec] received

Yaws :

Time taken for tests: 1.32380 seconds
Requests per second: 1937.27 [#/sec] (mean)
Time per request: 5.162 [ms] (mean)
Time per request: 0.516 [ms] (mean, across all concurrent requests)
Transfer rate: 9515.88 [Kbytes/sec] received

WEBrick :

Time taken for tests: 13.272744 seconds
Requests per second: 150.68 [#/sec] (mean)
Time per request: 66.364 [ms] (mean)
Time per request: 6.636 [ms] (mean, across all concurrent requests)
Transfer rate: 739.71 [Kbytes/sec] received

Based on the information above, I can infer that the followings are the fastest webserver in descending order (for 2000 connections at one time) :

  • Lighttpd
  • Apache
  • Yaws
  • WEBrick

The fastest webserver in descending order (for 200 connections at one time) are :

  • Yaws
  • Lighttpd
  • Apache
  • WEBrick

    Please note that the file used is a static webpage. The result may be different if we use a dynamic webpage.

    About these ads

    3 thoughts on “A Simple Webserver Performance Comparison Test

    1. Looks like there’s some inaccuracies in the test results.

      The test for YAWS 200 test says it took the shortest amount of time.

      -H

    Leave a Reply

    Please log in using one of these methods to post your comment:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s