The Participants

The objective of the benchmark is not testing deployment (like uwsgi vs gunicorn and etc) but instead test the frameworks itself.

The Methodic

Results bellow were received with MacBook Pro 2015, CPU: 2.7GHz i5, MEM: 8GB, OSX 10.10.4. I've used a WRK utility with params:

$ wrk -d20s -t10 -c200 [URL]

The tests were running from Gunicorn HTTP Server with 2 workers and Meinheld worker class for WSGI applications. Example:

$ gunicorn [APP] --workers=2 --worker-class=meinheld.gmeinheld.MeinheldWorker

The sources of the applications for tests can be found at here.

The benchmark has a three kind of tests:

  1. JSON test. Serialize a object to JSON and return `application/json` response.
  2. Remote test. Load and return http response from a remote server.
  3. Complete test. Load some data from DB using ORM, insert a new object, sort and render to template.

The Results

Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 58.96 59.61 64.49 3368
Bottle 15.89 16.18 15.61 12838
Django 41.41 42.14 42.52 4762
Falcon 11.7 12.14 11.72 17062
Flask 43.03 43.79 43.33 4630
Muffin 73.5 74.34 81.7 2703
Pyramid 26.84 27.25 27.13 7383
Wheezy.Web 10.23 10.48 10.24 19542
Tornado 77.17 78.09 77.51 2578
Weppy 23.66 24.32 24.05 8345
WSGI 9.47 11.61 9.17 21821
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 412.08 433.48 406.53 483.00 615.20
Bottle 2885.36 3964.56 3207.14 18.30 15.85
Django 2908.53 4165.52 3477.36 18.10 14.30
Falcon 2944.88 4842.57 3629.31 18.35 14.20
Flask 2679.56 3990.17 3344.27 18.15 13.25
Muffin 451.77 477.42 449.69 439.10 617.90
Pyramid 2980.71 3870.58 3305.18 18.00 14.10
Wheezy.Web 3125.03 4346.63 3573.43 18.35 13.70
Tornado 1044.8 1068.02 1036.97 187.80 103.65
Weppy 2593.71 4492.56 3468.05 18.25 16.15
WSGI 2607.26 4677.82 3578.6 18.40 16.15
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 10.51 1091.73 925.21 204.80
Bottle 1167.93 1207.48 1398.16 128.20
Django 2367.04 2742.09 2904.04 42.90 16
Falcon 1142.31 1186.59 1269.32 146.85
Flask 1281.38 1326.05 1440.24 123.00
Muffin 737.57 919.56 990.87 158.30 39
Pyramid 1247.13 1278.48 1497.29 120.65
Wheezy.Web 1129.41 1204.31 1377.88 138.05
Tornado 1395.43 1409.24 1344.69 143.00
Weppy 1578.43 1759.02 1899.08 88.50
WSGI 1165.66 1205.71 1331.75 144.35

Conclusion

Nothing here, just some measures for you.

Archive