The Participants

The objective of the benchmark is not testing deployment (like uwsgi vs gunicorn and etc) but instead test the frameworks itself.

The Methodic

Results bellow were received with MacBook Pro 2015, CPU: 2.7GHz i5, MEM: 8GB, OSX 10.10.4. I've used a WRK utility with params:

$ wrk -d20s -t10 -c200 [URL]

The tests were running from Gunicorn HTTP Server with 2 workers and Meinheld worker class for WSGI applications. Example:

$ gunicorn [APP] --workers=2 --worker-class=meinheld.gmeinheld.MeinheldWorker

The sources of the applications for tests can be found at here.

The benchmark has a three kind of tests:

  1. JSON test. Serialize a object to JSON and return `application/json` response.
  2. Remote test. Load and return http response from a remote server.
  3. Complete test. Load some data from DB using ORM, insert a new object, sort and render to template.

The Results

Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 49.50 52.52 52.11 3902.15
Bottle 10.53 11.47 12.95 16780.10
Django 47.54 50.31 49.54 4074.25
Falcon 8.38 8.86 8.72 22935.15
Flask 30.13 32.01 31.02 6473.15
Muffin 48.01 51.4 49.67 4004.90
Pyramid 19.39 21.2 21.52 9329.30
Wheezy.Web 7.00 7.42 7.18 27852.90
Tornado 61.25 70.64 63.54 3102.15
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 323.15 364.60 330.86 599.10 0
Bottle 3068.78 4825 3726.93 18.65 94
Django 2819.29 3614.57 3267.60 18.80 80
Falcon 2750.97 4015.53 3289.80 18.60 83
Flask 3192.16 4113.51 3460.58 18.70 72
Muffin 329.16 365.13 330.14 600.80 0
Pyramid 2566.04 4803.61 3592.70 18.25 83
Wheezy.Web 2835.03 3907.26 3341.06 18.80 72
Tornado 949.88 1157.15 1006.69 189.75 0
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 12.47 434.76 366.2 483.15
Bottle 455.78 474.61 505.7 401.05
Django 1620.56 2144.48 2156.89 81.95
Falcon 437.45 456.57 660.25 359.15
Flask 507.55 533.47 693.09 308.30
Muffin 390.03 650.67 695.73 376.75
Pyramid 486.93 513.26 561.51 361.20
Wheezy.Web 430.66 450.07 635.73 360.60
Tornado 597.85 613.6 579.97 332.00

Conclusion

Nothing here, just some measures for you.