The Participants

The objective of the benchmark is not testing deployment (like uwsgi vs gunicorn and etc) but instead test the frameworks itself.

The Methodic

Results bellow were received with MacBook Pro 2015, CPU: 2.7GHz i5, MEM: 8GB, OSX 10.10.4. I've used a WRK utility with params:

$ wrk -d20s -t10 -c200 [URL]

The tests were running from Gunicorn HTTP Server with 2 workers and Meinheld worker class for WSGI applications. Example:

$ gunicorn [APP] --workers=2 --worker-class=meinheld.gmeinheld.MeinheldWorker

The sources of the applications for tests can be found at here.

The benchmark has a three kind of tests:

  1. JSON test. Serialize a object to JSON and return `application/json` response.
  2. Remote test. Load and return http response from a remote server.
  3. Complete test. Load some data from DB using ORM, insert a new object, sort and render to template.

The Results

Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 45.31 55.86 50.67 3974.30
Bottle 10.36 10.77 13.75 17385.40
Django 46.87 48.06 47.20 4203.35
Falcon 8.32 8.67 8.60 23256.95
Flask 33.48 35.56 33.94 5907.25
Muffin 43.59 44.5 45.04 4481.30
Pyramid 18.55 19.06 18.47 9111.35
Wheezy.Web 6.85 7.11 6.94 28848.00
Tornado 61.8 63.23 63.27 3164.40
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 301.09 338.34 301.78 644.65 0
Bottle 2730.96 4087.95 3358.19 18.50 77
Django 2541.40 3909.72 3288.68 18.75 86
Falcon 2522.51 4218.00 3329.89 18.50 89
Flask 2551.92 4537.17 3472.74 18.75 79
Muffin 303.88 346.66 322.74 615.10 0
Pyramid 2526.57 4413.16 3454.03 18.85 83
Wheezy.Web 2654.00 4577.51 3472.52 18.75 87
Tornado 1014.49 1131.97 1018.09 191.00 0
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 39.95 397.47 345.65 451.10
Bottle 435.92 459.08 563.92 363.05
Django 1495.69 1868.92 1872.45 97.70
Falcon 802.64 862.8 992.49 189.30
Flask 557.61 599.63 676.11 301.55
Muffin 421.85 710.4 802.08 325.75
Pyramid 466.76 483.67 570.83 364.30
Wheezy.Web 455.37 474.73 545.89 381.65
Tornado 616.44 630.17 602.07 322.45

Conclusion

Nothing here, just some measures for you.

Archive