The Participants

The objective of the benchmark is not testing deployment (like uwsgi vs gunicorn and etc) but instead test the frameworks itself.

The Methodic

Results bellow were received with MacBook Pro 2015, CPU: 2.7GHz i5, MEM: 8GB, OSX 10.10.4. I've used a WRK utility with params:

$ wrk -d20s -t10 -c200 [URL]

The tests were running from Gunicorn HTTP Server with 2 workers and Meinheld worker class for WSGI applications. Example:

$ gunicorn [APP] --workers=2 --worker-class=meinheld.gmeinheld.MeinheldWorker

The sources of the applications for tests can be found at here.

The benchmark has a three kind of tests:

  1. JSON test. Serialize a object to JSON and return `application/json` response.
  2. Remote test. Load and return http response from a remote server.
  3. Complete test. Load some data from DB using ORM, insert a new object, sort and render to template.

The Results

Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 46.88 47.47 51.86 4223
Bottle 10.73 11.2 13.39 18193
Django 45.47 46.18 46.61 4330
Falcon 8.24 8.39 8.26 24201
Flask 29.11 29.6 29.38 6831
Muffin 49.45 49.99 54.22 4013
Pyramid 18.57 19.04 46.89 9178
Wheezy.Web 6.9 7.8 6.96 28714
Tornado 63.33 64.32 63.67 3142
Weppy 18.37 18.87 18.46 10853
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 331.4 347.95 316.05 615.20 0
Bottle 3050.27 4018.62 3382.84 15.85 76
Django 3134.88 4118.46 3352.47 14.30 80
Falcon 4141.92 4764.49 3989.88 14.20 72
Flask 3390.8 4743.08 3713.45 13.25 68
Muffin 319.08 336.01 308.84 617.90 0
Pyramid 3120.22 4334.69 3443.16 14.10 76
Wheezy.Web 3431.26 5887.89 4059.2 13.70 73
Tornado 1876.79 2007.2 1834.42 103.65 0
Weppy 3072.21 4719.56 3689.12 16.15 76
Name 50% (ms) 75% (ms) Avg (ms) Req/s Timeouts
Aiohttp 64.97 397.81 384 501.50
Bottle 462.88 481.32 725.3 316.80
Django 1589.02 2012.29 2294.97 67.90
Falcon 436.01 457.18 906.96 295.10
Flask 511.83 535.05 865.54 259.45
Muffin 425.5 526.15 761.01 418.60
Pyramid 491.75 519.03 880.64 265.30
Wheezy.Web 422.67 436.88 609.16 369.40
Tornado 605.23 619.83 591.05 327.35
Weppy 667 679.66 764.5 258.60

Conclusion

Nothing here, just some measures for you.

Archive