Last updated on August 17, 2018
I’ve been playing with the Lua engine in Nginx for a while. My primary intent is to offload most, if not all, of my WordPress security stuffy from running in the PHP environment to running in something that potentially won’t use as much in the way of resources. The first question I need to answer before I can reasonably consider doing this is what kind of of overhead doing extended processing in Nginx–Lua imposes in terms of performance.
To put some perspective on this, I’ve been running the WordPress security plug-in Word Fence for a while now. When I compare my production server (which has Wordfence enabled) and my development server (which doesn’t have word fence installed, but is otherwise running the same plugins and code base), I see on average a 10–20 ms increase page rendering times, and nearly 20 additional database queries per page.
The overhead from Wordfence isn’t creating a performance problem per say, however, shaving even 15 ms off a 50–60 ms page render time is an appreciable improvement. Additionally, less resources consumed by a bad actor means more resources are available for actual users.
In any even the question here is how much performance overhead does the Nginx-Lua module carry for doing some reasonable processing.
Test Lua WAF
I didn’t want to sit down and write a full firewall engine without having some idea how good or poorly all of this was going to preform. Most of the tests are string compares looking for potentially dangerous patterns (I also check for some spam keywords in submitted comments.)
The test codebase preforms a number of repeated sting compares using string:find and luau’s pattern matching engine.
The test codebase was designed to be fast to write, not necessarily most efficient. However, I have followed the recommended optimizations on the Nginx-Lua github page.
Test Hardware
All of the testing was run on my development server. The relevant hardware is as follow:
CPU: Intel Xeon E3-1220v2 (4c/4t, 3.1 GHz)
Memory: 8GB DDR3-1333/PC3-10600 (2x4GB)
Storage: 1x 128GB Samsung 940 Evo (/root)
2x 4TB Seagate IronWolf (/data, ZFS mirror 7200 RPM)
Software
OS: Ubuntu 14.04.2 (4.4.0-78-generic kernel)
Nginx: 1.13.0
PHP: 7.1.5 (opcache enabled)
MySQL: 5.6.33
LuaJit: 2.0.2
CPU was run with the performance power governor mode.
Test Protocol & Config Notes
Nginx configuration was changed. Nginx was reloaded with service nginx reload. Apache Bench (AB) was run twice, the results form the second run were saved.
The WordPress/PHP 7.1 tests were loading pages from my development site running WordPress 4.7.5. Wordfence was not installed.
Nginx on this machine is configured to use 1 worker process. PHP-FPM is configured to allow up to 5 child processes, though only 2 concurrent clients were being by AB, so only 2 child processes were ever active.
Testing
Test 1 ( PHP 7.1, Processing )
Pulling a PHP rendered file (blog page).
> ab -n 1000 -c 2 -f TLS1.2 https://test.dev/blog/2016/12/test-inverse-block/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /blog/2016/12/test-inverse-block/
Document Length: 33596 bytes
Concurrency Level: 2
Time taken for tests: 18.481 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 33940000 bytes
HTML transferred: 33596000 bytes
Requests per second: 54.11 [#/sec] (mean)
Time per request: 36.961 [ms] (mean)
Time per request: 18.481 [ms] (mean, across all concurrent requests)
Transfer rate: 1793.47 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.1 1 3
Processing: 33 35 0.5 35 40
Waiting: 32 35 0.5 35 39
Total: 34 37 0.5 37 41
Percentage of the requests served within a certain time (ms)
50% 37
66% 37
75% 37
80% 37
90% 37
95% 38
98% 38
99% 38
100% 41 (longest request)
Test 2 ( PHP 7.1, Processing, FW Disabled)
Same as the previous test, but without nginx loading and running requests through the Lua firewall code.
> ab -n 1000 -c 2 -f TLS1.2 https://test.dev/blog/2016/12/test-inverse-block/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /blog/2016/12/test-inverse-block/
Document Length: 33596 bytes
Concurrency Level: 2
Time taken for tests: 18.268 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 33940000 bytes
HTML transferred: 33596000 bytes
Requests per second: 54.74 [#/sec] (mean)
Time per request: 36.536 [ms] (mean)
Time per request: 18.268 [ms] (mean, across all concurrent requests)
Transfer rate: 1814.34 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.1 1 3
Processing: 32 35 0.3 35 37
Waiting: 31 34 0.3 34 37
Total: 33 36 0.3 36 38
Percentage of the requests served within a certain time (ms)
50% 36
66% 37
75% 37
80% 37
90% 37
95% 37
98% 37
99% 37
100% 38 (longest request)
Test 3 (Static Image File)
Loading a non-PHP file (image file) with the Lua code processing the request.
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /wp-content/uploads/2014/08/mam01_18197059-480x345.jpg
Document Length: 29640 bytes
Concurrency Level: 2
Time taken for tests: 0.796 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 29944000 bytes
HTML transferred: 29640000 bytes
Requests per second: 1255.89 [#/sec] (mean)
Time per request: 1.593 [ms] (mean)
Time per request: 0.796 [ms] (mean, across all concurrent requests)
Transfer rate: 36724.84 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.1 1 2
Processing: 0 0 0.0 0 0
Waiting: 0 0 0.0 0 0
Total: 1 2 0.0 2 2
Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 2
95% 2
98% 2
99% 2
100% 2 (longest request)
Test 4 (Static Image File, FW Disabled)
Same as the previous, loading a non-PHP file without the Lua code being executed.
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /wp-content/uploads/2014/08/mam01_18197059-480x345.jpg
Document Length: 29640 bytes
Concurrency Level: 2
Time taken for tests: 0.765 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 29944000 bytes
HTML transferred: 29640000 bytes
Requests per second: 1307.11 [#/sec] (mean)
Time per request: 1.530 [ms] (mean)
Time per request: 0.765 [ms] (mean, across all concurrent requests)
Transfer rate: 38222.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.1 1 2
Processing: 0 0 0.0 0 0
Waiting: 0 0 0.0 0 0
Total: 1 1 0.0 1 2
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 2
80% 2
90% 2
95% 2
98% 2
99% 2
100% 2 (longest request)
Test 5 (Block A Query String)
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /blog/2016/12/test-inverse-block/?t=wp-config.php
Document Length: 166 bytes
Concurrency Level: 2
Time taken for tests: 0.918 seconds
Complete requests: 1000
Failed requests: 0
Non-2xx responses: 1000
Total transferred: 311000 bytes
HTML transferred: 166000 bytes
Requests per second: 1088.83 [#/sec] (mean)
Time per request: 1.837 [ms] (mean)
Time per request: 0.918 [ms] (mean, across all concurrent requests)
Transfer rate: 330.69 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.0 1 2
Processing: 0 0 0.0 0 1
Waiting: 0 0 0.0 0 1
Total: 2 2 0.1 2 3
Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 2
95% 2
98% 2
99% 2
100% 3 (longest request)
Test 6 (Block based on Query String, PHP Code)
Blocking done at the PHP level was done by the lightweight WordPress security plug-in BBQ: Block Bad Queries.
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.pointsinfocus.dev (be patient)
…
Server Software: nginx
Server Hostname: www.pointsinfocus.dev
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
Document Path: /blog/2016/12/test-inverse-block/?t=wp-config.php
Document Length: 0 bytes
Concurrency Level: 2
Time taken for tests: 5.787 seconds
Complete requests: 1000
Failed requests: 0
Non-2xx responses: 1000
Total transferred: 160000 bytes
HTML transferred: 0 bytes
Requests per second: 172.79 [#/sec] (mean)
Time per request: 11.575 [ms] (mean)
Time per request: 5.787 [ms] (mean, across all concurrent requests)
Transfer rate: 27.00 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 1 0.1 1 3
Processing: 9 10 0.2 10 11
Waiting: 9 10 0.2 10 11
Total: 11 12 0.2 12 13
Percentage of the requests served within a certain time (ms)
50% 12
66% 12
75% 12
80% 12
90% 12
95% 12
98% 12
99% 12
100% 13 (longest request)
Preliminary Conclusions
| Benchmark | Requests Per Second | Time per request (mean) [ms] |
|---|---|---|
| PHP + Lua | 54.11 | 36.961 |
| PHP | 54.74 | 36.536 |
| Static + Lua | 1255.89 | 1.593 |
| Static | 1307.11 | 1.530 |
| Bad Request + Lua | 1088.83 | 1.837 |
| Bad Request PHP | 172.79 | 11.575 |
With the PHP test running Lua added an additional 425 microseconds of latency. In the tests against static files, lua added 63 microsecond of latency.
The real big gain is in the blocking performance, which is an order of magnitude faster than handing things off to PHP. The main benefit of this is that Nginx handles connections asynchronously without tying up backend processes.
Even doubling the latency imparted by the Lua code imposes 3 orders of magnitude less overhead than dropping to PHP, or at least dropping to Wordfence in PHP.