Set up A/B testing using Nginx
A/B testing, also known as split testing, is a powerful method used by businesses to compare two versions of a webpage or app to determine which one performs better. It’s an essential tool for optimizing user experience, increasing conversions, and maximizing results. There are several A/B testing tools available, such as Google Optimize, Optimizely, VWO, or even custom solutions. However, all of these may cost you.
With Nginx, a rudimentary A/B testing system can be set up. The basic concept is very simple
- Divide traffic into buckets
- Set cookies for these buckets
- Pass these cookies with every request to and from your application
- Basis on these cookies modify your application logic
Nginx has a module named ngx_http_split_clients_module
. As per the documentation this module
creates variables suitable for A/B testing, also known as split testing
This module does the main work, divide traffic into buckets. Here is an example:
split_clients "${remote_addr}" $cookie_group {
50% "A";
50% "B";
}
In this example, the incoming requests or traffic is divided into two groups, A and B. Once this is done, next steps are easy
set $ab_cookie_value '';
if ($cookie_group = "A") {
set $ab_cookie_value 'AB_cookie=A; Path=/';
}
if ($cookie_group = "B") {
set $ab_cookie_value 'AB_cookie=B; Path=/';
}
# Send cookie to application
proxy_set_header Cookie "$ab_cookie_value";
# Send cookie to client
add_header Set-Cookie "$ab_cookie_value";
Set all this in your http or server context. Or in plain English, this goes inside your site configuration file.
This is a simple setup and may not scale well. In case you have Cloudflare set up checkout this article which talks about using Cloudflare workers for A/B testing.