Set up A/B testing using Nginx

Kumar Chetan Sharma
2 min readApr 8, 2024

A/B testing, also known as split testing, is a powerful method used by businesses to compare two versions of a webpage or app to determine which one performs better. It’s an essential tool for optimizing user experience, increasing conversions, and maximizing results. There are several A/B testing tools available, such as Google Optimize, Optimizely, VWO, or even custom solutions. However, all of these may cost you.

With Nginx, a rudimentary A/B testing system can be set up. The basic concept is very simple

  • Divide traffic into buckets
  • Set cookies for these buckets
  • Pass these cookies with every request to and from your application
  • Basis on these cookies modify your application logic

Nginx has a module named ngx_http_split_clients_module. As per the documentation this module

creates variables suitable for A/B testing, also known as split testing

This module does the main work, divide traffic into buckets. Here is an example:

  split_clients "${remote_addr}" $cookie_group {
50% "A";
50% "B";

In this example, the incoming requests or traffic is divided into two groups, A and B. Once this is done, next steps are easy

set $ab_cookie_value '';  

if ($cookie_group = "A") {
set $ab_cookie_value 'AB_cookie=A; Path=/';
if ($cookie_group = "B") {
set $ab_cookie_value 'AB_cookie=B; Path=/';

# Send cookie to application
proxy_set_header Cookie "$ab_cookie_value";

# Send cookie to client
add_header Set-Cookie "$ab_cookie_value";

Set all this in your http or server context. Or in plain English, this goes inside your site configuration file.

This is a simple setup and may not scale well. In case you have Cloudflare set up checkout this article which talks about using Cloudflare workers for A/B testing.



Kumar Chetan Sharma

C0d3🐵 from 🇮🇳. selfTaught. 😨 of CS, Algos & DS. World famous 👨‍🍳 at home. Pro Level Pro-Crastinator. Man child. Dad of (✿◠‿◠)(◠‿◠✿)