Tag Archives: varnish

Varnish A/B Testing

Cloudflare doesn’t cache any of our PHP files so don’t need to worry about that. Cloudflare connects to our nginx SSL reverse proxy.

nginx has a split_clients directive, so could potentially add header there:

Performing A/B Testing with NGINX and NGINX Plus

The SSL reverse proxy connects to a varnish server on the same system.

Varnish can also set the a/b testing header:
https://info.varnish-software.com/blog/live-ab-testing-varnish-and-vcs

Where ever the header is set, Varnish will need to Vary based on that header.

Varnish then connects to nginx, which then connects to the php-fpm server(s). At this point, there needs to be a different wordpress instance loaded, and there are a couple different ways.

The main idea I have now is to bind mount different plugin and theme directories on top of a base directory. The base directory contains the basic wordpress install, and the images, etc (basically everything not themes/plugins). One easy way to do this is with containers, but you should be able to do it outside containers as well.

No Containers Method:
– bind mount base directory to branch directory (ie /vhosts/base -> /vhosts/test-a)
– bind mount plugins and theme directories on top (ie /usr/src/test-a/plugins -> /vhosts/test-a/wordpress/wp-content/plugins)
– map header to root directories in nginx: (needs to be in http block)
https://serversforhackers.com/c/nginx-mapping-headers
#should use mapping so that there is always a default (don’t trust header)
map $http_ab_group $ati_ab_dir {
default “all-that-is-interesting”;
a “test-a”;
}
server {
server_name allthatsinteresting.com;
server_name www.allthatsinteresting.com;
root /vhosts/$ati_ab_dir;
}

Containers Method:
– switch nginx config to use containers (basically need to use a reverse proxy instead of fastcgi)
– which reverse proxy to use would be mapped in from the header
– containers would have the base dir, plugins dir, and themes dir mounted as volumes

Containers method would require more work, and the worker pool will be segmented among the php-fpm backends, so it’s more difficult to size the max workers per container. Like say the server has memory to handle 128 workers, if you have 3 separate worker pools, do you assume that all 3 pools will be maxed, or do you oversubscribe since chances are the load will be uneven? It’s simpler and safer to just have a single PHP backend. (not to mention the added complexity of Docker)

using docker to build Varnish VMODs

Docker is containers. Let’s use it to create a build environment we can delete after we’re done. Part one is figuring out what we need to build vmods.

I like using Fedora:
mkdir /tmp/vmods
docker run -v /tmp/vmods:/data -it fedora:24 bash
dnf install -y git
dnf install -y autoconf automake libtool m4 perl-Thread-Queue make
dnf install -y varnish-libs-devel.x86_64 varnish
# dnf install -y dnf-plugins-core /usr/bin/rpmbuild
cd /usr/src
# dnf download --source varnish # no longer have to build Varnish from source to compile VMODs
# rpm -ivh varnish*src.rpm
# dnf install -y groff jemalloc-devel libedit-devel ncurses-devel pcre-devel python-docutils

dnf install -y GeoIP-devel.x86_64
git clone https://github.com/varnish/libvmod-geoip.git
cd libvmod-geoip
bash autogen.sh
./configure VMOD_DIR=/data
make
make install

dnf install -y libmhash-devel
git clone https://github.com/varnish/libvmod-digest
cd libvmod-digest
bash autogen.sh
./configure VMOD_DIR=/data
make
make install

So we split that into a docker file and a script to build the vmod:

FROM fedora:24
RUN dnf update -y
RUN dnf install -y git autoconf automake libtool m4 perl-Thread-Queue make python-docutils
RUN dnf install -y varnish-libs-devel.x86_64 varnish

ADD build_vmods.sh /

CMD bash /build_vmods.sh


OUT_DIR=/data

usage() {
echo github URL must be set for each VMOD
exit 1
}
build() {
[ -z "$URL" ] && usage
[ ! -z "$DEPS" ] && dnf install -y $DEPS

src_dir=$(mktemp -d)

git clone $URL $src_dir
cd $src_dir
bash autogen.sh
./configure VMOD_DIR=$OUT_DIR
make
make install
}

DEPS="GeoIP-devel.x86_64"
URL="https://github.com/varnish/libvmod-geoip.git"
build

DEPS="libmhash-devel"
URL="https://github.com/varnish/libvmod-digest"
build