In this article I show a few ways to test CDN setups, changes in hosting, etc before sending real users to the new setup.
Example scenario: example.com does not currently use a CDN. They serve their CSS file using the link https://static.example.com/style.css . Now they go to a CDN provider, and setup such that static.example.com
is served by the CDN.
To allow the CDN to serve the files, the site owner must delete the existing DNS record for static.example.com
and make a CNAME to the one given by the CDN provider, say example.com.clients.somecdnprovider.net
There are few ways to proceed from here.
Method 1: The dumb way
Make static.example.com
CNAME to example.com.clients.somecdnprovider.net
. Flush the browser cache. Make sure DNS changes have propagated to your ISP/computer (use ping to check the IP address). Refresh a few times. See everything is in order... and then pray that the site is not broken for your customers. Monitor your mailbox and Twitter for complaints. Nobody saying anything bad? Then you are probably fine. Congratulations!
Method 2: Adjust hosts file
Rather than point your live traffic to something untested, its nice to run through some "what if" scenarios.
- Find the IP of
example.com.clients.somecdnprovider.net
- Map that IP to
static.example.com
in your hosts file - Clear browser cache and OS DNS cache
- Browse around your site keeping your browser's Dev Tools open to see if anything is broken
Method 3: The 1337 way!
This is the method I use very often.
First some primer on how HTTP clients work.
When you issue a GET request for https://static.example.com/style.css, many things happen behind the scenes.
- The client gets the IP for
static.example.com
from the OS - The client establishes a HTTP Connection with the returned IP
- The client asks the server for /style.css , and amongst the other request headers, it includes "Host: static.example.com"
- The server uses the information given by the Host header to know which virtual zone it may be mapped to. This is the most important request header.
So, the server the client connects to, and the value of the Host header, is completely under the clients control. We will manipulate this information to our advantage to test the new setup.
My favourite user agent for ad hoc testing is the GET and HEAD commands provided by lwp-request. I also use ApacheBench and curl.
To fetch the example file from the CDN we would issue the following command.
GET -H 'Host: static.example.com' https://example.com.clients.somecdnprovider.net/style.css
Or if you like curl:
curl -H 'Host: static.example.com' https://example.com.clients.somecdnprovider.net/style.css
ApacheBench:
ab -H 'Host: static.example.com' https://example.com.clients.somecdnprovider.net/style.css
What these commands do is a little different from before:
- The client gets the IP for
example.com.clients.somecdnprovider.net
from the OS - The client establishes a HTTP Connection with the returned IP
- The client asks the server for /style.css , and amongst the other request headers, it includes
Host: static.example.com
- The server uses the information given by the Host header to know which virtual zone it may be mapped to. This is the most important request header.
So, now you are connecting to example.com.clients.somecdnprovider.net
but overriding the Host header to static.example.com
. This is exactly the same behaviour if you had made the DNS change and a real user was requesting the file.
Some other checks I do before going live:
Cacheability
Some CDNs include the cache information in HTTP response headers. For them examine the headers:
HEAD -H 'Host: static.example.com' https://example.com.clients.somecdnprovider.net/style.css
Run this command many times to prime the cache. On cache HITs, CDNs usually include the following headers:
CloudFront:
X-Cache: Hit from cloudfront
NetDNA/MaxCDN:
X-Cache: HIT
Internap - Any one of them needs to have a HIT
X-Cache: HIT from cdce-sin002-002.sin002.internap.com
Level3 - Does not indicate; try making requests and monitoring your access log
BitGravity - Does not indicate; try making requests and monitoring your access log
Fastly
X-Cache: HIT
Test this over and over. Test for query string behaviour. If you set the CDN to ignore query strings, it should be mostly HITs, even if you try random query strings. On the other hand, if you've set the CDN to treat query strings as unique, each time you try a different query string, it should be a MISS.
Compression
Test compression by manipulating the Accept-Encoding header. Look for Content-Length and Content-Encoding response headers. Make sure you are testing a compressable object like CSS/JS/HTML:
HEAD -H 'Host: static.example.com' -H 'Accept-Encoding: gzip' https://example.com.clients.somecdnprovider.net/style.cssand
HEAD -H 'Host: static.example.com' https://example.com.clients.somecdnprovider.net/style.css
Should provide different results for encoding and length.
Bonus tip
Test your CDN from all over the world.
Head on over to https://whatsmydns.me/, enter example.com.clients.somecdnprovider.net
make a note of all the IPs. For each IP, do the following:
GET -H 'Host: static.example.com' https://the-returned-ip/style.css
You can run the all the above tests on all the IPs returned for a more thorough testing.
Note: This does not work for Anycast based CDNs like StackPath or Cloudflare.
Only when you are completely confident make the real switch in your DNS.
As far as I remember, this method was first introduced to me by Kirit Sælensminde years ago for a different purpose.
Note: If you are behind an ISP that runs a transparent proxy, methods #2 and #3 are not accurate unless you know how to circumvent the proxy.
Method 4: webpagetest.org FTW
While the command line methods are very powerful, and my method of choice for these tests, it may be a little complex for some people unfamiliar with the command line. Webpagetest.org to the rescue!
By using the setDNSName command, you can fool the test agent/browser into thinking that the CNAME switch has already been done. Based on the above scenarios, the script needed would be:
setDnsName static.example.com example.com.clients.somecdnprovider.net navigate https://example.com/
Make sure to run multiple tests from each location, and check caching and compression headers.