CDN Finder is a tool that enables you to quickly and easily find out which CDN(s) a website is using, or identify the CDN behind a hostname.
CDN Finder has been quite popular since its initial launch in 2011, but unfortunately, for the past few years the CDN Finder tool did not provide a great service. Too often, the tool was broken and users would get the "Oops, something went wrong" message :(
Also, when the tool did work, the results were not always good: it either wrongfully didn't detect a CDN, identified the wrong CDN or presented the old name of the CDN (e.g. Level 3 instead of CenturyLink).
The good news is: all those problems are gone now! The CDN Finder tool has been completely rebuilt from scratch, resulting in a reliable, fast and accurate service.
Read on to learn how we made CDN Finder better.
CDN Finder is Closer To The User
Not only the CDN Finder, but the whole CDN Planet website was rebuilt in May 2020 and runs on Cloudflare Workers Sites.
Cloudflare has POPs in 200 cities across 90+ countries, so most CDN Planet users are just milliseconds away from the nearest edge location.
The CDN Finder runs entirely at the edge in a Worker and needs less than 1 millisecond to execute, so the total time of a CDN Finder run really depends on how fast the Worker can fetch responses from remote websites/services.
Fast, Efficient Page Analysis
The old CDN Finder loaded and rendered the page in a headless browser. This was nice because CDN Finder had full visibility into everything that was loaded into the page, including third-party content/trackers/widgets/ads/etc. As result, CDN Finder could provide an answer to the question "are all resources on this page served from a CDN?".
While the answer to that question provides value to some users, the full page render step adds technical complexity to CDN Finder, makes the tool more error prone and - as it turns out - is not absolutely required to provide a good answer to the primary question: "which CDN is this website using?".
The new CDN Finder leverages the power of Cloudflare's HTMLRewriter, an awesome and fast API to parse the HTML of a remote web page.
preload and a
Next, CDN Finder uses this list of page resources to identify the CDN for each unique hostname:
- Fetch the page resource and inspect the response headers
- Perform a DNS lookup and inspect the full CNAME chain
In most cases, this is sufficient to accurately detect the CDN, but only after we greatly improved our 'Header-to-CDN' and 'CNAME-to-CDN' lists:
Improved CDN Matching Lists
Next, we needed to tackle the 'server header problem': because CDN Finder now runs on Cloudflare in a Worker, every URL that is fetched by CDN Finder comes back into the Worker through the Cloudflare CDN cache and so the response always has the Cloudflare CDN headers, including
server: cloudflare. This is unfortunate because for quite a few CDNs (including Cloudflare!) the CDN Finder relied on the value of that header to identify the CDN.
We solved this problem for all impacted CDNs by finding a different uniquely identifying header (e.g.
lswcdn-country-code for Leaseweb) or by improving our CNAME chain inspection logic. But for Cloudflare, we needed something extra.
Identify the CDN by IP Address
When CDN Finder can't detect the CDN from the response headers or from the CNAME chain, the last step is to lookup the name of the network the server IP(s) belong to.
IPtoASN also runs on Cloudflare so lookups from CDN Finder are super fast.
Soon after implementing this IP-to-ASN lookup and test driving the new CDN Finder with many different sites, we discovered this is not just a solution for detecting Cloudflare, but a necessary step in accurately detecting other CDNs in specific cases.
Our current ASN-to-CDN list is this:
On The Roadmap
We'll continuously improve CDN Finder based on user feedback, analysis of the CDN Finder logs and market developments (e.g. a new CDN enters the market).
Three items are currently on our todo list:
- Identify popular hosting services (e.g. AWS S3) and label these as 'not a CDN'
- Better inform the user what happened in case of error
- Publish a changelog
Have an idea for how we can make CDN Finder even better? Let us know on Twitter
CDN Finder is the first service that is built on top of the CDN Planet API.
Do you want API access to CDN Finder, for example to do batch processing? Send an email to email@example.com