varnishcache/varnish-devicedetect

VCL based device detection for Varnish Cache.

Device detection in Varnish

Introduction

The goal of this VCL set is to provide a simple & easy way of getting device detection going in Varnish Cache. (https://www.varnish-cache.org)

Specific problems we want to solve:

1. Detect the most common and easily detected (mobile/tablet) platforms.
2. Provide example VCL for using devicedetect.vcl with your existing VCL.

These are explicit not goals for this project:

• Create a comprehensive set of capabilities per client. We don't want to spend a lot of effort on maintenance.
• Be perfect. It usually works, but if you need guarantees you should consider the commercial offerings instead.

This project is maintained and updated by the community. If you see any false positives or missing strings, fork the git repository and send a pull request.

Requirements

You need a recent Varnish release. It may function with some adjustments on previous versions, but you are on your own.

It is worth noting that there is no compilation/linking required. This is VCL code only.

Your backends needs to be able to do produce different content based on what type of device this is. This can be signalled to the backend through the URL, with an extra HTTP header or with a cookie. See the INSTALL.rst file for examples.

Use cases

The following uses are envisioned for this VCL:

• Serve different content per device type on the same URL. This has the upside of not requiring a redirect on slow mobile networks.
• Redirect mobile/tablets to a different URL. (http://example.com/article/1234.html -> http://m.example.com/article/1234.html)

Installation

See the INSTALL.rst file for details.

Similar efforts

These similar efforts for User-Agent insights are known to us:

• https://deviceatlas.com/ (commercial)
• https://github.com/OpenDDRdotORG (free)
• https://www.varnish-cache.org/vmod/deviceatlas-mobile-detection (commercial VMOD for Deviceatlas by Varnish Software)
• https://github.com/TheWeatherChannel/dClass
• https://github.com/serbanghita/Mobile-Detect
• https://github.com/willemk/varnish-mobiletranslate (Generate VCL from Mobile-Detect data)

Contact

This project lives on Github:

https://github.com/varnishcache/varnish-devicedetect/

Feature requests, bug reports and such can be added to the Github issue tracker.

This code is currently maintained by Christopher Slowe <[email protected]>.

varnish-devicedetect's Issues

Google Mobile bot

Google has 'mobile' crawlers that identify as smartphone and feature phone. These rules are not classifying it as mobile.

Reference: https://developers.google.com/webmasters/mobile-sites/references/googlebot

It looks like there isn't a definition for a mobile-bot.

[regex] detection logic based on Mobile_Detect.json

Hi,

I'd suggest improving the detection getting the detection logic from https://github.com/serbanghita/Mobile-Detect/blob/master/Mobile_Detect.json

It's just an idea, we made that .json to allow people to better port the script. We update it also on every release.

WURFL and Varnish Cache

Hi

You may want to add a reference to WURFL and ScientiaMobile as one of the possibilities to enrich Varnish-Cache with Device Detection abilities:

http://www.scientiamobile.com/blog/post/view/id/25/title/HTTP-and-Mobile%3A-The-Missing-Header-

Luca Passani

Performance Benchmark?

Greetings,

I really want to try this out. I have a Symfony app running behind Varnish and the performance is really good, but as always, it can be be better.

All requests must reach the app then, depending on certain parameters, a specific response is generated. I cache db heavy fragments of the page using ESI. One of those parameters is Device detection. I currently use https://github.com/serbanghita/Mobile-Detect.

I have been profiling my app and the response times for the home page is 179ms, a whooping 125ms of which is for device detection.

I am curious to the performance of mobile detection using Varnish, has anyone ever benchmarked it out? Of course, written in C should make it really fast but how much? I know the best way is to profile myself but you see, it will mean changing my app code entirely which will take days, I was wondering if anyone had rough estimates/ideas.

Regards.

Microsoft Surface handling

I wrote this on a comment on the commit 17c109a but it might be better to do it here:

This pattern is not correct. The second [] should be ().
The architecture is only valid on IE 10 under Windows 8 or later.

This needs to be studied a bit further but I don't think this change is correct.

`Set-Cookie: ..SameSite=None` on supported browser

Hi,
I'm looking for sharing an existing VCL which parses User-Agent to detect if the browser supports the SameSite=None i.e. Set-Cookie: x=y; SameSite=None.

That's required to be set to allow 3rd party Cookies for the next Chrome version 80; see https://www.chromium.org/updates/same-site/incompatible-clients going mainstream in a couple of days.

I don't know if this repository is the right one, or if you have other suggestions ?

Need to differentiate between mobile bot and desktop bot

Right now all bots are grouped together into a single X-UA-Device of 'bot'. But Google, for example, has bots that are specifically used for crawling mobile sites. See:

https://developers.google.com/webmasters/smartphone-sites/googlebot-mobile
http://googlewebmastercentral.blogspot.com/2014/01/a-new-googlebot-user-agent-for-crawling.html

I'm not sure of whether there are other "mobile bot" user agents, but it might make sense to have a 'mobile-bot' X-UA-Device.

FirefoxOS detection

Hi,

Could you please add FirefoxOS to list of mobile devices.

According to Mozilla, User-Agent is:

  Mozilla/5.0 (Mobile; rv:15.0) Gecko/15.0 Firefox/15.0

Please, avoid confusion with Firefox for Android:

  Mozilla/5.0 (Android; Mobile; rv:15.0) Gecko/15.0 Firefox/15.0

Thank you very much.

Tablet PC in varnish-device detect

I compared varnish-device detect method with UA Parser and I noticed that varnish method not detected Tablet PC devices as a mobile (where regexp_like(useragent, '(Tablet._PC|PC._Tablet)', 'i')).

I uploaded sample of user agents on the gist:
https://gist.github.com/pongacm/b7775bebf2b26c09e1f8

Wikipedia says that tablet PC is a tablet:
http://en.wikipedia.org/wiki/Microsoft_Tablet_PC

I'm analyst so I can't prepare pull request. Is any chance to improve varnish-device detect by somone with you?

Why 'pc' type instead of the usual 'desktop' one?

Well, it's obviously not an issue but I'm curious about this choice. Any particular reason?

Add more User-Agents

For more user-agents you can use following device-detect.vcl from repo:
https://github.com/miniyarov/varnish-mobiletranslate

If there is a remote backend, the front server cache nothing

I am using Varnish 6.2.1 + Ubuntu 20.04.

Wrong X-UA-Device

Hi, we're using this VCL and we get wrong detections in several computers.
Sometimes it detects chrome on windows as smartphone and sometimes it detects android chrome as a desktop.
Is this a known issue or something went wrong with the implementation?

Facebook's iPad in-app browser is detected as mobile I/o tablet

Despite Facebook using 'iPad' in their user agent, the Facebook browser is detected as being a mobile device instead of a tablet in the vcl.

new google mobile-bot

Hi,
according to this page:
https://support.google.com/webmasters/answer/1061943
google has a new mobile bot, maybe (as per our analysis) since April 19th

In order to catch it, I've made a little change in the vcl, this one:

43c43

< (req.http.User-Agent ~ "iPhone" && req.http.User-Agent ~ "(compatible; Googlebot/2.1; +http://www.google.com/bot.html")) {

        (req.http.User-Agent ~ "(iPhone|Android)" && req.http.User-Agent ~ "\(compatible; Googlebot/2.1; \+http://www.google.com/bot.html")) {

let me know if you need a pull request

thanks,
corrado

iOS User-Agent used in OS process retrieving icons for “Add To Home Screen” not recognized as a mobile device.

I just stumbled upon a piece of weird behaviour in the latest iOS release.

We are using varnish to redirect specific urls when accessing them with a smartphone. This works perfectly except when using the iOS “Add to Home Screen” option for bookmarking pages as icons on the app home screen. The bookmark works as expected, but the icon for the app (a link to which is defined in the source of the bookmarked html page) does not.

Safari “correctly” identifies itself using this User-Agent header:

Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D167 Safari/9537.53

Which is caught by the Varnish devicedetect rule just fine, but when inspecting the requests coming out of the iPhone upon using “Add to Home Screen” the User-Agent header for the requests for the icons was:

MobileSafari/9537.53 CFNetwork/672.1.13 Darwin/13.1.0

Probably because the requests for the icons are handled by the OS rather than Safari (my guess), but the effect is that this evades the devicedetect rule altogether and causes the requests for the icons to 404.

Licence?

Hi,

I'm about to use this code at my company. Could you specify a licence under which we can use it?

Thanks.