User-Agent Sniffing Only Way to Deal With Upcoming SameSite Cookie Changes

Lars Eidnes
ITNEXT
Published in
10 min readJan 26, 2020

--

On February 4th 2020, Chrome will introduce changes to the handling of SameSite cookies that require changes to anyone taking in cookies from cross-origin requests. The cookie format required by the new Chrome cannot be read by a significant chunk of older browsers.

Photo by Noah Buscher on Unsplash

One proposed way to deal with this is to set two versions of each cookie, but as explained below this approach is infeasible for many apps. This leaves user agent sniffing as being both the most error prone practice in web development as well as the only compatible approach to cross-origin cookies. For this reason, we’ll conclude this post with a comprehensive guide to getting this user-agent sniffing right, synthesizing the recommendations from the major players in a way that tackles the full list of incompatible browsers, and validating it on a dataset of more than 40000 unique real-world user-agent strings in use today.

The web developer community has done its best to avoid user-agent sniffing in JavaScript for over a decade, but server-side user-agent sniffing has hardly been done since the 90s, and even then it felt bad. Now it’s back, and for certain use cases it’s mandatory.

First let’s look into how this came about. If you just want to see how to reliably fix this, scroll on down.

How did we get here?

  • Many implementations of third-party logins, like OAuth, require you to take a request initiated from another site, and read a cookie in that request.
  • The 2019 SameSite standard mandates that cookies should not be sent in POST requests from other sites, unless they are marked with SameSite=None; Secure. If this property is not set, the cookie will default to SameSite=Lax;, meaning no cookie will be sent on cross-origin POSTs. SameSite=None must be set along with the Secure attribute, meaning these request must happen over HTTPS. This will be implemented in Chrome 80, which will be auto installed for users on February 4th, 2020. Firefox and Edge have signaled they will adopt these changes as well, but have no set release date yet.
  • An April 2016 draft of the SameSite standard doesn’t know about SameSite=None, and specifies that when unknown values are encountered (e.g. "None"), the cookie should be ignored. This is implemented in Chrome 51-66, and anything deriving from it, like Android WebView or Chromium. Some versions of UC Browser < 12.13.2 on Android also has this behavior.
  • A January 2016 draft of the SameSite standard specifies that unknown SameSite values (e.g. “None”) should be treated as being SameSite=Strict. This behavior is implemented on any browser on iOS 12 and Safari on MacOS 10.14 (Mojave). The Chrome team insist that this behavior is a bug, but it is actually in line with this particular version of the spec.
  • For any browser on iOS 12 and Safari on MacOS 10.14, this behavior is tied to the OS — users must update the OS to get the new behavior. Apple will not be back-porting a fix to iOS 12. Users of iPhone 5S, iPhone 6/6 Plus, and the 6th generation iPod Touch that are on iOS 12 cannot upgrade to iOS 13 in any case, so they must buy a new device to get the new cookie behavior.

What this means for cross-origin cookies

  • If you need to read cookies from cross-site requests in Chrome 80, you need to explicitly set them with SameSite=None; Secure. On its own, this is a breaking change in cookie behavior.
  • If you need to read cookies from cross-site requests on any browser on iOS 12, Safari on MacOS 10.14, Chrome 51–66 or UC Browser < 12.13.2 for Android, you need to set them without the SameSite attribute.
  • These two behaviors are incompatible. Because a long list of browsers treat SameSite=None either as SameSite=Strict or as something to ignore, there is no way to set a cookie that will be sent on cross-site requests that works both these and for Chrome 80. You must do user-agent sniffing when setting it, or go for a double cookie approach, as explained below. What works in one browser, will break in others.
  • If you’re using third party authentication via OpenIdConnect, SAML 2.0, WsFederation or anything else where you need to read a cookie that is posted to your site in a request initiated from a different site, you’re impacted by this.

Why did this happen?

The point of the changes introduced by Chrome is to get stronger Cross-Site Request Forgery protections by default. These are breaking changes on their own — no matter what older browser were doing, the changes pushed by Chrome mean that SameSite=None; Secure would have to be set for intentional cross-site cookie passing to work with POST requests. These changes drastically reduce the chance of accidentally introducing CSRF vulnerabilities, so they clearly have some value. Had this been introduced in Netscape 1.0, advocating for it would be a no-brainer. When these breaking changes are introduced a quarter of a century after the introduction of cookies, there are clearly a lot of things that will break as a result.

The bigger problem is that it turned out that older versions of the spec were incompatible with the new one. A bunch of older Chrome versions, and all browsers on iOS 12 follow versions of the 2016 standard were SameSite=None is either ignored or treated as SameSite=Strict. The author of the 2019 spec was also an author of the 2016 specs, but it's not clear whether the mutual exclusivity of these specifications were understood.

Browser usage shares

According to w3counter, 4.75% of web traffic comes from an iOS 12 device. Caniuse.com provides break downs of usage shares per Chrome version, and say Chrome 51–66 make up 0.75% of web traffic in total. UC Browser is listed as having 0.3–2.9% global market share depending on the source, although this varies greatly from country to country. For example, it is reported to have 22% share in India. For UC Browser, the sources do not differentiate by version number. In summary, using statistics from December 2019, about 6% of web traffic will be unable to handle SameSite=None; Secure. Coming February 4th, auto-updating Chrome installations, making up about 55% of web traffic, will refuse to pass along cookies across origins if they are not set with SameSite=None; Secure.

Who needs to care about this?

First, think about whether or not you ever need to read cookies from third-party POSTs. If you don’t, you can set cookies to SameSite=Lax (or perhaps leave it unset, which will default to SameSite=Lax behavior going forward). If you need to protect against cross-site GET requests, go with SameSite=Strict.

If you actually need cookies in POSTs originating from other sites, you need to take special steps. Two ways to handle this are explained in what follows below.

A fix with caveats: Double cookies

One way to fix this is to read and write two different cookies, one with SameSite=None; Secure, and one without, and check for both in your cookie handling. Your HTTP would look something like this.

HTTP/1.1 200 OK
Date: Fri, 17 Jan 2020 10:10:01 GMT
Content-Type: text/html; charset=utf-8
Set-Cookie: myCookie1=value; SameSite=None; Secure
Set-Cookie: myCookie2=value; Secure

This may well be a viable solution for you, but are a couple of reasons why this might not be a good idea, and you should think twice about whether or not it is.

  1. One of the browsers you’re targeting, Safari on iOS 12, has hard limits on cookie size per domain. Safari allows about 4KB in total per domain. If you’re setting two cookies, you’re halving the available space, and chances are good you’ll exceed this limit. It all depends on what you’re storing in the cookie.
  2. Changing how a cookie is set has some complexity to it but is likely to be centralized to one location. However, changing where a cookie is read might be spread out over a larger portion of the code base. You also need to change any place where cookies are deleted, and you need in every case to make sure you handle each of the cases where either one cookie is present, the other cookie is present, or both, or neither. In all, there is a lot of complexity to this, and it’s complexity that’s likely related to authentication. That is not where you want to hack around and take chances.

While the double cookie approach is what is recommended by Google, the ASP.NET team concluded otherwise — they found that going with a user-agent sniffing approach would be the safer approach. User-agent sniffing is hard to get right. It’s hard to cover all the relevant browsers, and it’s hard to do so in a way where you can be reasonably sure it won’t break in user-agents of the future. But when someone gets it right, that becomes a readily copiable solution that anyone can use.

The other fix: User-agent sniffing

Parsing user-agent strings is hard, because it doesn’t actually follow any particular format, and also because every browser lies about who they are, and states that it is a lot of other browsers as well as itself. The reason browsers lie is to get around bad code that does the wrong thing when parsing user-agent strings. So because user-agent string parsing was hard originally, the universe conspired to make it even harder now. There are really no rules for how to parse them, and you must rely on statistical evidence to see if you got it right. You need to check your code against known user-agent variants, and be specific enough about checking version numbers that you can be confident it won’t break in future releases.

There’s good reasons why we generally avoid user-agent sniffing at all cost. However, these SameSite issues are an extreme where it may be the solution causing the least breakage.

Being a JavaScript error logging service, CatchJS has a lot of data on user agents, and some expertise in parsing them. What follows are some recommendations based on this. First we’ll outline how to reliably detect any browser on iOS 12, Safari on MacOS 10.14, and Chrome versions 51–66. Then we’ll expand on this to include the full list of known incompatible browsers.

  • You can reliably detect iPods and iPhones running iOS 12 server side by looking for the substring “iPhone OS 12_” in the user agent.
  • You can reliably detect iPads running iOS 12 server side by looking for the substring “iPad; CPU OS 12_”.
  • The above two checks will also reliably detect Chrome on iOS 12. It also generally includes web views on iOS 12, including at least the apps from Facebook, Instagram, Snapchat and Shopify.
  • Safari on MacOS 10.14 can be reliably detected by finding all of the substrings “ OS X 10_14_”, “Version/” and “Safari”.
  • You can avoid SameSite=None on Chrome 50-69 by checking for substrings "Chrome/5" and "Chrome/6". We only really care about Chrome 51-66, but including a few more versions is fine, since these don't do anything with SameSite=None in any case. It will also include Chrome >= 500. Assuming the current release schedule with a major version every eight weeks, you have about 64 years before that is a problem.

The above code detects almost all SameSite=None incompatible browsers, and is fine if you want a low-complexity solution that covers 99.X% of browsers in use. If you want to cover the long tail of rare browsers, the following paragraphs shows how.

One browser group ignored above are the older installs of UC Browser on Android. In most markets, this may be fine. Detecting it requires more expensive parsing to pull out the version numbers. The browser will generally automatically update, and has relatively low usage share in most markets, so the share of visits coming from un-updated UC Browsers will for many sites be microscopic. However, in some Asian countries UC Browser is huge, and you might not be able to ignore older installs. In that case, you can use the code below to reliably detect the installs that shouldn’t get SameSite=None cookies.

After including the check for UC Browser, we’re still leaving out older installs of Chromium, and embedded web views on MacOS 10.14. These are fringe, but they can be detected. Chromium can be detected similiarly to how we detected Chrome, and the Mac embedded browser can be detected by checking that the user-agent string includes “ OS X 10_14_” and ends with “(KHTML, like Gecko)”. In the end that gives the somewhat long, but complete check below.

We can compare with user-agent sniffing recommendations given by other sources.

The ASP.NET blog provides sample code for doing this that is used by the Azure Active Directory team. It is much inline with our lightweight version above, in that it uses substring checks and covers only the older Chrome, iOS 12 and MacOS 10.14 cases.

The Chrome team provides a pseudo code implementation of user-agent sniffing, that detects the full list of known incompatible browsers. It relies on running multiple regular expressions to extract and read version numbers. A quick informal benchmark showed it to be about 2.5 times slower. This may not matter at all, depending on your load volume.

We compared results from Google’s version and our version on a dataset of 40000 unique real-world user-agents. They differ only in the fringe of the fringe: For example, Google’s regex requires a space after Chrome/[versionNumber], and therefore won’t recognize the (hardly used) Chromium derivative Vivo Browser 6.0, which puts Chrome/[versionNumber] at the end of the user-agent string. Our version requires an underscore in “iPhone OS 12_” to future proof against iOS 120, but therefore misses the Google Maps app’s embedded web view on iOS 12, which reports itself as “iPhone OS 12.n” (with a dot). The usage share of either of these is essentially zilch. Figuring out the trade-off between supporting a future iOS 120 vs supporting Google Maps web view on iOS 12 vs full version number parsing is left as an exercise for the masochistic reader. For most, either solution is good enough.

Originally published at https://catchjs.com.

--

--