Webviews and User-Agent strings

Much is made of the comparative times spent browsing the web vs engaging with native apps in the apps vs web debate. An often overlooked part of the discussion is that when engaged with a native app some portion of this time is spent actually on the web, via a webview. We’ll get to what a webview is in a minute, but for now, what this means is that although the user is in an app, he or she is effectively browsing the web. Apart from skewing the numbers, or at least muddying them a little, in the app vs web debate, there are some consequences of this for developers and publishers to consider:

• the numbers of web visitors coming via webviews is likely to be significant when we consider popular apps like Facebook and Twitter, and the ease with which links are shared in these apps
• web developers should be considering webviews testing in their release cycle
• app publishers should be applying responsible user-agent adoption practices, or traffic from their apps won’t be identifiable or targetable

In the following sections, we cover what a webview is, how they can be detected, and since, as we’ll see, webview User-Agent strings are a bit of a mess right now, we make some recommendations about how to clean them up!

What is a webview?

A webview is pretty much a browser wrapped inside of an app. Let’s say you’re in Facebook, and a link is shared with you; you tap the link, and instead of switching you over to your browser, Facebook opens the link instead for you. What’s happening is that Facebook is making use of the phone’s webview component to open the link. It is basically the rendering component of the browser, without the UI. Apps such as Facebook like this functionality as it allows the user to view the link without ever leaving the app, and gives the app further user-behaviour signals for user profiling, and of course app authors don’t want users to leave their apps; they might never come back!

Whether or not this is the best experience for the user is up for debate. Some would argue that it makes the experience more seamless. Others would argue that it locks the user into a silo, and costs the user the use of his or her bookmarks, tabbed browsing, and all the other benefits to having control of his or her own browser.

Webview (left) and Chrome browser (right), on Android

The images above show a couple of things. The first shows google.com loaded via Facebook webview, and the second via Chrome browser, both on Android.

This first notable difference is in the UI: Chrome shows the full URL, while Facebook webview shows the page title. This is not too much of a big deal, although it does make changing URL impossible.

Note that the content is different too. Chrome is personalised with all my preferences and cookies. So when I visit a site I’ve been to before, say, one that I’m logged into for example, its state is restored; I don’t need to log in again—I can pick up where I left off. This is why we see personalised content in the Chrome image above. While the webview will handle and retain cookies across sessions, unless you log in again in the webview, you won’t be logged in! And if you do login in on the webview, it’s a different session than the one on your native browser. And, while we’re talking about cookies, we should probably mention you’ve no control over them in the webview anyway!

Webview menu (left) vs browser menu (right)

Now take a look at the second pair of images, this time showing the respective menus of the webview and native browser. The main thing to notice is the difference in control you have. In the native browser you have access to your bookmarks, browsing history, text search and more. In the webview you have access to sharing options, and that’s it. A app purist might argue that this makes the app experience better—you get to see the shared URL and that’s it, everything else is a distraction. But that’s a hard sell. For one thing, wouldn’t it be nice if you could keep track of the sites you visited, just like in the native browser? You can be sure that apps like Facebook are doing just this with links opened in the webview.

Forcing users to use webviews also denies other full-fledged browser benefits. For instance, if you are bandwidth-challenged you might prefer to use the bandwidth-saving page-shrinking features of Opera Mini, or Opera’s Turbo mode. Or if you are privacy-conscious (and you should be!) you might want to disable third-party cookies. These things aren’t possible with today’s webviews.

Detecting webviews

When requesting pages, webviews send User-Agent headers, just like regular browsers. The User-Agent string can be changed by the app. This is important, because it means we can differentiate between the webview traffic and browser traffic, and between the webview traffic from different apps. And this in turn means that we can tell, for example, that a certain portion of traffic came via the Twitter webview, and some other portion came with Facebook webview. Without the ability to change the webview User-Agent string, it would be a lot harder to differentiate webview traffic.

Changing the User-Agent string of a webview is simple. In an Android app for example, you would change it with a line of code something like this:

What do app webview User-Agent strings look like?

Let’s take a look at some real world examples.

Facebook for Android User-Agent string:

This looks like a typical browser User-Agent string, but with some Facebook specifics tacked on the end.

Twitter for Android User-Agent string

This one is unremarkable, going a different route to Facebook, and not including any Twitter specifics at all.

Now compare with the default browser on the same device:

Chrome for Android User-Agent string

There’s a lot of overlap with both the Facebook and Twitter User-Agent strings.

Things get even more interesting when we look at some popular iOS apps:

Facebook for iOS User-Agent string

It’s a lot longer than its Android counterpart, but otherwise the structure is more or less the same: platform webview User-Agent, followed by Facebook data.

Twitter for iOS User-Agent string

Nothing particularily interesting here; it just looks like a normal browser… and that’s why it’s interesting, it doesn’t appear to have any Twitter specifics.

Tweetbot for iOS User-Agent string

Wait, haven’t we seen this one before? So the Tweetbot webview has the exact same User-Agent string as the Twitter webview. Two webviews, one UA!

Let’s take a look at the default browser, Safari:

Safari for iOS User-Agent string

Well, at least with the Version/8.0 included this is distinguisahable from the iOS webview.

Let’s have a look at one more app:

Whatsapp for iOS User-Agent string

This is slightly different again from our previous apps. This time the Version/8.0 substring that distinguished webview from browser in our previous examples is included this time, scuppering our plans. But at least we have an extra appendage of Safari/600.1.4 which could potentially be used to distinguish it.

A brief analysis of webview User-Agent strings

In these examples, we can see the following:

1. The Facebook User-Agent string is quite similar to default browser’s string on both iOS and Android, but with some Facebook identifiers appended
2. On both Android and iOS, the Twitter User-Agent string includes no Twitter information, and looks like it is using the default string for the platform’s webview
3. On Android, the Chrome version listed is different for the browser and webview strings
4. On both Android and iOS, the webviews are distinguishable from the default platform browser in all examples
5. On both Android and iOS, there is no common pattern to the User-Agent strings of the webviews

There are some implications to consider here. The first is that the webview, in the Android case anyway, is essentially a different or older browser, and as such the set of features supported is not the same. Generally the webview supports a subset of the features supported by the default browser.

The second implication is that the webview is indeed detectable, and can be distinguished from the native browser. This might be important if your app or site makes use of any features supported by the browser, but not by the webview, which highlights the need to consider browser compatibility workarounds.

That the browser versions used are different has been noted on the Chrome developers site. But it’s not all bad—the gap in feature support between browsers and webviews is narrowing (see table below).

Webview and browser feature support differences are narrowing (Adapted from Chrome developers site)

The third thing to note is that the webview User-Agent is sort of a superstring of the native browser User-Agent. That is, the webview User-Agent more or less contains the device User-Agent (browser version notwithstanding). This is important because including the device User-Agent as the most significant portion of the string means that it can be treated by web sites and apps in the same way as the native browser.

While this is welcome, things are far from perfect. Not only is there no clear pattern or consensus to setting an app’s webview User-Agent string, but some apps use identical User-Agent strings, making detecting and targeting a specific app webview pretty difficult.

So, what have we learned? It’s all a bit of a mess really. It doesn’t have to be this way though. As we will see a bit later, not only does the HTTP standard suggest things could be done a little better, there are benefits to everyone involved if they are.

Webview User-Agent string best practices

So, if you’re an app developer, what should you do in terms of defining your own app User-Agent string? Let’s take a brief diversion to the HTTP/1.1 standard (RFC7231) to see what it recommends for User-Agent strings.

First, what are they for?

The “User-Agent” header field contains information about the user
agent originating the request, which is often used by servers to help
identify the scope of reported interoperability problems, to work
around or tailor responses to avoid particular user agent
limitations, and for analytics regarding browser or operating system
use.

The structure of a User-Agent string is then defined (where RWS stands for Required White Space)

User-Agent = product *( RWS ( product / comment ) )


And this is explained further:

The User-Agent field-value consists of one or more product
identifiers, each followed by zero or more comments (Section 3.2 of
[RFC7230]), which together identify the user agent software and its
significant subproducts. By convention, the product identifiers are
listed in decreasing order of their significance for identifying the
user agent software. Each product identifier consists of a name and
optional version.

Applying this to app webviews, any User-Agent string you come up with should:

1. be unique to your app and app version
2. contain the underlying default webview User-Agent string

However, you should also be cognisant of potential privacy issues raised in the HTTP/RFC7231 document:

Overly long and detailed User-Agent field values increase request latency and the risk of a user being identified against their wishes (“fingerprinting”).

In particular, any data unique to an individual device should not be included. For example, device ids, IMEIs, usernames, phone numbers, even preferred language should not be included. The User-Agent string is not the right place for information like this, and could lead to browser-fingerprinting, and unsolicited, invasive user-profiling. Cookies and associated opt-in and opt-out mechanisms already exist for such tracking.

An example of just such overly long and detailed information can be seen in the Facebook for iOS User-Agent string example mentioned above:

In particular, the inclusion of operator information vodafoneIE increases fingerprinting susceptibility.

So, taking all of this onboard, your new webview User-Agent string could take the following form:

That is, it should contain the default User-Agent string, with your app information appended. This will ensure maximum compatibility, and also that you will appear in web traffic analytics.

And this would end up looking something like the following, for version 2.0 of app called Awesome Kitteh:

Why change the webview User-Agent string?

So, why bother? What’s in it for the app publisher? What might happen if you decide to go completely your own way with your User-Agent string? Well, there are a few reasons really: user experience, web analytics, and app webview targeting.

One of the reasons for keeping the underlying device User-Agent string as part of your app’s string is so that web-apps and sites will continue to deliver content appropriate to your visitor’s device. If you make the User-Agent completely unrecognisable and unmappable to anything already known, then the sites your users visit won’t know what to do with it, and they may receive a fallback or default experience.

And then there’s analytics. We’re talking about web analytics here, used by website owners, as opposed to app distribution analytics solutions used by app authors, such as Flurry, Quancast etc. If, as an app author, you haven’t bothered to change the webview User-Agent string from its default, then there is little chance that you’ll appear on anyone’s website analytics radar as being a source of traffic, even if your app sends a lot of traffic their way. Having your app webview traffic show up in analytics is definitely desirable. At the least it makes your app visible to web publishers. But more than this, it enables the web publisher to modify his or her behaviour toward your app. For example, if a publisher sees more traffic coming via a Facebook webview over Twitter, then he might decide to invest more marketing effort into Facebook. If you don’t bother with a distinguishable webview User-Agent string, then you run the risk of any traffic coming from your app being unidentifiable in a sea of web analytics data. This is, after all, what the User-Agent string is meant for.

And then there’s webview targeting. Having an identifiable webview User-Agent string specific to your app opens up opportunities for web publishers and advertisers to deliver tailored content specifically targeting your app. This would not be possible if all app webview User-Agent strings looked the same.

OS manufacturers and webview User-Agent strings

Is there a role for OS manufacturers in any of this? Why, yes, there is! OS manufacturers could be key in fact. If they implemented the default webview User-Agent string so that the app name and version of the invoking app are automatically appended, then the job would be more or less done, and the world would be a better place. The ability to alter the User-Agent string could be left untouched, so that app authors who need to modify it can still do so; while apps whose authors are lazy or don’t know any better will, by default, have an identifiable and targetable User-Agent string.

Final thoughts

If webviews are to be used, and used they will be, then using this simple approach to define their User-Agent strings will benefit everyone involved:

• user experience: webview users get a better experience by virtue of webviews being identifiable and targetable
• web analytics: specific app webviews become identifiable, improving business intelligence for web publishers
• targetability: web publishers and advertisers can target webviews, and specific app webviews
• visibility: app authors benefit from greater visibility of their apps

And the best bit: all this is acheived simply by following the advice in the HTTP RFC document on User-Agent strings!

Thanks to Ronan Cremin for data and feedback