Opera - vendor prefixes report for CSS animation, transform, transition, background-size

To assess the damage that CSS vendor prefixes could do to the Opera Browser, we have created a script that looks at the type of CSS used for certain properties. This survey has been run by Karl Dubost (twitter: @karlpro) for Opera Developer Relations in August 2012.

This survey (and associated scripts) is meant to give an idea of what is done on the web with regards to using prefixed properties. Statistics "lie". That's their goal. They often give a view that erases the individual cases. On the other hand, users browsing the web are all about individual cases. A specific site will break for a specific user of a specific browser. The fact it is breaking for a small market share, for a single country, etc., doesn't remove the fact that it is breaking. In the results below, you will notice we didn't give any percentages. A finer grained analysis with a manual checking is necessary to give relevant percentages and associated error margins. The numbers are here to show a trend and difficulties that might result from it.

Before going into the results, we will provide information on the data set, what we tested and the many caveats of this survey.

How to report bugs and suggest features on this survey

Everything about this survey is open source, released under an MIT License — you are welcome to take the source code and run your own modified version. It would be appreciated though if you could submit patches, suggest features, etc. This is a first draft or alpha if you prefer. A lot of things can be done to improve and make it more useful for all of us.

Alexa data sample

We used Alexa's top 1000 sites list (2012-07-30), extracted from the top 1 million sites. Alexa provides a list of domain names only, in the following format:

970,opera.com

We applied a transformation, so we could get a URI to process:

http://opera.com/

Obviously the main web site address will sometimes be different. See the section on caveats

User agent strings

We used two user agent strings (one for mobile and one for desktop) and ran the survey twice to assess if the results are different.

For Opera desktop:

Opera/9.80 (Macintosh; Intel Mac OS X 10.8.0) Presto/2.12.363 Version/12.50

For Opera Mobile:

Opera/9.80 (Android 2.3.5; Linux; Opera Mobi/ADR-1202082305; U; en) Presto/2.10.254 Version/12.00

Main data logged in the survey file

We logged a few things in the log file. The main data about vendor prefixes is expressed with lines such as:

2012-08-14 11:31:50,127 - INFO - SURVEY: http://www.opera.com/ ['transition', 'moz', 'o', 'webkit', 'prefixless']

The data items — from left to right — are:

• The date and time
• The term INFO
• The keyword SURVEY
• The URL being analyzed
• A list with the first item being the name of the property found, followed by the list of vendor prefixes used for this property. prefixless means that the property is present without prefixes .

There are other things logged into these files — we'll let you explore them for now, although it might be necessary to document them later on. The script is raw on the edge and the logs are too. Any proposal for improving the log and the storage of results to make it more usable are welcome. We compiled the results using a combination of sed, grep, uniq, sort and awk — they are fast, flexible and do the job. But we are open to better ways of doing it.

Stats for the survey

To extract all the lines related to the survey. sort is used to put similar lines in order . uniq -c will compile and count the same lines:

grep "SURVEY"  log-2012-08-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c

We can then extract related items, for example transition:

grep "SURVEY"  log-2012-08-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transition'

or items related to a specific web site such as youtube:

grep "SURVEY"  log-2012-08-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'youtube'

This will give us all sites which are not adding -o- in their rules:

grep "SURVEY"  log-2012-08-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep -v "'o'"

And the sites without -o- AND without the prefixless property:

grep "SURVEY"  log-2012-08-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep -v "'o'" | grep -v "'prefixless'"

Python script caveats and limitations

The script has many flaws. First, it has been written by me (Karl Dubost). It uses three external libraries — Requests, lxml, cssutils — and the script might be dependent on the bugs of these libraries, in addition to its own bugs. Some of its caveats are described in the following sections.

Performance

The script is not really very fast, so there are certainly issues to address if we wanted to process a larger volume of data. The sample of 1000 sites took around 7 hours and 22 minutes (442 minutes) to process; 1 site every 26.52s (as a mean). Some sites takes longer than others. The process order is as follows:

• Initial HTTP request
• HTTP redirections
• Connections failed
• HTML parsing
• CSS parsing
• The code itself

We could probably improve the performance of the script by first downloading all the resources and analyzing them locally. It's a path to explore for future versions. That would greatly help for running stats on the same sample of data.

Sample and Countries

Alexa has domains from different countries, but it is not necessarily representative of the local sites used for each country. This is important when addressing specific local market shares. It is usually hard to have a good overview for each country, so in this report, we are not trying to address this aspect of the data. It would be an interesting additional study to do for this survey and explore the differences between the markets.

HTTP Redirections

Because we are starting with a URL, which is based on a domain name, some sites will redirect the HTTP requests according to the user agent. For example, for the Opera domain we first access http://opera.com/ then http://www.opera.com/. When accessing with the mobile user agent, we get additional redirections for certain sites such as sfgate:

Starting new HTTP connection (1): sfgate.com
"GET / HTTP/1.1" 302 235
Starting new HTTP connection (1): www.sfgate.com
"GET / HTTP/1.1" 302 39
Starting new HTTP connection (1): m.sfgate.com
"GET / HTTP/1.1" 302 0
"GET /sfchron/index.htm HTTP/1.1" 200 None

The script is interacting at the HTTP layer and doesn't have a JavaScript interpreter. So all sites dependent on JavaScript for redirecting the browser have not been working accordingly to the intent of the owner. It can be difficult to assess the exact mobile version:

• A separate mobile site without redirection
• A redirection made only through JavaScript
• No separate site (media queries used for different screen sizes)
• A redirection to a specific device application

On the Mobile survey, the results then are a mix of mobile (the redirection happened) and desktop Web sites (there was no redirection for one the previous reasons). We tried to figure out roughly the number of redirections. There are some patterns with domain names:

grep "Starting new HTTP connection" log-2012-08-opera-mob.txt | sed "s/.*connection (1): //" | uniq | grep "^m\." | wc -l

152 domains starting with m. :

grep "Starting new HTTP connection" log-2012-08-opera-mob.txt | sed "s/.*connection (1): //" | uniq | grep "^mobile" | wc -l

24 domains starting with mobile (a handful had a completely different domain name):

It is another issue to solve and re-run the survey for only mobile Web sites.

Type of CSS

We parsed only CSS set through a style element or a link rel element. We did not take into consideration styles set through the style attribute, @imports or through JavaScript. Some sites modify their CSS depending on the user agent and a few other rules.

Shorthand properties

In this first run, we tested only shorthand properties, so individual property rules (either hardcoded or created by CSS generators) are not counted in this current survey. The main code has to be modified to take into account these cases. We opened issue 13 to account for this.

It would also be interesting to assess what the ratio of sites using the shorthand properties compared to the full individual rules is.

Manual checking

The results given in this report do not equate to the featured web sites being completely broken for their users. It would require a lot of work by a big group of volunteers for testing each individual site on the list to assess site compatibility for each browser.

Survey results

The data for the survey is available on github. Feel free to reuse them and point out mistakes. As we said before this is much a work in progress and we have certainly made mistakes. Read the introduction again … it is imperfect.

Frequency of CSS types

Usually high traffic Web sites try to keep the number of HTTP requests to a minimum, because the time an HTTP request takes to be completed is significant to the overall performance of the Web site. Because we were requesting the Web sites' CSS URLs, we took the opportunity to count them.

1. Rank Alexa 73: http://www.adobe.com/ has 10 link rel elements.
2. Rank Alexa 74: http://www.rakuten.co.jp/ has 16 link rel elements.
3. Rank Alexa 744: http://www.alarabiya.net/ has 22 link rel elements.

On the Desktop survey, 492 of the 1000 Web sites were using one or more style elements. On Mobile, this number was 455 Web sites.

There are certainly a few graphs needed to make this easier to understand. Be my guest.

The CSS shorthand transition property

Let's take a look at transitions, and how many of the 1000 sites used them at least once. In each case we are looking for something along these lines:

.foo {
  -webkit-transition: background-color linear 1s;
  -moz-transition: background-color linear 1s;
  -ms-transition: background-color linear 1s;
  -o-transition: background-color linear 1s;
  transition: background-color linear 1s;
}

Let's grep:

grep "SURVEY"  log-2012-08-opera-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transition' | awk '{print $2}' | sort | uniq | wc -l

Below you can see the numbers of sites found that use this property, with the number of sites which have all vendor prefixed versions of this property in at least one CSS rule on the site (['transition', 'moz', 'ms', 'o', 'webkit', 'prefixless'])

• Desktop: 188 (65)
• Mobile: 170 (46)

What about the frequency for each vendor prefix?

grep "SURVEY"  log-2012-08-opera-mob.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transition'| grep "'o'" | awk '{print $2}' | sort | uniq -c |wc -l

We can see that web developers are most commonly using -webkit-, then -moz-, then -o-, then -ms-, meaning IE will have the hardest time of all of the browsers.

How many of thee sites used transition without the prefixless property? This is important, because these are Web sites which are likely to break once browsers stop using the prefixed versions. It shows also how much a browser vendor might be encouraged to drop their prefixed properties, or not. Remember when a site is breaking, it is breaking for individual people. Below you can see how many of those sites didn't used the unprefixed fallback for trandition.

.foo {
  -webkit-transition: background-color linear 1s;
  -moz-transition: background-color linear 1s;
  -ms-transition: background-color linear 1s;
  -o-transition: background-color linear 1s;
}

• Desktop: 89
• Mobile: 91

Let's look again at the frequency for each vendor prefix, but this time without the prefixless version alongside it.

Why this is interesting? Because it shows the ability for a browser vendor to drop the prefixed version from its implementation. If the prefixed version is dropped and there is no prefixless version, the site will break for this specific browser. And the irony of this situation is that it is indeed a lot harder for webkit to drop prefixes than other browsers because a lot more sites will break. Basically, web developers used only -webkit- prefixes in many CSS rules without the prefixless fallback, creating issues for other browsers now and creating issues for webkit in the future.

grep "SURVEY"  log-2012-08-opera-mob.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transition'| grep "'o'" | grep -v "'prefixless'" | awk '{print $2}' | sort | uniq -c |wc -l

How many of these sites without the prefixless version use only one vendor prefix, such as

.foo {-webkit-transition: background-color linear 1s;}

For example to extract web sites using only -webkit-, we could use this:

grep "SURVEY"  log-2012-08-opera-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transition'| grep -v "prefixless" | grep -v "'o'" | grep -v "'ms'" | grep -v "'moz'"

We can start to feel the disturbance in the force. Basically some sites have CSS rules targeting only WebKit. If you are wondering about the 1 site in Desktop that only targets Opera, it is Facebook — which has 10 CSS rules with an -o-transition only. Remember the survey has been run with an Opera user agent; I suspect Facebook sends different stylesheets to different user agents. To verify:

10 http://www.facebook.com/ ['transition', 'o']

Often the sites use a -webkit- prefix only for one or two CSS rules, but then use other vendor prefixes elsewhere in the code. This could be the result of more than one person working on the CSS with different coding styles, or it might be just negligence.

What is a lot more disturbing is the following list of mobile web sites, which (if we have not made a mistake) only have -webkit-transition, with no rules at all for other vendors. Why is it disturbing? Because these sites have taken care of identifying the Opera Mobile user agent, redirected in most cases to the mobile site, but still served CSS that will create difficulties or incomplete user experiences for this specific user agent.

# mobile sites list
http://3g.kaixin001.com/
http://java.com/en/
http://m.avito.ru/
http://m.bing.com/?mid=10006
http://m.livejasmin.com/en/home
http://m.myspace.com/home.wap?bfd=webnext
http://m.nordstrom.com/
http://m.photobucket.com/?src=www
http://m.retailmenot.com/
http://m.sears.com/
http://m.subito.it/
http://m.taobao.com/?sprefer=sypc00
http://m.tmall.com/?sprefer=sypc01
http://mi.nba.com/_touch/
http://mobile.news.com.au/
http://mobile.wetter.com/
http://tw.m.wretch.yahoo.com/
http://wap.sogou.com/?fr=s-sogou&clk=s-sogou
http://www.buzzfeed.com/?s=mobile
http://www.etsy.com/
http://www.tripadvisor.com/
http://www.verizonwireless.com/b2c/index.html
http://www.wikia.com/Wikia
https://online.citibank.com/US/Welcome.c
https://www.outbrain.com/

There are only 5 domains using transition without any prefixes.

The CSS shorthand transform property

We have applied the same type of queries that we used for transition to the transform property as well. To find any combination of prefixes, we can use this:

grep "SURVEY"  log-2012-08-opera-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transform' | awk '{print $2}' | sort | wc -l

Below you can see the number of sites that use at least one transform, with the number of sites that have all vendor prefixes (['transform', 'moz', 'ms', 'o', 'webkit', 'prefixless']) in at least one CSS rule in parentheses:

• Desktop: 137 (65)
• Mobile: 111 (18)

What is the frequency for each vendor prefix?

For Opera, for example:

grep "SURVEY"  log-2012-08-opera-mob.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'transform'| grep "'o'" | awk '{print $2}' | sort | uniq -c |wc -l

How many of these sites without the prefixless version use only one vendor prefix?

The CSS shorthand animation property

We are applying the same type of queries again, to the animation shorthand property. To find any combination of prefixes, we can use this:

grep "SURVEY"  log-2012-08-opera-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'animation' | awk '{print $2}' | sort | uniq |  wc -l

Below you can see the number of sites that use the animation shorthand property at least once, with the number of sites that have all vendor prefixes (['animation', 'moz', 'ms', 'o', 'webkit', 'prefixless']) for at least one CSS rule on the site in parenthesis:

• Desktop: 24 (5)
• Mobile: 22 (3)

What is the frequency of each vendor prefixes?

For Opera for example, we can use this:

grep "SURVEY"  log-2012-08-opera-mob.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'animation'| grep "'o'" | awk '{print $2}' | sort | uniq -c |wc -l

How many of these sites without the prefixless version are using only one vendor prefix?

Notice that the situation for the shorthand animation seems less problematic. As mentioned in the caveats we would have to double check for the individual properties. Still we can take this as a good sign.

The CSS background-size property

We are applying the same type of queries again for the background-size property. To search for any combination of prefixes on this property, we can use this:

grep "SURVEY"  log-2012-08-opera-dsk.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'background-size' | awk '{print $2}' | sort | uniq |  wc -l

Below you can see the number of sites found that have at least one instance of the background-size property present, with the number of sites that have all vendor prefixed versions (['background-size', 'moz', 'ms', 'o', 'webkit', 'prefixless']) present in parentheses:

• Desktop: 74 (5)
• Mobile: 132 (3)

What is the frequency for each vendor prefix?

For Opera for example, we can find out like this:

grep "SURVEY"  log-2012-08-opera-mob.txt | sed 's/^.*SURVEY: //' | sort | uniq -c | grep 'background-size'| grep "'o'" | awk '{print $2}' | sort | uniq -c |wc -l

How many of these sites without the prefixless version use only one vendor prefix?