Tag Archives: clicks

Click Here!

Click here — the myriad of buttons on the web that are trying to pull user clicks

When a user clicks on a web page, it’s a good sign. They’re engaged with your content, they’re using your site. Clicking is certainly better than abandonment: where a user doesn’t interact with the page.

I recommend that you track percent abandonment of each of your pages: it’s a simple measure of how you’re doing in engaging customers. The lower the abandonment, the better.

There are a few exceptions, where abandonment doesn’t necessarily imply a problem. For example, a page that tells the time, or otherwise answers a question, doesn’t require a click. Indeed, a click might be bad news – the answer wasn’t a good one.

The more clicks, the more interesting the link to users?

Not necessarily. Suppose you’ve got a page with many links in many places. All things being equal, expect to see many more clicks on elements that are higher on the page. For elements at the same vertical height, expect to see more clicks on elements that are on the left of the page (the opposite is true for languages that are right-to-left such as Hebrew). All up, if a link is in the top left of the page, it’ll get clicked much more. In general, expect an extreme value distribution of the clicks.

Since users scan left to right, top to bottom, you should organize a page so that the most important things for users are closer to the top left, and the less important things are closer to the bottom right. This is pretty intuitive – you expect a web site to have the most common, useful links in its header, and the less useful more obscure stuff in the footer.

Hmmm. So, which links on my page do users like?

If you want to understand the relative performance of links on a page, you could consider swapping them and comparing the number of clicks. For example, suppose you’ve got a header on a page with three elements “About”, “Jobs”, and “Help”. You could measure the number of clicks on these for a week. You could then swap the “Help” and “About” links, and measure for another week. Does “Help” get more clicks in week two than “About” did in week one? If yes, the second ordering is better; if no, stick with the original ordering.

You need to be careful what you compare. It’s pretty safe to compare “Help” and “About” text in a header between two experiments. But you’ll find that there’s text you can create that will get more clicks, regardless of whether it’s more useful. It’s a well-known industry fact that “top ten” lists on tabloid sites get way more clicks than other stories. Text such as “click here” gets more clicks. Images attract the eye to nearby text, so that it gets more clicks. If you move something that customers are used to, expect them to click it less. And so on.

Experiments at scale

In a large-scale web business (such as eBay), we generally don’t do experiments sequentially in time. Instead, we’ll show the first alternative to some fraction of users, and the second alternative to another fraction of users. We can then compare the two populations over the same time, which both speeds up experimentation cycles, and also reduces any effects of seasonality or other differences between experiments that aren’t carried out at the same time. (There’s some issues this creates – a topic for another time.)

A Click is a Vote

If a user clicks on a link, this tells you something about that link. If it gets more clicks than you’re expecting, it’s usually good news (more on this topic later). If it gets less clicks than you’re expecting, it’s typically bad news.

What’s not widely known is that a click on a link tells you something about the links that come before it. Specifically, if a user skips a link on a page and clicks on the next link, this tells you that then former link isn’t relevant to the user. It doesn’t tell you links below it are irrelevant to the user — users leave the page when they see the first thing that’s relevant. This is a well-known phenomena in search engines.

Good clicks and bad clicks

A click is a good, basic signal. As I said at the start, it’s generally better to get a click than to not get one.

But there are ways you can make a click an even more reliable signal of user happiness. One simple trick is to factor in how long the user dwelled on whatever it was that was clicked. If they click on a link, press the back button immediately, and return to the original page, it’s actually a sign of unhappiness. They didn’t find what they wanted. If you wanted a rule of thumb, I’d say any click that dwells less than 10 seconds is a bad sign. I’d say any click that dwells more than 30 seconds is a good sign. You could try counting clicks, bad clicks, and good clicks, and drawing your conclusions from there.

Please share this post using the buttons below (hopefully they’ll pull your clicks!). See you next week!

Clicks in search

Have you heard of the Pareto principle? The idea that 80% of sales come from 20% of customers, or that the 20% of the richest people control 80% of the world’s wealth.

How about George K. Zipf? The author of the “Human behavior and the principle of least effort” and “The Psycho-Biology of Language” is best-known for “Zipf’s Law“, the observation that the frequency of a word is inversely proportional to the rank of its frequency. Over simplifying a little, the word “the” is about twice as frequent as the word “of”, and then comes “and”, and so on. This also applies to the populations of cities, corporation sizes, and many more natural occurrences.

I’ve spent time understanding and publishing work how Zipf’s work applies in search engines. And the punchline in search is that the Pareto principle and Zipf’s Law are hard at work: the first item in a list gets about twice as many clicks as the second, and so on. There are inverse power law distributions everywhere.

The eBay Search Results Click Curve

Here’s the eBay search results click curve, averaged over a very large number of queries. The y-axis is the total number of clicks on each result position, and the x-axis is the result position. For example, you can see that the first result in search (the top result, the first item you see when you run a query) gets about eight times as many clicks on average as the fifteenth result. The x-axis is labelled from 1 to 200, which is typically four pages of eBay results since we show 50 results per page by default.

eBay Click Curve. The y-axis is number of clicks per result, and the x-axis is the result position.

As a search guy, I’m not surprised by this curve (more on that topic later). It’s a typical inverse power law distribution (a “Zipf’s Law” distribution). But there are a couple of interesting quirks.

Take a look at the little bump around result position 50 on the x-axis. Why’s that there? What’s happening is that after scrolling for a while through the results, many users scroll to the very bottom of the page. They then inspect the final few results on the page (results 46 to 50), just above the pagination control. Those final few results therefore get a few more clicks than the ones above that the user skipped. Again, this isn’t a surprise to me — you’ll often see little spikes after user scroll points (in web search, you’ll typically see a spike in result 6 or 7 on a 10-result page).

I’ve blown up the first ten positions a little more so that you can see the inverse power law distribution.

Search click curve for the first 10 results in eBay search.

You can see that result 1 gets about 4 times as many clicks as result 10. You can also see that result 2 gets about 5/9ths of the clicks as result 1. This is pretty typical — it’s what you’d expect to see when search is working properly.

Interestingly, even if you randomize the first few results, you’ll still see a click curve that has an inverse power law distribution. Result 1 will almost always get more clicks than result 2, regardless of whether it’s less relevant.

Click Curves Are Everywhere

Here are some other examples of inverse power law distributions that you’ll typically see in search:

  • The query curve. The most popular query is much more popular than the second most popular query, and so on. The top 20% of queries account for at least 80% of the searches. That’s why caching works in search: most search engines serve more than 70% of their results from a cache
  • The document access curve. Because the queries are skew in distribution, and so are the the clicks per result position, it’s probably not surprising that a few documents (or items or objects) are accessed much more frequently than others. As a rule of thumb, you’ll typically find that 80% of the document accesses go to 20% of the documents. Pareto at work.
  • Clicks on related searches. Most search engines show related searches, and there’s a click curve on those that’s an inverse power law distribution
  • Clicks on just about any list: left navigation, pagination controls, ads, and any other list will typically have an inverse power law distribution. That’s why there’s often such a huge price differential between what advertisers will pay in search for the top position versus the second position
  • Words in queries, documents, and ads. Just like Zipf illustrated all those years ago, word frequencies follow an inverse power law distribution. Interestingly, and I explain this in this paper, Zipf’s formal distribution doesn’t hold very well on words drawn from web documents (a thing called Heap’s law does a better job). But the point remains: a few words account for much of the occurrences

What does this all mean? To a search guy, it means that when you see a curve that isn’t an inverse power law distribution, you should worry. There’s probably something wrong — an issue with search relevance, a user experience quirk (like the little bump I explained above), or something else. Expect to see curves that decay rapidly, and worry if you don’t.

See you again next Monday for a new post. If you’re enjoying the posts, please share with your friends by clicking on the little buttons below. Thanks!