Check out our archives:

January 26, 2017

Overfitting: is your website "too good"?

It's the beginning of the year and a time when you might look over your plans and goals going forward. This reflection is important because sometimes your measurements of success can tell a very different story than the success you’re trying to measure.

When we focus too much on imperfect measurements – and many of our measurements are imperfect – we can fall victim to "overfitting" and miss our goals completely.

To illustrate overfitting in the everyday world, let's imagine you're trying to be more healthy. There are many different ways to measure what "healthy" looks like, but it's difficult to get started unless we pick some kind of measurement. So, let's say you decide to aim for a particular weight. By using the metric of body weight as a proxy for your health, you can trick yourself into thinking you're becoming more healthy by skipping important meals or taking unhealthy supplements. By your chosen method of measurement, you may have succeeded. But by your original goals, you may have failed.

Metrics are useful, but it’s important to understand their context.

Similarly, without stepping back and re-evaluating how you measure success for your website and digital communications, you can get caught chasing the numbers at the cost of success.

So how can you avoid overfitting?

First, acknowledge that web and social analytics are imperfect measurements. You probably know this already: one metric that's well known for its imperfections is website page views. When you have more people loading pages on your website, that might mean more engagement or new traffic – but it can also mean your users are having difficulty finding information, or have come to the wrong website altogether. As a metric, page views are well known for being imperfect, but plenty of our other metrics are imperfect in some way or in some context too. High engagement on social media, for example, can indicate both positive and negative feelings.

Next, analyse change with more than just your key metrics in mind. If your metrics show some spike in activity try not to immediately assume that's good or bad based on your key metrics. Cross-check against other measures or conduct user research to try to understand that change.

Finally, recognise when your analytics are failing and redefine how you measure success. Key metrics should change as your business grows, or your technology changes, or users' expectations change.

Web and social analytics are an integral part of effective digital communication. But doing whatever you can to meet your target metrics won’t always lead to a good outcome. With occasional reflection, thoughtful investigation and the courage to change, you can avoid overfitting – and find success.

If you want to talk more analytics, get in touch with us! Maria Sjögren, Consultant at +46 76 109 05 55 or maria.sjogren@comprend.com.

 

This article was inspired by Algorithms to Live By: the Computer Science of Human Decisions by Brian Christian and Tom Griffiths.

Let's meet!


Don't hesitate to contact us to find out how we can help you.

Stockholm

Switchboard
+46 8 407 22 00

Staffan Lindgren
CEO
+46 70 971 12 12

  •  

  • Sveavägen 20, PO Box 3666
    103 59 Stockholm
    Sweden 

London

Switchboard
+44 (0)20 7336 8429

Phil Marchant
Managing director UK
+44 7740 933 415 

  •  

  • Second Floor, 59 Lafone Street,

    Courage Yard
    London SE1 2LX
    United Kingdom

Further reading

January 30, 2017

Snam, Wärtsilä and Eni the best online communicators in Webranking by Comprend

January 25, 2017

Swiss Post crowned best online communicator among Swiss non-listed companies

Cookies
This website uses cookies as described in our Cookie policy. To see what cookies we serve and set your own preferences, please use your web browser's settings. Otherwise, if you agree to our use of cookies, please continue to use our website. Cookie policy