How a simple equation rules the world

From the very first days of the world wide web, engineers used webcrawlers (automated bots that move randomly between websites) to collect information about the newly created sites.

But the question was how to put all the information they collected together to find the most interesting content?

To answer this question, the founders of Google, Larry Page and Sergey Brin, had an idea. They realised that the real information was not to be found in the words and pictures on the internet, but hidden in the process of hopping itself: because the webcrawlers moved around at random, and there were more links to more popular pages, the most popular pages would be visited more often.

To turn this insight into an algorithm that became Google Search and made them both billionaires, Page and Brin made use of a 90-year-old equation, known as the stationary distribution of a Markov chain. This equation allowed them to automatically generate the search results we use many times per day.

I use the name The Influencer Equation when I refer to the Markov chain equation; both because it is easier to remember and, mostly, because it captures the way it has been used, not only by Google, but by other social media giants, like Amazon, Netflix and Instagram.

It has made searching more convenient – it is easier to find the most popular books, films and Instagram selfies – but it has also had a side effect. Since the equation doesn’t look at the content of web pages, it means that the only factor it accounts for when recommending things to us is popularity. In this way, The Influencer Equation warps our view of the world. 

The fact that the whole system of searching is based on just one equation also gives us an opportunity: we can reverse-engineer it and see more clearly again. The Influencer Equation skews how we see each other.

You might feel that other people are more popular than you on Instagram and wonder what makes them special? The answer is, more often than not, nothing at all! They are popular because they are popular. A product of the equation that reinforces popularity and ignores content.

The Influencer Equation isn’t the only piece of little-known mathematics that has transformed the Internet. Around about the time of South Korean megahit Gangnam Style, YouTube had a problem. It was 2012 and although hundreds of millions of us clicked on videos and visited the site, we weren’t staying there.

Novelty videos like Charlie Bit My Finger, Double Rainbow, What Does the Fox Say? and Ice Bucket Challenge only held our attention for 30 seconds before we went back to doing something else.

In order to attract advertising revenue, YouTube needed to become a place where users would stick around.

To solve the problem YouTube called three engineers – Paul Covington, Jay Adams and Emre Sargin. They set about feeding the watching patterns of YouTube users into a neural network, a piece of computer software that can be best visualised by thinking about a funnel. One side of the funnel takes in data about videos and users, the other end makes predictions about how long we will watch a film clip for.

At first, the errors in predictions are very large, but by using an equation based on differentiation – a technique for minimising errors which originates from the work of Sir Isaac Newton – the predictions are gradually improved. The funnel ‘learns’ our watching preferences.

The success of the funnel was astounding. In 2015, the time 18 to 49-year-olds spent watching YouTube increased by 74 per cent. By 2019, it had 20 times as many views as it did before the Google researchers started their project, with 70 per cent of those views coming from recommended videos.

The lesson you should learn from this is clear: if you believe that you are exploring your own interests on YouTube, but find yourself clicking on the suggested videos, then you are sadly misguided. The funnel has effectively turned YouTube back into a traditional form of TV, with scheduling decided by AI.

Understanding how we are manipulated when we go online starts with understanding the equations social media companies use to control us. In addition to the Influencer and Learning equations, the other key equations used online today are The Reward Equation – that is used to keep us hooked to notifications on our phone – and The Advertising Equation – that tailors adverts on Facebook to our interests.

You don’t need to know all the mathematical details to understand how these equations work, but you do need to know that they exist and that they affect your view of the world. 

David Sumpter is professor of applied mathematics at the University of Uppsala, Sweden and the author of The Ten Equations that Rule the World: And How You Can Use Them Too, out now (Allen Lane, £20)