Statistical models and mental models

In my current life as a data scientist, I often build statistical models. Sometimes, I try and explain to people what it actually means to build models. To be honest, I’m not always sure they care as much about this as I do, but it’s important question: when we build models, what are we actually doing?

I find the typical textbook definitions of statistical models a tad unsatisfying. “A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of some sample data and similar data from a larger population”, says Wikipedia. I mean, sure, but that’s perhaps a bit too dry and technical for most people to follow.

I think my favourite definition of a model is probably a fairly well known one: “A model is a simplification or approximation of reality”. I think part of reason I like this quote is that even though it’s a quote by statisticians, about statistical models, it hold up well for every other kind of model too.

When we make a statistical model, we use a bit of maths to encode a simplified set of assumptions about the world. “Let’s pretend the world behaves in this simple way”, we say. “If we do, what can we then learn or predict from our data?”. This simplification is unavoidable, since any system we’re interested in understanding with statistics – be it the human body, or human society, or usually sports in my case – clearly can’t be described precisely with just a few formulae. No model contains any objective truth, but we build them anyway, because even our simplified versions of reality can help us understand the world and make useful predictions about it. “All models are wrong, but some are useful”, the famous saying goes. Is our model one of the useful ones?

Mental models

If I asked you to categorise someone’s political views, how would you do it? Many people would probably try and position that person on a scale from Left to Right, or from Liberal to Conservative. People’s views don’t really exist on a straightforward linear scale from Left to Right, of course – why should they? People are far too complicated to capture in that way, and sometimes it’s important to remember that it’s just a model: a simplified way of thinking about the world that’s easier to work with than reality. After all, it would surely be overwhelming to try and understand the nuances of the political beliefs of everyone you ever meet, so some kind of model seems pretty sensible here. This is all I mean by a mental model here – a mental abstraction or a thought process that we use to think about something in the real world. What politics does your friend have? “Oh, they’re left wing”.

If you use a mental model like this one a lot, it’s probably tempting to start pretending that it really is reality, but it isn’t – and once you realise it’s a model, of course, you can start to question it the same way that you might question a statistical model. What are the assumptions behind it? Do they make sense?

The left-right model is a fairly obvious one, and lots of people have probably felt at some point that it doesn’t really capture their political views particularly well. Some think that open–closed is a more useful model than left-right at the moment, and there are plenty of other alternatives too.

There are more subtle models hiding in plain sight out there, however. A particularly influential model is GDP. It’s easy to conflate GDP with the performance of an economy, and by extension, the wellbeing of a nation. But GDP isn’t the economy. It’s a model of the economy, and the economy is much harder to model than many people probably realise. You may remember the mini-scandal when drugs and prostitution added £10bn to the UK economy, but in reality, nothing much changed about the economy: it was our model of the economy that changed.

Are all the struggles of millennials the fault of the baby boomers? Well, that’s a framing that involves bundling an entire generation of people into a single bucket. It’s a model, and it’s up to you to decide whether that model is useful.

How would you describe your personality? If you call yourself an ESTJ, or something similar, you’re probably a fan of the Myers-Briggs personality test. If you’re not familiar with Myers-Briggs, it’s a personality test: you answer a bunch of questions and are assigned 1 of 16 personality types. There aren’t exactly 16 personality types, because of course there aren’t. It’s a model. The reason personality tests are interesting is that I’ve met plenty of people who talk about them seemingly without any appreciation that they’re models. They really think they are an extrovert, who prefers sensing to intution, thinking to feeling, and judgement to perception, without realising just how flimsy that diagnosis might be. I’m picking on Myers-Briggs here just because it’s perhaps the most public – or at least, publicly criticised – of the personality models. There are many such models if you’re into that kind of thing, like the Big 5, or HEXACO, but because they’re models, we can interrogate them to work out how useful they are. How likely are you to get the same results if you take the same test more than once? How well can the models predict future behaviour?

There are so many things in the world that we have mental models about, and it’s unrealistic for us to be aware of all of them, let alone skeptical of all of them too. To survive in the world we just seem to rely too much on the kind of fast, associative decisions categorised as “System 1”¹ by Daniel Kahneman in his famous book “Thinking, Fast and Slow”. After all, if we were able to think critically and dispassionately about every situation we encountered, the list of apparent cognitive biases on Wikipedia wouldn’t be so comically large². But if we can bring the mindset we need when considering statistical models – which at heart, is perhaps just curiosity and scepticism more than anything – we can start to interrogate some of our mental models a little more, and maybe understand the world a little better too.