muddy boxes and their contents

but I *want* to open the black box!

This week finds me at another conference, once more talking to really clever and interesting people I would never otherwise have met.  It hasn’t got old yet, I’m not going to lie.  It’s a conference of marketing people; and of course, in the last couple of decades marketing has become one of the most data-driven industries that there is.  Consequently, I ended up having a lot of conversations about the doctrine of “nothing is to be gained by opening the black box”. 

This is a really important principle of management cybernetics, which I will try to do justice to in a future post; I can’t give up on it like I kindasorta did with POSIWID.  But on the other hand, I feel a lot of sympathy with the view expressed to me at the conference that there is something hinky about “black box” type indicators – if someone walks into your office with a system that takes inputs and gives outputs, but tells you that you’re not allowed to ask how, then that’s bad.  And I share this intuition – I’ve seen something very similar in the stock market dozens of times, of course. “Proprietary” indicators have a really bad track record when it comes to being snake oil.

As a sort of trail of next week’s discussion, I will link here to Kiearan Healey’s epic paper in sociology, “Fuck Nuance”; a key theme of which is that if someone tells you “it’s more complicated than that”, then they had better be talking about an ultraviolet photolithography machine or a particle accelerator or something.  In the meantime, I am going to retreat to the comfort zone by considering the subject of black boxery in the context of my favourite go-to example, “Value At Risk”.

I might do Friday’s post on “VaR: My Part In Its Downfall”, because I had quite a ringside seat for the rise and fall of this often (and usually correctly) maligned risk metric.  For the time being, though, let us admire it as a work of variety engineering, and as an almost perfectly executed black box.  Value At Risk was invented at JP Morgan, on the orders of Sir Denis Weatherstone, who was CEO at the time. 

Being CEO of JP Morgan requires you to construct a homomorphism – a mental representation of JP Morgan that you can hold in your own brain and make decisions about.  As part of that project, Weatherstone commissioned the construction of a black box with thousands of inputs (basically, all the traded securities positions on the books of the bank, recalculated daily) and one output – a number, which was called Value at Risk.

A pure black box approach would have been literally that – the point of the black box analysis is that you think about things in terms of their inputs and outputs while ignoring their internal states.  But in fact, JPM produced voluminous technical manuals explaining how it was calculated, one of which I pinched from the Bank of England Supervisory Policy Division (sorry guys, but the statute of limitations has passed and given all that’s happened in banking risk since 1996 I doubt you want it back).

This means that, in Stafford Beer’s terminology, it’s a “muddy box” – you can treat it as a black box if you want to, or you can read the manual and clarify it to yourself.  It’s not intrinsically a black box, as a system made up of human beings might be.

But what do you gain from doing so?

A senior bank supervisor I knew did read the books, and summarised the information thus to me – “it’s just a fancied up bloody standard deviation”.  Which is true.  Value at Risk was the weighted sum of the volatility of daily returns, weighted by the size of the positions, and by a matrix which took into account the correlations between them.  This was then subjected to a simple arithmetic transformation, giving a number which would correspond to:

“an amount of money, …

such that on 95% of trading days …

this trading book would not have a result worse than a loss of that amount…

in an utterly counterfactual world in which all securities trading profits and losses were normally distributed[1]”

Tragicomically, the bit in italics tended to fall off the end, with consequences I’ll deal with on another occasion.

So what do we gain by knowing this?

Well, we gain the knowledge that there’s a stable and reliable relationship between the inputs and the output.  Which is not nothing.  There were plenty of vendors hanging around in the 00s offering “VaR Plus” models, which combined the fancy standard deviation with a “proprietary” forward looking predictive blah blah.  This was, in my view, a lot worse.

But what do you actually gain? Although the technical manual helps you understand the calculation, you’re not really going to reproduce it yourself with pen and paper, are you?  For all practical purposes, it’s a black box; if someone installs a system to produce Value at Risk and makes a serious programming error, you absolutely cannot be sure that you’ll be aware of this just by looking at the output.

Except – the threat to clean up the muddy box is important, because it keeps people honest.  You could find that massive programming error, which means that the person installing it for you is going to take care and carry out quality assurance. I think this is what people actually mean when they say they hate black boxes.  They don’t like unauditable things.  And they’re right to, even if they actually never use the great big technical manual and are kind of hypocritical about black boxes.

I feel like this post has created a load of loose ends that will take weeks to tidy up, but so be it; it’s too long already.

[1] yes, it’s more complicated than that.  See paragraph 3 of this post.