How Bayesians Read a Think Piece

How likely is it that an opinion critical of [topic] will get expressed by someone on the internet?

My good friend (call her Anne) texted me this week. Anne sent me a link to a blog that declared some of her preferred works of art (i.e. musicals) to be inferior. She loves art, so to be told that her tastes were not exceptionally good was disappointing.

In my reply I wanted to make sure that Anne wasn’t putting too much weight on this new evidence:

How should we incorporate blogs into our beliefs about reality? (I see the irony – I’m writing a blog right now.)

The non-technical summary: you should be skeptical of what you read online.

The technical summary: the fact that some writer said “H” on the internet, should make you only slightly more confident that “H” is true.

I can’t improve on the Wikipedia presentation of Bayes’ theorem, so I’ll just paste in:

Let’s consider the probability that it is true that Anne’s favorite musical is bad. We’ll call that hypothesis “H”. What’s the probability of H, given that one person wrote an article stating that the musical is bad?

The evidence, E, is the article.

Instead of just evaluating whether the article is convincing or not, Bayesian inference requires that we consider

Were we confident that H was true BEFORE seeing the article? Was there good data up until this point that convinced us H is true?
If H is true, what’s the probability of this article being written?
What’s the overall probability of this article being written, regardless of whether H is true?

The probability that musical is bad given that someone wrote an article saying so is :

P(H|E) = P(bad|article)

P(bad|article) = ( P(article|bad) x P(bad) )/ P(article)

The right side of the equation asks whether we are likely to see the article if the musical is bad. If the musical is actually bad, then we are likely to see it condemned in print. HOWEVER, if we had a prior belief that the musical is not bad, then the numerator gets smaller.

Finally, we consider the denominator, P(E) or the probability of seeing an article that is derogatory towards the musical. If that probability is high, then the probability of the musical actually being bad goes down.

Here’s how Anne should think:

P(bad|article) = ( likely that article will be written if bad x prior evidence suggests not bad) / snobby think pieces get written regardless

P(bad|article) = (big x small)/ big = small probability that Anne’s favorite musical is actually bad

You should be just the right amount of skeptical when it comes to internet content. Be Bayesian.