Beer, Models, and the Looming Dystopia

The title of this post is a bit of an exaggeration, but I am genuinely worried about where our society is going.  In short, I think our inability to understand or criticize the "science" that we increasingly rely on is a major problem.  I'll use a somewhat silly example to explain what I mean.

Consider this (not yet available) app that will produce a "map" of beers.  I won't try to give a detailed explanation, which I would likely mangle.  But the point is that it takes user-generated data from a beer review website and then uses it to determine the degree of similarity between beers.  The app then creates a map such that geographical proximity indicates similarity.

Now, the app might "work" in several different ways.  One way it might work is for the beers to cluster into "regions" that correspond to style categories.  This seems to have happened to a pretty large degree.  Another way for it to work is for it to generate successful predictions of what a beer drinker will like, based on the answers to a few questions.  This is apparently the intended purpose of the app, and I expect it will do fairly well.  As I said, the app isn't yet available, so I can't test its success in this area.

Now I should say that I think it is a cool app, and I will probably buy it if/when it becomes available (assuming I have a device that can run it).  But what strikes me about this endeavor is that it doesn't really have much in the way of theoretical underpinnings.  I don't mean the math—the math looks very impressive (though how would I know).  I mean that if it works, we won't know precisely why it works.  We have a general idea, of course, but in a way it would be serendipitous for the underlying data to yield useful predictions.  If it turns out that the neutrino travels faster than the speed of light, then physicists will have some fucking 'splaining to do.  But if the beer app can't generate a ranked list of 5 stouts I will like based on a ranked list of 5 IPAs I like, then it doesn't really have any broader implications.  It's just not as useful as we might have hoped.

And there's more.  The app might "work" for essentially bad reasons.  You can imagine that people might use a word to describe a particular "style" of beer (whether or not it is meaningfully distinct from other "styles" in any fundamental way), and that you could then get "false positives" in the software.  That is, it might indicate that two beers are similar to each other even though their similarity is purely a social phenomenon.  (You can also imagine that identical beers with different labels might end up in very different "locations" on the map.)  Of course, the program might still faithfully report the way people experience beer, so it would be a success in a way.  But the point is that it is very hard to tell what success means.  Do the style clusters on the map mean anything?  I suspect so, but I don't think the app can tell us that.

And moreover, the app might "work" one day and not the next, or "work" for one person and not another, and we wouldn't know why.  It is essentially a black box.  We know the algorithm, but we don't know why it works sometimes and not others.

It seems to me that a lot of knowledge is like this.  I think there is a temptation to call this kind of thing "scientific" or "empirical," and in a sense it is.  It is certainly data-driven.  But it seems like a very provisional kind of knowledge.  It is deeply qualified and we don't know what the qualifications are.  It might be the best we can do, and we might be willing to trust it quite a bit (for instance, to allocate marketing resources for a brewing company).  But it rests on an unproven foundation and it is very hard to identify the appropriate scope for applying the knowledge.

I recognize that this is sort of a general Humean problem, but I feel it is a bit more of a real-world concern in cases like this than in cases like whether the sun will rise tomorrow.  As I noted, there is a temptation to call these fancy models "scientific" or "empirical" and to discount criticism of them as backwards and ignorant.  But it is dangerous to put more and more of our ideas into black boxes, with all dissent labelled as anti-scientific.  We risk being far too confident in our "knowledge," and allowing that confidence to bestow unearned political victories on particular ideologies.

To give a real-world example (because who really cares if a beer-mapping app doesn't have theoretical underpinnings?), Esther Duflo has been canonized as a saint of the modern Gladwell-inflected technocratic movement.  These are TED people, the kind of people who believe that democracy is dirty and corrupt, and that much better outcomes would be achievable if only we could put "the smart people" in charge.

Now I am sure Esther Duflo's work is very good, but it suffers from some of the same fundamental limitations as the beer app, and I don't think this is appreciated widely enough.  (Here is a paper by Angus Deaton (PDF) making the case.)  Duflo has a lot of influence that stems from what you might call the rhetorically advantageous position of her work.  She may use the influence for good, but that would be somewhat serendipitous.

But having brought her up, let us somewhat abruptly put Duflo aside.  Even in the worst case her work is useful, although it may be given too enthusiastic a reception  by the technocratic "elite."  It is the mindless enthusiasm of that reception that scares me.  I am afraid that we are entering a time when all sorts of policies will be "black boxed," derived from models that are not subject to meaningful inspection.  This is roughly what I think cost-benefit analysis is, at least in theory (in practice I don't think it amounts to much beyond a threat of litigation).  Let's say the black box tells us that we should tear down a poor neighborhood and build a highway or something.  Traditionally this would have been controversial, a political and constitutional issue.  But increasingly I think we will be tempted to "put it beyond politics" by running the numbers to determine whether it is the "optimal policy."  Dissenters will be called anti-scientific (just as people who believe that blacks and whites are equal are branded as anti-scientific), and they will be ridiculed.  After all, they just don't get quantitative science (or genetic science as the case may be).

This is a worry for me partly because I think we will be justified in relying heavily on quantitative models that don't have theoretical underpinnings that can be articulated or assessed.  We don't have a choice, really.  (And I am open to the possibility that all knowledge is like this, though again, I don't think you have to go full Hume to see my point.)  But I suspect we will be very bad at drawing lines and maintaining open minds as we go down this path.  Already I think far too many people think of democracy as a hindrance, a sort of necessary evil that hopefully won't get in the way of the "smart people."  This is of course a deeply flawed idea, but I don't think that will slow it down much.