Brian Weatherson has written a post prompted by the New York Times symposium on experimental philosophy. The post makes a lot of valuable points (the Austin bit is particularly interesting, and not something I’ve thought of), but there’s also a line that touched a pet peeve of mine. So I wrote a comment. Since the line is something I’ve heard around these parts, I decided to reproduce the comment here. Let me know what you think!
It seems that “I like the idea of experimental philosophy, but it just relies too much on the survey method” has become a common refrain in criticisms of experimental philosophy. I’ve always found this line of attack a bit puzzling, or at the very least, imprecise.
As far as I understand, the survey method — in the salient sense — simply involves collecting data by asking for people’s self-reports, rather than, say, by observing and coding their behavior or scanning their brains. (Another sense, in which surveying involves no experimental manipulation, clearly does not apply to many experimental philosophy studies.) This method is commonly employed in social and (some) cognitive psychology. For example, studies in the confabulation literature use the survey method. They ask people to report on their judgments and how they come to those judgments. Of course, the experimental manipulations of those studies are what allows the interpretations of the data that led to the conclusion that people are unreliable in their self-reports about some mental processes.
I take it the thrust of the anti-survey refrain is in fact not about the survey method at all, but about the manipulations and interpretations of some experimental philosophy studies. Fair enough. But not all experimental philosophy studies use the same experimental manipulations and the same approach to interpreting the data. So, it seems to me, specific criticisms of manipulations or approaches to interpretation would be much more helpful than broad ones. At the very least, broad criticisms should identify what the problematic manipulations or approaches to interpretation have in common, and which studies fall prey to these problems. It looks like the experimental philosophy community has done a decent job keeping itself in check on those counts (e.g., this and that) even though progress certainly comes incrementally and slowly.
Moreover, once clarified, the thrust of the anti-survey refrain impacts philosophy’s interactions with empirical disciplines generally, and not just experimental philosophy. Perhaps the stake-manipulation study-designs do not adequately address epistemologists’ concerns and the straightforward inference from folk responses to philosophical conclusion is overly hasty. Even so, these are simply the kind of problems that frequently arise with interactions between philosophy and psychology (and likely other empirical disciplines too). Psychologists designing the experiments could easily, and perhaps even more likely given their lack of conceptual familiarity, miss philosophers’ concerns too. They might not know what the appropriate questions to ask either. Similarly, philosophers could misinterpret psychological findings. Additionally, in my experience, psychologists not infrequently misinterpret their own findings by making stronger conclusions than the data warrants; it would be bad, too, if philosophers were to draw philosophical implications by simply taking psychologists’ words at face value. This is not to say that philosophy ought not interact with empirical disciplines, but just that it’s hard generally. Consequently, I find the singling-out of experimental philosophy in this line of attack puzzling.
There are a lot of good methodological criticisms of specific experimental philosophy studies or sets of studies, including criticisms from experimental philosophers themselves, as noted earlier. As such, let’s lay to rest the anti-survey criticism of experimental philosophy in general. Minimally, it’s bad sloganeering.
- In the interest of full disclosure, I used to adopt the unfortunate “survey” terminology for some experimental philosophy projects as well. So let this post serve as self-correction.
- There are some experimental philosophy projects, as well as projects in psychology, that do not involve manipulation. Instead, they might do correlational analyses on people’s self-reports; Chandra Sripada’s recent work using structural equation modeling is one example. Even with these projects, though, what experimental philosophers are doing is not merely polling and taking the majority view, as the survey terminology might suggest.