Experimental Metasurvey

[cross-posted at: http://experimentalphilosophy.typepad.com/experimental_philosophy/2011/05/experimental-philosophy-metasurvey-results.html. This is a summary of the results of a meta-survey about experimental philosophy conducted by myself (Billy Dunaway), Anna Edmonds and David Manley.]

Some current experimental philosophy is devoted to conducting surveys among non-philosophers to gather information about their dispositions to apply philosophically relevant concepts. And those who report the results of these surveys sometimes make claims about how surprising these results are to philosophers. (Here is a representative quote: “[W]e think that a critical method for figuring out how human beings think is to go out and actually run systematic empirical studies… Again and again, these investigations have challenged familiar assumptions, showing that people do not actually think about these issues in anything like the way philosophers had assumed.” (Nichols and Knobe, “An Experimental Philosophy Manifesto” in Experimental Philosophy, ed. Knobe and Nichols, p. 3)) But whether an empirical result is surprising to a group of people is itself an empirical question, and so we designed a survey of our own to test this.

Our hypothesis was that that philosophers would, for the most part, correctly guess what kind of response non-philosophers would give. This was confirmed by our study. We selected several published surveys of folk subjects, each of which had been claimed in the literature to have surprising results. The surveys we chose cover a variety of philosophical topics: causation, intentionality, and moral responsibility. We asked philosophers to suppose that ordinary, non-philosophical folk are presented with the relevant cases, and to say how they thought the folk would respond. (Subjects were firmly instructed to opt out of a given question if they had prior familiarity with experimental research that might bias their answer.) For each question, at least 77% (and up to 95.8%) of philosophers correctly predicted how the non-philosophers would respond.

A brief overview of the questions from the experimental philosophy literature we asked about and the results from our study are printed below. For a more detailed presentation of the questions we asked (which include verbaitim descriptions of the vignettes from the original studies conducted by experimental philosophers) and the results, go here:

http://www-personal.umich.edu/~dunaway/MetaSurvey.html

Boo Surveys?

Brian Weatherson has written a post prompted by the New York Times symposium on experimental philosophy. The post makes a lot of valuable points (the Austin bit is particularly interesting, and not something I’ve thought of), but there’s also a line that touched a pet peeve of mine. So I wrote a comment. Since the line is something I’ve heard around these parts, I decided to reproduce the comment here. Let me know what you think!

—

It seems that “I like the idea of experimental philosophy, but it just relies too much on the survey method” has become a common refrain in criticisms of experimental philosophy. I’ve always found this line of attack a bit puzzling, or at the very least, imprecise.

Diagnosis precedes prescription

The title principle seems obvious enough. Which makes it all the more puzzling that most normative political theorists ignore it in practice. Why is this? What are the implications?

The Apriority of Some Experimental Philosophy

Some experimental philosophy are apriori, so I claim. (More carefully, the conclusions that these projects draw are apriori.) On the face of it, this claim is rather implausible. If there is one thing that is most distinctive about experimental philosophy, it is the empirical methods that it borrows from psychology, cognitive science, and other allied fields. I want to argue that, however, in order to address a common objection against experimental philosophy, proponents would do better to concede the apriority of some projects that employ experimental methods. Fortunately for them, this concession can be made because there is an important distinction to be drawn between empirical/rational and aposteriori/apriori. The upshot is that both proponents and detractors would do well to note that experimental philosophy come in both apriori and aposteriori varieties.

Here is a rough taxonomy of projects that fall under the “experimental philosophy” umbrella. First, there are projects that are not survey based, but instead involve some observation on the experimenter’s part. Josh Greene’s fMRI works are paradigmatic examples. I think it’s uncontroversial that these are aposteriori. Second, there are “debunking” survey-based projects. These projects often argue against traditional philosophy claims from diversity of opinions. The cross-cultural studies on direct reference and knowledge are paradigmatic examples. I think these are aposteriori too, though I am relatively less confident. Third, there are “positive” survey-based projects. From people’s response to cleverly-designed thought experiments, experimenters draw conclusions about folk concepts. Of this kind of projects, Josh Knobe’s works on the moral component of intentionality are paradigmatic examples. In this post, I will argue that this last kind of experimental philosophy projects are apriori.

The common objection against experimental philosophy is that the responses that they get from ordinary people, which they call “intuitions”, are nothing relevantly like philosophers’ intuitions. Perhaps the folk do not have the relevant concepts employed in philosophical discourse. Perhaps the folk do not offer their considered, reflected judgments as philosophers do. If this objection succeeds, then experimental philosophy ought not have the impact on current philosophical practice that its proponents claims that it should. These so-called “intuitions” are simply not what philosophers ought to admit as evidence for their inquiries—in the same way that the fact that people sometimes say “I don’t believe God exists, I know it!” ought not count as evidence for knowledge not requiring belief.

Now, I find this common objection against experimental philosophy rather unconvincing, but I won’t debate that here. Instead, I want to simply note a dialectical point. To successfully respond to this objection, experimental philosophers need to do enough to show that the responses they get from ordinary people are relevantly like philosophers’ intuitions. The crucial point, then, is this: philosophers’ intuitions are apriori. If ordinary people’s responses are not, then that would seem like a relevant difference. To be more explicit, we can say that the content of ordinary people’s responses are apriori. Of course, experimental philosophers’ collections of those responses, or what we might call their observations of ordinary people’s responses, are empirical and aposteriori.

Philosophical Thought Experiment as a Genre?

A recent post by Brian Weatherson over at the Arche Methodology Project Weblog raises an interesting idea: can philosophical thought experiments be treated as a genre like, say, science fiction? This idea is also explored in Jonathan Weinberg’s article “Configuring the Cognitive Imagination” in New Waves in Aesthetics. Weinberg spells out the idea, without endorsing it, on page 214:

Yet, what if philosophical thought experiments were a genre—at least in the sense that engaging in them successfully requires mastery along the same lines as I have sketched for the mastery of literary genres? There are rules to engaging properly with a hypothetical scenario, after all. To make just some of the more obvious generalizations about our imaginative practices with thought experiments: one should embellish as little as possible; generally it is a practice conducted in an affectively `cool’ manner; and our inferential systems must often be brought to bear in this particular sort of imaginative project as well. And there are surely other, and more subtly articulable, rules for the proper performance of thought experiments still to be detailed.

While I was initially attracted to the idea—especially given my interest in imagination, fiction, and genre—I now think that it won’t do, on the more interesting interpretation. Roughly, the idea is to treat philosophical thought experiment as a genre relevantly similar to other genres of fiction. I have two worries with the idea, interpreted thus. In this post, I’ll press the first worry: we engage with philosophical thought experiments relevantly differently from the way we engage with fictions in other genres.

Intuitions: Role and Reliability

At Thoughts Arguments and Rants, Brian Weatherson gives a new argument for the reliability of intuitions. His main idea is that, given how many falsehoods are counterintuitive, there is a strong prima facie case for intuition being reliable. To deny this prima facie case, one must either deny that there is a fact of the matter about the reliability of intuitions or that there is a singular notion of intuition, but both of these options look bad. I found both Weatherson’s argument and the ensuing discussion in comments thought-provoking, so here are some of my thoughts.

My main point will be that it only makes sense to talk about whether intuitions are reliable with respect to what one thinks their role is in philosophical enquiry.

Think about a different case first. Suppose I claim that the Bush administration is reliable in interpreting military intelligence data. Well, there is that whole Iraq thing. But think about the good cases: they haven’t invaded Fiji on false intelligence, or Madagascar, or Sealand, or many other nations. The Bush administration’s interpretations are in fact correct in most cases. Boring, in the sense that these interpretations simply agrees with commonsense, but correct nonetheless. Therefore, I claim that the Bush administration is reliable in interpreting military intelligence data.

x-phi + a-phi = phi

This is the mathematical equation for reconciling philosophical methods that Dave Chalmers put up at the end of his excellent talk that closed the ‘Experimental Philosophy Meets Conceptual Analysis’ conference here at ANU.

Dave’s talk raised some interesting questions about what exactly experimental philosophy is and what exactly conceptual analysis is. ‘A-phi’ refers to ‘a priori philosophy’, which Dave suggested better captures the traditional projects of analytic philosophy better than ‘conceptual analysis’ does. After all, it does seem as though many ‘experimental philosophers’ are engaged in conceptual analysis of a sort, so using these as dividing lines might be a bit misleading.

A distinction was also drawn between ‘experimental philosophy’ and ’empirical philosophy’; my work on moral disagreement was cited as an example of the latter, since I didn’t actually do any of the experiments myself. The kind of work done by Joshua Knobe and Shaun Nichols, on the other hand, falls squarely into the former category.

Finally, ‘negative’ experimental philosophy was distinguished from ‘positive’ experimental philosophy; the former, Dave claimed, seeks to raise doubt about the philosophical status of intuition by finding disagreement in those intuitions, while the latter attempts to detect interesting and informative patterns in intuitions. I’m a bit skeptical of the stability of this distinction. My work on disagreement initially seems to fall into the former category, but I think it can also be argued that cross-cultural variation in moral beliefs is, in its own way, an ‘interesting and informative pattern’. So it seems that it’s possible to read one paper in both of those ways.

The overall message of the conference was extremely positive; the point made by lots of speakers (including myself) was that experimental, empirical, and a priori philosophy are all part of the same project, and that these methods can usefully complement one another. Thus while aspects of a question may require empirical or experimental philosophy- as in the case of moral disagreement, for example- other aspects of these questions may require more straightforwardly a priori inquiry — such as the question of whether expressivism is a conceptually coherent view.

Slides of Dave Chalmers’ talk should be available on his website soon, although they were not at the time of this writing. Those of you with access to GoBleen will be able to see the slides from my talk, ‘Moral Disagreement: Empirical and Conceptual Issues.’ In the meantime, feel free to use this space to discuss: what is at issue between experimental philosophers and their a priorist counterparts? Can’t we all just get along?