Should Scientific Methods and Data be Public?

At the last Eastern APA meeting in Philly, I attended an excellent session on The Epistemology of Experimental Practices, with Allan Franklin and Marcel Weber. During the discussion, I asked whether scientific methods and data should be public – that is, whether different investigators applying the same methods to the same questions should get the same data.

Franklin argued that publicity is not necessary, because some experiments might be too difficult or expensive to replicate, and different data analyses by different groups count as different experiments. This seems pretty wrong to me.

For one thing, I got the impression that Franklin didn’t fully understand what method publicity amounts to. Publicity does not require that all experiments be replicated; only that it is possible for different investigators to apply the same methods, and if they did, then they would get the same results. (Of course, much hinges on what we mean by “possible” and who counts as an investigator; for some more details, see here.)

For another thing, it’s better to say that actual replication of experiments is often unnecessary, as Marcel Weber said. Weber pointed out that experimentalists are part of a scientific network that shares techniques and materials, so they often feel they already know what was done. Nevertheless, Weber maintained that publicity is essential to science (and is implemented in the network itself, by the sharing of techniques etc.).

In fact, in his own talk, Allan Franklin listed a number of arguments/reasons for believing the results of experiments, along the lines of those listed in his Stanford Encyclopedia of Philosophy article on Experiments in Physics. All of Franklin’s reasons seem to have to do with publicity and the public validation of data.

Does anyone else have opinions on this? Should scientific methods and data be public or is this methodological principle obsolete?

I care about this because there are philosophers who have argued that introspection is a private yet legitimate method of observation, and this shows that method publicity is not necessary for science. I think this view is a disaster. If we reject method publicity, it’s not clear why we should reject all kinds of pseudo-scientific methods.

(And incidentally, I’ve also argued elsewhere that introspection is not a private method of scientific observation; rather, it’s a process of self-measurement by which public data are generated.)

(cross-posted at Itsonlyatheory.)