Should Scientific Methods and Data be Public?

At the last Eastern APA meeting in Philly, I attended an excellent session on The Epistemology of Experimental Practices, with Allan Franklin and Marcel Weber. During the discussion, I asked whether scientific methods and data should be public – that is, whether different investigators applying the same methods to the same questions should get the same data.

Franklin argued that publicity is not necessary, because some experiments might be too difficult or expensive to replicate, and different data analyses by different groups count as different experiments. This seems pretty wrong to me.

For one thing, I got the impression that Franklin didn’t fully understand what method publicity amounts to. Publicity does not require that all experiments be replicated; only that it is possible for different investigators to apply the same methods, and if they did, then they would get the same results. (Of course, much hinges on what we mean by “possible” and who counts as an investigator; for some more details, see here.)

For another thing, it’s better to say that actual replication of experiments is often unnecessary, as Marcel Weber said. Weber pointed out that experimentalists are part of a scientific network that shares techniques and materials, so they often feel they already know what was done. Nevertheless, Weber maintained that publicity is essential to science (and is implemented in the network itself, by the sharing of techniques etc.).

In fact, in his own talk, Allan Franklin listed a number of arguments/reasons for believing the results of experiments, along the lines of those listed in his Stanford Encyclopedia of Philosophy article on Experiments in Physics. All of Franklin’s reasons seem to have to do with publicity and the public validation of data.

Does anyone else have opinions on this? Should scientific methods and data be public or is this methodological principle obsolete?

I care about this because there are philosophers who have argued that introspection is a private yet legitimate method of observation, and this shows that method publicity is not necessary for science. I think this view is a disaster. If we reject method publicity, it’s not clear why we should reject all kinds of pseudo-scientific methods.

(And incidentally, I’ve also argued elsewhere that introspection is not a private method of scientific observation; rather, it’s a process of self-measurement by which public data are generated.)

(cross-posted at Itsonlyatheory.)

0

3 Comments

  1. I’m not an experimentalist, so I’m just venting from an armchair. It seems to me that publicity in this sense is essential to what we think of as scientific rationality. Clearly, if another group *can’t* reproduce a result, then something is wrong. I would expect hard cases — experiments difficult to reproduce for various reasons — but I’d hope that they were the exception.

    0
  2. Well, anyone who says that publicity is not necessary is actually helping pseudo-science and bad science. It is well-known to people who do experiments (I do in corpora linguistics) that even if you describe your setup, it might be extremely hard to get the same interesting results. People simply do not mention the most critical details in their papers to stop others from publishing new papers on it – if you’re really into the problem, you can guess what has been omitted, but sometimes you cannot even if you really try, as results are simply fake. If philosophers of science are happy with allowing fake papers to be published and help in scientific career, then they’re giving more respect to theft than to honest toil…

    0

Comments are closed.