Stem cells and scientific papers

I don't know how many people are shocked by the revelation that much of the evidence in THE stem-cell paper was faked. If you don't know what I'm talking about, a team of scientists, led by a Korean, have claimed to succeed at therapeutic cloning; that is, inserting a human's DNA into an unfertiziled human egg, then extracting the stem cells from the embryo and directing the stem cells to grow into specific kinds of tissue. (I'm probably not explaining this entirely correctly.)

In the last few weeks, one disturbing revelation after another has emerged: first that some of the senior scientists took their eggs from some of the junior scientists, an ethical no-no; second that some of the data had small but fixable errors; now that some of the data has been faked outright. Senior authors of the paper are suddenly disavowing the study and asking for withdrawal of the paper. What began as the clarion call of a new, glorious age has collapsed in the stench of corruption, and the Korean scientist who only a few weeks ago was a hero to his nation is now defending his integrity.

One would think that this should be impossible in the rigorous world of science. If you want your paper published in a scientific journal, you expect that it will be carefully fact-checked. This should especially be the case with the world's premiere scientific journals. After all, aren't scientists obsessed with discovering great truths of the world?

In my own experience, I have refereed some articles. I read them carefully, trying to understand the result and the details of the proof. In one case, I found a fatal error; in the report, I pointed out a counterexample to the authors' claim, then suggested how they could fix the error and still save the result (which was a very nice result, if it was in fact true).

It's difficult to meet the twin standards of rigor and clarity. My advisor didn't believe me when I told him that I had discovered evidence that suggested the major result in my dissertation. One problem I had then, and that I still have, to a lesser extend, is in explaining just what I've proved. I have a nasty habit of saying things hastily and carelessly, and it's burned me on several occasions.

In that case, there was significant consensus among many researchers (my advisor included) that the opposite fact was true. My advisor was convinced that there was a fatal flaw in the counterexample I claimed to have. You can imagine my embarassment and disappointment when he discovered a serious flaw! Yet he also saw that it was not fatal; there was something substantial going on. Even though my explanation was wrong, the claim itself had a troubling aura of correctness. Indeed, he found a way to correct the flaw that very evening. I worked independently for another year or so, until I came to the final result. That was last year, and we are still working on a journal article to document the result. The article has substantial revision on two or three occasions, but that has been in terms of clarity, not of rigor; the proof has remained unchanged for the past year. God willing, the paper should see the light of day Real Soon Now. (I'm waiting on my former advisor to finish another look at it.)

The point of this example is that we humans are humble creatures who struggle with logic and understanding; we stretch out our hands into the mist of the unknown and stumble upon something we've never felt before, then try to describe it in words that are terribly inadequate for the occasion. Plato's allegory of the cave is incredibly appropriate to the venture.

I am inclined to believe that the research team actually did accomplish something of scientific importance, and they probably accomplished what they claimed to accomplish. During the time between the result and the completion of the article, Bad Things Happened, and it was easier to cut corners than to redo the entire experiment, or consider the implications of these Bad Things. I say this despite the fact that I consider the cloning and destruction of embryos inherently unethical.

I also wonder about the thoroughness of the refereeing. Several mathematicians have told me that they never check the details of an article. They don't have the time, and they view such work as uninteresting drudgery. Instead of checking the details of the proof, they check the structure of the article. Is it well-organized? Is the problem clearly explained, and did the author in fact solve what he claimed to solve? Are the lemmas and theorems phrased clearly and correctly; do they appear relevant and concise? If the paper satisfies these criteria, as well as criteria of importance and completeness, then the paper is recommended for publication. Otherwise, the paper is recommended for rejection, or at least for revision.

This may seem like a strange way to check science, but from an intuitive point of view it makes sense. An author who takes the time to structure a paper for clarity and sense is almost certainly expending the requisite effort to verify the details. Mathematical proofs of new results are typically non-trivial, and the more interesting results usually require innovative techniques with which the referee wouldn't be familiar. Learning such techniques requires effort and time, and even then the referee may not understand the paper. Many results can be verified with simple benchmark examples, and it is a fact of human nature that unusual examples convey a compelling sense of correctness.

Something else that helps or hurts a paper is the name and reputation of the author. This shouldn't matter, and many journals send articles out to referees with the author's name removed. This doesn't always help. Only experts in the author's field can understand a paper; since most experts know all the other experts in their field, it is common to recognize the author(s) of a paper simply by the title of the paper. Certain authors specialize in certain topics; for example, my advisor was for a while the only researcher working on the computer algebra of composed polynomials. (To my knowledge, he pioneered it.)

I don't know if world-class investigators of stem-cell science also consider the same "non-scientific" criteria when refereeing a paper, but I have a strong suspicion that this is exactly what happened in the case of this research. The result was so important in the minds of many scientists that they wanted to believe it was true, and the senior authors' names conveyed sufficient weight of credibility that the referees let some things slip.

I'm not sure how they would have caught some problems, such as the photos that were manipulated by a computer program. It's quite possible that none of the errors could have been caught; this may have been beyond the level of even a dedicated referee. And as I say, I suspect the scientists really did achieve what they claimed to have achieved.