Peer review

"It's peer review, not God review." was my wife's response when I asked her whether she thought peer review meant that a paper was 100% correct in all details. She's right, of course. Humans are fallible, and scientists are human. It seems to surprise nonscientists that we've noticed both of those things. In fact, this is part of why peer-review was invented. Scientists do try to do good, careful, accurate work that is without error. But we know that we probably never accomplish this in our own original writing. Having peer reviewers gives a chance for errors of omission and commission to be caught before you take the paper public (rather, before the journal takes the paper public). Or just plain have some other folks, not as intimately involved in your work as you, look over your explanations for clarity and completeness.

Even after peer review, misteaks can remain. Fallible humans can miss errors even when they're just the reviewers. Again, we know this, too. That's why the significance of peer review is not that it is a stamp of perfection, but a stamp of 'there's a pretty fair chance that this work is worth your time to look at'. Conversely, things which aren't peer reviewed are not even peer reviewed. It doesn't mean that the paper is necessarily bad -- a usenet faq of mine (not formally peer reviewed!) has been cited in the peer reviewed literature (Noerdlinger and Brower) as being the first discoverable time that someone computed how much sea level rise there can be from melting the ice packs. (It isn't zero, see the faq for why and how much. See Noerdlinger and Brower for the more complete considerations and experimental demonstration.) But, given that humans are fallible, and time and reading speed is limited, a mark that there's a better chance than usual that the paper at hand is not obviously (to the reviewers) false is a very helpful thing.

The real test, though, is not whether a paper was peer reviewed. Many papers survive peer review that are never cited by anyone other than perhaps the author. These are dead ends. Since I heard that, I check periodically to see whether my papers get cited. They all have been, and not just by me or someone I was working with at the time. Good to know. The really good papers are the ones which get used often to do new science. Science hangs together, in that if I did my work right, someone else can use it to study, say, polar bears, eider ducks, or something -- things which I had no idea of when I did my work. (I'm not a biological science type, though I find it interesting to hear about.)

The polar bear guy used my work because in the areas he could test it (tracking dead polar bears with transmitters), my work on sea ice drift matched well. He then had confidence that in the areas he couldn't test, the work was probably also worth paying attention to. This is where the good science has happened -- somebody else, for some other reason, can see your work pass an independent test and use it.