Excessive precision is one of the first methods mentioned in How to Lie With Statistics. It's one that my wife (a nonscientist) had discovered herself. It's very common, which makes it a handy warning signal when reading suspect sources.
In joke form, it goes like this:
Psychology students were training rats to run mazes. In the final report, they noted "33.3333% of the rats learned to run the maze. 33.3333% of the rats failed to learn. And the third rat escaped."
If you didn't at least wince, here's why you should have. In reporting scientific numbers, one of the things you need to do is represent how good the numbers are. In order to talk about 33.3333% of the rats, you'd have to have a population of a million rats or more. 33.3333% is saying that the figure is not 33.3334% or 33.3332%. You only should be showing as much precision as you have data for. Even though your calculator will happily give you 6-12 digits, you should be representing how accurate your number is. In the case of the rat problem, if 1 more rat had been run, one of those 33% figures would change to 25 or 50. The changes of +17% or -8% are so large that they should not even have reported at the 1% level of precision. What the students should have done was just list the numbers, rather than percentages, of rats all along.
As a reader, a useful test is to look for how large the population is versus how many digits they report in percentages. Every digit in the percentage requires 10 times as large a population. Need 10 for the first digit (again, the psych. students shouldn't have reported percents), 100 for the second, and so on. A related question is 'how much would the percentages change with one more success/failure?' This is what I looked at with running the extra rat.
Related is to consider how precise the numbers involved were at the start. When I looked at that bogus petition, for instance, I reported 0.3 and 0.8%. Now the number of signers was given in 4 or 5 digits. That would permit quite a few more than the 1 I reported. The reason for only 1 is that I was dividing the number of signers by the size of the populations (2,000,000 and 800,000) -- and the population numbers looked like they'd been rounded heavily, down to only 1 digit of precision. When working with numbers of different precisions, the final answer can only have as many digits precision as the worst number in the entire chain.
An example, and maybe the single most commonly repeated one from climate, is this page, which gives (variously, but table 3 is the piece de resistance) the fraction of the greenhouse effect due to water vapor as 95.000% That's a lot of digits!
Let's take a look at the sources he gives, and then think a little about the situation to see whether 5 digits precision is reasonable. Well, the sources he has valid links for (1 of the 9 is broken, and one source doesn't have a link; I'll follow that up at lunch at work in a bit) certainly don't show much precision. Or being scientific, for that matter (news opinion pieces and the like). My favorite is the 21st century science and technology (a LaRouche publication), whose cover articles include "LaRouche on the Pagan Worship of Newton". The figures given are 96-99% (LaRouche mag), 'over 90%', 'about 95%', and the like. Not a single one gives a high precision 95.000%, or a high precision for any other figure. This should have been a red flag to the author, and certainly is to us readers. Whatever can be said about the fraction of greenhouse effect due to water vapor, it obviously can't be said with much precision. Not if you're being honest about it. (We'll come back in a later post to what can be said about water vapor, and it turns out that even the lowest of the figures is too high if you look at the science.)
Now for a bit of thinking on water vapor. The colder the atmosphere is, the less water vapor there can be before it starts to condense. (It's wrong to call it the atmosphere 'holding' the water vapor, but more in another post.) It also turns out to vary quite a lot depending on temperature. In wintertime here (0 C, 32 F being a typical temperature), the pressure of water vapor varies from about, say, 2 to 6 mb. In summer, it's more like 10 to 30. (30 million?! It gets very soggy here, though not as much as Tampa.) On a day that it's 30 mb here, it can be 10 mb a couple hundred km/miles to the west. Water vapor varies strongly through both time and space. As a plausibility test, then, it makes no sense for there to be 5 digits precision to the contribution of something that varies by over a factor of 10 in the course of a year, and even more than that from place to place on the planet.
Make Me Dream, Under the Stars
40 minutes ago