Testing Ideas 2

Finally, in comments on my cherry-picking article, I was invited to take a look at http://rankexploits.com/musings/2008/ipcc-central-tendency-of-2ccentury-still-rejected The commentator didn't really say what I should be learning from the link or what relevance he thought the page had to cherry-picking. I discussed some basics about making good tests in part 1 on testing ideas. Please check out both prior posts as I'll assume you know their contents. It's a little work, I know, but in order to have intelligent disagreement, or agreement, we have to know what the other is saying.

The title of that article is "IPCC Central Tendency of 2C/century: Still rejected". Certainly catchy. But let's look at the substance: Does the IPCC say there's a central tendency (and if so, what is it that is supposed to have that central tendency) of 2C / century? An immediate red flag for me is that the site does not provide a reference to where the IPCC said any such thing. Being better about doing my homework, I went to the IPCC WG1 report itself and looked in the summary for policy makers and the chapter (10) which discussed the global climate projections. I invite you all to do so as well. In my look for 'central tendency', I found no such term in either section. In my reading, which has not been exhaustive, of the two sections, I still found nothing that could be construed as a 'central tendency'. The projections were projections, not forecasts, and were made for a number of different scenarios (assumptions about greenhouse gas emissions and other things). Some included ensemble means, but none of those were 2.0 C. One was 1.8, but calling that 2 is back to the problem of deciding whether I'm tall and letting me round to the nearest foot or meter. Not even out of the title and already there are problems.

In the lead paragraph, the author writes "... compared to the IPCC AR4’s projected central tendency of 2C/century for the first few decades of this century." Again, I don't find the IPCC saying that, and again, the author doesn't say where the claim comes from. The nearest match I find is in the Summary for Policy Makers, p. 12, where it says "For the next two decades, a warming of about 0.2 C per decade is projected for a range of SRES emission scenarios." The author mutated a term of precision, two decades, in to a vagary, the first few. Then the fuzzy term 'about 0.2 C per decade' got cast as a hard term of 2C/century. If nothing else, in reading this site, you're not reading a reliable reporter. Once I've reached that point, I generally stop reading a source. There was no need to misrepresent the original. Nothing was saved or simplified.

In terms of part 1, remember, I mentioned that you had to be careful to make a good test, including that it had to test the thing at hand. In that case, you had to be looking at my height and weight in order to decide whether I was 'tall and thin'. In this case, you have to compare (in some 'good' way) observations that bear on the thing actually predicted by the IPCC. When you read chapter 10, you'll discover several things. One is that the variable being projected is the global average surface air temperature. Another is that the models have interannual variability (they'd better -- nature does, as I mentioned in the cherry-picking and detecting climate change articles). And you'll see that 30 years is the normal period for averaging temperatures to make climate conclusions; but even so, the projections were being (gingerly, you'll notice if you know how scientists write) made about 20 year averages. One final thing buried in those 100 pages: The projections assume that there are no volcanoes and the sun's input remains fixed at the observed average value. We haven't had any major volcanic eruptions since Pinatubo, but the sun has been quiet since 2001 (below the average output).

So several things to go in to making a good test:

• We have to compare 20 year average vs. 20 year average (the variable being suggested as meaningful in the report, and 30 would be better).
• We have to look only at global mean surface air temperature.
• We have to adjust for the fact that the sun has been giving less energy than assumed in the projections.
• In making the test, we have to allow for the fact that the 0.2 C/decade itself has error bars (due to interannual variability and between-model variability)

The author does none of these things, and presents no reason why they are not necessary. Instead, she computes trend lines for January 2001 - present, not 20 year averages. She includes satellite observations of temperatures through the lower-mid troposphere (the UAH and RSS), rather than using only surface air temperatures. She ignores that the sun has been quiet. And she makes no allowance for the error bars on the IPCC projection.

Even if absolutely everything that was done in the statistics were right, which I'm not in a position to say much about, the test is not a good test so the result is at best meaningless. Chances are good that it's misleading (tests that aren't good are usually misleading).

The satellite temperatures show a cherry pick themselves. But first, why they shouldn't be used in this context in the first place. The thing is, they're not observing the variable that is being predicted. While it is connected, which might lead us to thinking that it's ok to use them, it still may not be. Back to the question of whether I'm tall. I think that the data you want is a measurement from the floor to the top of my head. It's true that leg length is related to height, at least in the sense that tall people generally have longer legs. So maybe you'd accept that instead. And say you took Michael Phelps' leg length too. If you looked at the two, you'd conclude that I'm taller than Phelps. (His legs are short for his height, mine are long for mine.) You'd be wrong, however; the data don't address well enough the question you're asking.

The cherry pick is that only 2 of the 4 satellite temperatures were taken, and it happens that the two are the two which show the least warming (you'd have to know about this, which is easy enough to find if you look but isn't universal knowledge). The author gives no reason for this selection. Further, even ignoring that, one of the two also is not a global data set. The RSS goes only 70 S to 82.5 N. There are good reasons for this, and, conversely, to not be confident about the UAH figures, but they get technical. It suffices here that only 2 of 4 data sources were picked, one of them isn't even global, and, worse, neither measures the variable of concern.

Even without knowing anything about statistical methods, we can see that the given site does not support its headline (and have some question about the headline as well). All that is needed is to check what the source (IPCC in this case) actually said, versus what was being tested. They're different things, so the test doesn't tell us about IPCC's projections.

We can't make the converse conclusion from this, that the IPCC projections are correct. We didn't test that idea, so no conclusion about it is possible here. We only checked whether the site was making a good test. It wasn't.

A non-digression to something related. While I've taken probability and statistics courses, they weren't very deep (I felt). I was thinking about taking more, and asked a coworker -- a statistician -- about doing so. We talked some about what I'd studied (and remembered, this was a good 15 years after I'd had the classes). She concluded that there was no real point for me to take the courses. More important, she said, was to understand the system I was working with. The appropriate statistical methods would suggest themselves, or at least I'd be able to hand a well-constructed question to a statistician. But without understanding the system, there was no point in applying statistics.

Project: Pull down the 5 data sources given at that site and compare the last 20 years (7/2008 to 8/1988) average global temperature against the previous 20 years, 8/1968 to 7/1988. Is the more recent average greater or not? Can't do that for the satellites as they don't go back that far. But try 8/1979 to 7/1994 versus 8/1995 to 7/2008. Only 15 years versus 14 years, so not very informative about climate, but it's all the data we have. Won't be a good test, but the best (perhaps misleading) that can be done given those data. (Try changing the periods too.)

johnathansawyer: as you invited me to look at the source, did the above affect your opinion about it? Why or why not? As usual, substantive reasons (either way).

[Update 28 August 2008] See my next two notes what is climate - 2? and evaluating climate trend for more on what appropriate averaging periods look like, and what happens if you compare the average of the last 7.6 years to the previous 20 years' average, respectively.

[Update 29 August 2008] rankexploits made a lengthy commentary on things related to this post, though misrepresenting even my first paragraph. Comments on that post are disabled (I get an error message in response to my comment attempt), my response is #5317 in http://rankexploits.com/musings/2008/ipcc-central-tendency-of-2ccentury-still-rejected

Folks who are getting heated about my 'advocacy', or defense of IPCC, or whatever. Take a minute. Read what exactly I said. Saying that a particular test was not good is far from saying that there can be no such test (I give an example of one that would be better myself!). Nor, as I said directly, does it mean that the projection is good. Let's see what happens when a good test (which mine isn't, just better) is made. Until one is presented, a poor test is still not useful.