25 March 2015

How to pick cherries

The not-so fine art of contriving to support the conclusion you predetermined is cherry picking.  Really not a good thing for a scientist to do or condone, but pretty common in politics.  The latest example comes from politician (now presidential candidate) Ted Cruz, being condoned/defended (even praised) by scientist Judy Curry.

Suppose you're interested in global warming, just in understanding what's going on -- not in 'proving' that there is warming, or cooling, or that temperatures are unchanged.  You're an actual skeptic -- looking for evidence and where the evidence leads.  One thing you learn pretty quickly in your skeptical explorations is that you need 20-30 years of data to define a global climate temperature.  Shorter than that, and your answer depends sensitively on your averaging period.  As a skeptic, you don't want such unreliable methods.  Apply the 30 years to a number of data records (below), and you get the answer that climate has been warming, 1.3-1.7 K/century (2.3-3.1 F).

As a cherry-picker, committed to finding a particular answer, however, you go straight for the option of using short spans -- look for a record length that will give you the answer you want, then ignore the fact that your answer changes if a couple years are added or subtracted.
Since there are different ways of looking at 'surface' temperature, you also cherry pick which data source to use.  For instance there are satellite observations (actually seeing the lowest 10 km of the atmosphere, peak sensitivity about 2.5 km above sea level).  And we've got thermometers 2 meters above the surface (much closer to where I live).  For both sorts of data, there is more than one group analyzing the data.  Cherry picker's delight.

I've pulled down satellite data from RSS and UAH for TLT 'global', and from NCDC and HadCRUT4 for thermometer records.  Then, compute trends to the present (for NCDC, I repeated 2014 in to 2015, so that all sets are showing trends to 2015/present).  That's what you see in the figure above.  30 years ago is 1985, which is where I read off the trend numbers I gave above.  The plotted trend is degrees per year, so I multiplied by 100 (giving degrees per century) to get the numbers above.

If you're a skeptic, you look at those four curves and wonder why RSS (purple) gives such different answers than the other three in the most recent 20 years.  It might be correct and the other three wrong, but you have to wonder, and be very uneasy about basing a conclusion solely on it.

But if you're a cherry-picker, you notice that some of the time, RSS shows a near zero trend, or even negative.  You go straight to that one source, and ignore all others.  Then pick the oldest time when you can get your desired result -- 17 years ago (as I write).  You ignore that if you chose 15 or 19 years instead you'd have a positive trend.  And you ignore that for climate, for reasons which I illustrated six years ago (and which are much older than that, and based on principles used in many fields), you want 20-30 years.  If you pick just the right time span, from just the right data set, you can have the cherry you want.

If you're a skeptic, however, you follow up on the RSS data set and see what might be going on.  If you're a hard core data type, you get the MSU and AMSU original data and reprocess itself.  Most people, though, are better served by checking out the original source.  See what Carl Mears of RSS, the person leading that product's construction has to say about his product. 

Do read the article.  A couple of representative quotes (but, of course, don't take my word for it):

Does this slow-down in the warming mean that the idea of anthropogenic global warming is no longer valid?  The short answer is ‘no’.  The denialists like to assume that the cause for the model/observation discrepancy is some kind of problem with the fundamental model physics, and they pooh-pooh any other sort of explanation.  This leads them to conclude, very likely erroneously, that the long-term sensitivity of the climate is much less than is currently thought.
The truth is that there are lots of causes besides errors in the fundamental model physics that could lead to the model/observation discrepancy.  I summarize a number of these possible causes below.  Without convincing evidence of model physics flaws (and I haven’t seen any), I would say that the possible causes described below need to be investigated and ruled out before we can pin the blame on fundamental modelling errors.


and:

Measurement Errors:
As a data scientist, I am among the first to acknowledge that all climate datasets likely contain some errors.  However, I have a hard time believing that both the satellite and the surface temperature datasets have errors large enough to account for the model/observation differences.  For example, the global trend uncertainty (2-sigma) for the global TLT trend is around 0.03 K/decade (Mears et al. 2011).  Even if 0.03 K/decade were added to the best-estimate trend value of 0.123 K/decade, it would still be at the extreme low end of the model trends.  A similar, but stronger case can be made using surface temperature datasets, which I consider to be more reliable than satellite datasets (they certainly agree with each other better than the various satellite datasets do!).  So I don’t think the problem can be explained fully by measurement errors.

Hmm.  Even a cursory check in with the producer of the data shows no signs that he believes it disproves warming, or shows no warming.  Some issues worth researching on why the difference between models and his data is all (for now).

But cherry-pickers don't need to read the data sources.  It's skeptics who want and need to understand what they're looking at.

1 comment:

jg said...

It's been awhile since I dropped by -- my loss. I'm glad to see you're still simplifying climate science. I enjoyed this article.
jg