How to find climate normals?

It's that time of the decade when the official 'climate normals' are computed -- according to the rules of the WMO and NOAA (in the US).  But can we find a better way of deciding 'normal'?  I'll do some thinking out loud here, and invite you to play too.  Could we even be so lucky as to find a way of defining climate normal in such a way that we don't need to worry about an early period of 'coming out of the Little Ice Age', or a later period of Anthropogenic Global Warming.

In previous posts, I illustrated that there are principles which lead to the requirement of 20-30 years to define a climate average and, separately, 20-30 years to define a climate trend.

In those approaches, we were able to make good use of the adage that climate is what you expect.  It was only after 20-30 years of data that our expectations for the mean or trend would be stable -- would not depend sensitively on how long a period we chose to be our data period.

This time, I'll pick up with a different notion or description.  A common description of climate is also that sometimes its warmer and sometimes it's cooler, but it all averages out in the end.  Let's start by looking at the Hadley-CRU temperatures, back to 1850, month by month.  I'll start with looking at temperatures relative to the average over the entire data set:

For the earlier part of the record, we see the deviations being both above and below the reference value.  On the other hand, the last 26+ years, every month since December 1984 -- 317 consecutive months, has been above the reference value.  That's not a sign of sometimes warmer, sometimes cooler.  That's the recent period being different from 'normal'.  For instance, because of a warming trend.  If you're younger than 26, you have never seen a month where the global mean was as cold as the 161 year average.  On the other hand, looking at the early part of the record, we see some years warmer than 'normal' even though most are cooler.  Further (look directly at the original data, don't take my word) there are no periods as long as even 20 years of continual below reference temperatures.  In other words, the Little Ice Age, insofar as it was global, had ended by 1850.

But, can we do something else to explore the notion of 'normal'?  Those wiggles in the curves are not all the same size.  If we were to add them up, would they sum up to zero?  They have to do so over the whole record -- that's what the average means; you have just as much above as below average, by just as much, observations.  If the numbers are mostly just wandering around, then the sums from the start of the record to a given month should also wander, sum times totalling more than zero, sometimes less.  Here's what happens for using the 161 year average as the reference:

That's clearly not a matter of climate wandering around the reference point we chose!  The first 80 years are almost uninterrupted decline in total temperature anomaly, while the last 30 are an aggressive warming -- in 30 years compensating for the 80 years of cumulative cooling at the start of the record.

A word about what this sum means.  It is an accumulation of heat or cold compared to our reference number.  It is the same concept as heating degree days, frost degree days, and cooling degree days.  Since it is summer here, I'll go with cooling degree days.  (I'll actually describe it in hours.)  Suppose you're ok with temperatures as warm as 77 F (25 C), but will want to run the air conditioner if it is warmer than that.  Of course the air conditioner has to run harder if it is hotter.  So hour by hour (day by day) what you do is add up how many degrees warmer than 77 you are.  A day with 5 cooling degree hours means you don't run the air conditioner much.  A day like today here will have something like 200 cooling degree hours.  You appreciate very much the invention of air conditioning on a day like this!

In the case of global mean temperatures, month by month, what we have above is cooling degree-months.  Let's say that glaciers all experienced global mean (which, of course they don't, but I'll use for the sake of description).  Further, let's suppose that glaciers were all in balance in 1850.  And (again not exactly true) let's assume that colder temperatures mean glacial growth.  That large accumulation of cooling degree months suggests that glaciers should have been growing from 1850 through 1930, then retreating until the present -- just now retreating to their position in 1850.  We actually observe that glaciers have retreated behind their 1850 locations, so, again this doesn't look like a good reference to choose.

The good reference temperature, and good definition of climate 'normal' will show us some temperature accumulations that cross back and forth across zero.  And about half the months should be above normal, or below normal.Given the original figure, we don't want to include the last 30 years at least.  So I took 1850-1979.  Well, again, didn't behave very well.  I backed it up a decade at a time until I finally found a period which gave a well 'behaved' in these senses 'normal'.  It is 1850-1939:

Now this looks like what we've been told climate is like -- sometimes warmer, sometimes cooler, and on average not going anywhere.  The sums hit zero 3 times in these 90 years -- about 1856, 1880, 1892, come close around 1902.  The zeroes in 1850 and 1940 are because that's how we constructed the curve, so they don't count.  Having found a period that behaves like a 'normal', now let's look at the rest of the resulting curve to the present:

Whoa!  Not only are there no more zero crossings, meaning that climate's accumulated tendency for the last 70+ years is one of warming, but it's hard to even notice any declines in the sums.  They're there, but you have to look very carefully on this scale.

The scale is another thing to look at carefully.  In the first 90 years, the largest accumulation we see is a net cooling of 30 degree-months.  It took about 20 years to get there (about 240 months), and about 20 years to warm back out of it.  Those mean a cooling or warming bias of about 1/8th (0.125) degree per month being a normal number in climate.  Numbers of that magnitude have appeared elsewhere (if you've got a memory like mine, you might recognize figures like this from earlier posts, otherwise, don't worry) which grants a certain degree of comfort.

But the more recent period, 70 years of it, shows 10 times as large an accumulation as the 'normal' period!  328 degree-months accumulating in 70 years.  This averages to a warming bias of 0.39 degrees per month, triple what happened in the normal period.

(Update) You can see the scale issue more easily in this figure, with the whole period on the same scale:

I confess I was quite surprised to see this result.  One response to that is to invite you all to find my mistake (if you can't find it, maybe I didn't make one).  My spreadsheet is available in Open Office and Excel formats.  The other thing I'll invite you to do is to use different climate data sources.  Maybe this conclusion depends on having used the HadCRU data?  Let us know what you get if you use the NCDC, NASA, JMA, and so forth data sets instead.

In the mean time, it looks like there has been a fundamental change in climate since 1940.  Something has been active since 1940 that wasn't active before 1940.  The good news being that we actually arrive at a period when we can call climate to be 'normal' -- 1850-1940.  Further, since the NOAA and GISS data start in 1880 rather than 1850, it's reassuring that our reference choice here shows 1850-1880 to be a span of zero accumulated cooling/heating.  So we could reference 1880-1940 instead.

Something else we can do is take a look at the reference value again, and how long it has been since we've been below that.  Using the anomaly values as given by HadCRU, the normal value for climate is -0.334.  The normal period they chose was quite a bit warmer than the 'normal' period of the climate system it turns out.  The last time the global mean was below the climate normal was March, 1976.  If you're 35 or younger, you have never seen a global mean below climate's real normal*.  The most recent month in the data, May 2011, is 0.667 K above the climate normal, 1.2 F warmer than normal.

*Assuming, of course, that I didn't make a mistake somewhere.

Update2: It seems that blogger is having problems with comments.  That's particularly unfortunate here as I think there are likely many good comments getting choked by the system.  If you've had this problem, please email them to me at bobg at radix dot net.  3: Problems seem to be resolved now.

Update4: Tamino has some illustration and discussion of hazards of using cumulative sums.  It's a statistical argument, naturally, so this affects portions of this post which are statistical in nature.  On the other hand, as one of the commenters there noted, there's a well-known and important paper on climate by http://www.aos.princeton.edu/WWWPUBLIC/gkv/history/Hasselmann76.pdf Hasselmann, 1976 which points out the physical importance of cumulative sums.  I'll take up these and related ideas in a full post.

Update 5 (8/8/2011): Let's also take a look at how close climate is to being random