Forecasts and their Value

Economic Value of Weather and Climate Forecasts, Richard W. Katz and Allan H. Murphy, editors, 1997 includes some hard core math.  But the idea explored is straightforward enough, and much of each paper included is spent on the considerations which direct the mathematics, so you needn't be up on the math to gain from the reading. 

Fundamentally, a forecast has zero economic value if it can't be, or isn't/won't be, used to increase a profit or reduce a loss.  The value lies in the decisions which can be (and are) made based on the forecast, and not the forecast's accuracy (abstractly considered) itself.

On an extreme example, the value of climate forecasts to James Inhofe is zero.  There is nothing, given his public statements, he would do in response to a climate forecast (regardless of how good) differently than with no information.   Also limited value of hurricane forecasts 5 days ahead to Rush Limbaugh, who dismissed (September 5th, 2017) the (extremely accurate, as it turned out) forecast of Hurricane Irma's landfall in Florida on the 10th.  On the 5th (follow link to news story with the details), he was dismissing the forecast as fake news / liberal conspiracy, and advising his listeners to ignore the forecast.  On the 8th, just 2 days ahead of the storm, he evacuated from Florida.  Given his listeners and advertisers, it may well have profited him to delay response.  People who couldn't evacuate because they listened to him for too long, different matter.

But it illustrates a different issue -- lead time and actions.  If you don't (can't, won't) do anything differently with 5 days' lead time than with 2 days' lead time, there's no value to you in the extra lead time for the forecast.  Scientists, of course, are very interested in the difference -- the better we understand hurricanes, the better (farther ahead) we can predict them.  But that's hard to put a dollar value to.  For me, certainly, 5 days lead time in knowing a hurricane is coming is far better than 2 days.  It gives me time to prepare the house for the winds and waters, and to make a considered retreat (meaning no traffic jam) outside the range of the hurricane.  With just 2 days warning, that becomes hard to do.  On the other hand, it would be no more helpful to me to know of a hurricane coming in August 16, 2019 than to know of one coming August 16, 2018.  And August 16th is probably no more useful to me than July 16th (from my vantage of June 3rd).

There are people and interests other than me and mine, however.  Home supply and repair stores, for instance, might profit greatly from knowing that they'll need a large stock of material and staff prior to a particular date.  Or even just that odds are higher than usual that their area will have a hurricane.

What are some weather or climate decisions you make?  How much difference does the accuracy of the prediction make?  How far ahead does it matter for your decisions to have the prediction?

Recent reading

I'm a bookaholic, I confess.  I have far more books than are strictly needed.  And I'm acquiring more essentially all the time.  (The freebies available via google, ibooks, kindle, nook, and many other venues don't exactly slow down my acquisition.)  On the other hand, I do eventually read them.  From recent (-ish) reading:

Hands on Meteorology by Zbigniew Sorbjan -- a book with something of everything for meteorology and middle school students (or older).  Some history, some biography, and a substantial chunk of hands on meteorology.  Plenty of experiments that you can do with minimal experience and equipment. 

Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving, Sanjoy Mahajan -- To get through to the end of this book, you'll want at least integral calculus.  But I mention it here because a) some of you have that background and b) those who don't: consider the title.  You can choose to consider mathematical problem solving as being something like a mixed martial arts, steel cage, match.  No holds barred either.  While math is often taught as a matter of exactness, and the one and only one correct answer, there's a broad swath in which coming up with a pretty good approximation is an excellent thing.  In practice, this is an enormous swath of science*.  See also my old post Fermi Estimate Challenge.

Native American Crafts and Skills, 2nd Ed., David Montgomery -- It's easy to make, say, a house when you already have plans, bricks, saws, (pre-cut!) lumber, plumbing, electricity, and so on.  But what do you do when you only have stone tools?  How about when you also have to make the tools themselves?  There's some serious intelligence involved in solving these problems.  This book has some of the solutions.  In a few cases, such as the shape and orientation of a Tipi, there's also a connection to meteorology and climate.

What We know About Climate Change, Kerry Emanuel -- This is a far smaller book than I expected from the title.  It also includes no math.  It's a good place to start reading on climate.  It won't take you long, and won't bury you in detail or math.

More to come ...


* Post to come about Ted Fujita, and his 'chapter one' rule.

Earth-Sun distance and Chandler Wobble

Continuing from The Pacemaker of the Chandler Wobble, Grumbine 2014:

The Chandler Wobble (CW) is a small variation in the orientation of the earth’s rotational axis [Chandler, 1891]. It has a period near 433 days [Liao and Zhou, 2004] (0.8435cycles per year, 0.0023095 cycles per day). Some source of energy for the Chandler Wobble  must exist because it dies out on a time scale of decades [Munk and MacDonald, 1960] if energy is not continuingly added. Gross [2000] found that atmosphere-ocean forcing on the earth’s rotation, computed in an ocean general circulation model driven by observed  meteorological parameters, provided that forcing. [O’Connor et al., 2000] also found wind forcing of the ocean to drive the pole tide. This source was questioned [Wunsch, 2001] partly on the grounds that the ocean was displaying a very narrow band response, but there was no reason to believe that the forcing itself was narrow band.

I suggest that the atmosphere-ocean variability near the Chandler Wobble period, among others, is paced by variation in earth-sun distance. The earth-sun distance, in addition to annual and semi-annual variations due to the elliptical shape of the earth’s orbit, varies due to perturbations from the moon (29.53 day period and others), Venus (292, 584, 417, 1455, ... days), and Jupiter (399, 199, 439, 489, ... days). The size of these variations is small, the largest being the 29.53 day lunar synodic period (31*10−6 Astronomical Units), amounting to approximately 0.08 W/m2 on a plane perpendicular to the sun at the top of the atmosphere. See Table 1 for more precise periods and the amplitudes of distance variations corresponding to them.

Horizons [Giorgini et al., 1996] was used to provided 6-hourly earth-sun distance and osculating elements for 1 Jan 1962 00 UTC through 31 Dec 2008 18 UTC. Table 1 was derived by harmonic analysis of those data at precise frequencies to determine purely cyclic variations in the earth-sun distance. The leading terms are, of course, the annual and semi-annual cycles from the elliptical orbit. Following this, however, are perturbations in Earth-Sun distance due to the moon, Venus, and Jupiter. Note that the orbital elements are not precisely locked to the periods given. The osculating (instantaneous) orbital elements vary; the osculating year varies from 364 to 366 days, for instance [Giorgini et al., 1996]. Consequently, there are residuals near the annual period. But they are far smaller than the main line. The anomalistic year, 365.259635 days [Observatory and Observatory, 2001], is the period between successive perihelia. This has been found to be the appropriate period for climate temperature analysis rather than the tropical (vernal equinox to vernal equinox) year [Thomson, 1995]. As we will be drawing the conclusion that earth-sun distance is important, even for small variations, the anomalistic year is the self-consistent one to use here. 

Previous analyses of orbital variation at relatively high frequency (high compared to, e.g., Milankovitch periods [Milankovich, 1941]) have used annual average orbital parameters [Borisenkov et al., 1985; Loutre et al., 1992], precluding them from examining periods shorter than 2 years and aliasing some of the periods examined here. Also, those works were examining the earth’s tilt, rather than earth-sun distance. Gravitational torques have been examined previously as the main driver of the Chandler Wobble and rejected [Munk and MacDonald , 1960; Lambeck , 1980], which means only non-gravitational external forces, such as earth-sun distance, force Chandler Wobble at these periods, if any external sources do. 

Read more »

Playing With Numbers: Triangles and Squares

You can play with numbers; which will be a surprise to some and extremely obvious to others.  I'm writing for those who will be surprised.  Consider the picture of dots here:
* *
 *

We've got a triangle, a small one.  It has 3 dots.  Now put another row of dots, keeping it a triangle:
* * *
 * *
  *

There are 3 dots in the first triangle, 6 in the second.  Next triangle will have 10 (as we add in a row of 4). 

For gaming: What is the 20th triangle number?  Is there a way you can look at a number and tell whether it is triangular?

Or you can play with squares:
*

* *
* *


* * *
* * *
* * *

So the first three square numbers are 1, 4, 9.  Next, the 4th square number, will be 16.  These are actually simpler to game than the triangular numbers.  What's the 20th square number?

And of course we can make more interesting figures, like hexagons:
 * *
* * *
 * *

So the first hexagonal number is 7.  What's the second?  Can you predict the 3rd, the 20th?

On the one hand, we're just playing some games here.  On the other, there are also serious mathematical papers on hexagonal numbers, and triangular, octagonal, and so forth.

Forecast Evaluation

Boy, blow one historic blizzard forecast and people get all cranky*.  Except, as H. Michael Mogil discusses, it was an almost perfect forecast.  For the specifics of that storm and its forecast, I refer you to Mogil's article.

I'm going to take up the more narrow topic of forecast evaluation.  (Disclosure: I do work for NOAA/NWS, but, as always, this blog presents my thoughts alone.  Not least here, because I agree more with Mogil than the head of the NWS, Louis Uccellinni, about this forecast.)  One school of forecast (or model) evaluation looks at computing large scale statistics.  The most famous one for global atmospheric models is the 5 day, 500 millibar (halfway up the atmosphere), wave number 1-20 (large scale patterns), anomaly correlation.  When people refer to the ECMWF model (or 'Euro') being better than the NWS's model (GFS), this is usually the number that is being compared.  But I don't live halfway up the atmosphere, nor do most of you.  We're somewhere near the bottom of the atmosphere.  And there is much more of interest than just average temperature through a layer of the atmosphere.  So there are many other scores (dozens of them) -- See http://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/ for some examples and discussion of what the scores mean.

Most of those scores, though, don't get to my personal -- weather forecast consumer -- interest.  Namely, I'm trying to make a decision of some kind.  NYC, which heard a forecast of 24" (60 cm) but got 9" (22 cm), presumably made decisions that they wouldn't have if they'd heard the perfect forecast that hindsight now provides.  It's here, I think, that we get to the meat of forecast evaluation.  Had this same error been made over the ocean, rather than over the most populated city in the US, with the rest being as it happened, the NWS would be getting praised for their great forecast.  The important part was not difference between reality and forecast, but number of people who made the wrong (in hindsight) decisions.

So let's explore evaluating forecasts by way of our decisions.  I don't make decisions for major metropolitan areas, and not about street plowing and so forth, so will leave that aside.  One realm of weather-affected decisions is in my running.  Let's ignore summer decisions (I'd as soon avoid thinking about what summers are like here) and go with the path as temperatures drop.  Normal gear -- in pleasant weather conditions, is t-shirt and shorts.  Once it cools below 60 F (16 C), I pull on a pair of gloves for my run.

Read more »

Interesting numbers

There's a sort-of theorem in mathematics that all numbers are interesting.*  But I'm thinking first of 1729, which is the subject of a story about how it is an interesting number.  The story is that Ramanujan, a brilliant, self-taught mathematician, was in the hospital (he died at only 32).  G. H. Hardy visited him and commented that his taxi cab was number 1729, which wasn't a very interesting number.  Ramanujan replied that it was indeed interesting -- it was the smallest number that was the sum of two cubes, in two different ways.  That is, 10^3 + 9^3 = 12^3 + 1^3 = 1729.  This number appeared also on a cab in an episode of the Simpsons.

There's a different bit of playing with numbers, one of the longest-unproved theorems in mathematical history.  That is Fermat's Last Theorem.  (Itself misnamed, as he never showed a proof for it, and he worked for years after stating it.)  That is, if we use only integers (1,2,3,...), the equation x^n + y^n = z^n has no solutions for n > 2.  He said this in 1637, and it wasn't proven until 1995.

Let's look specifically at n = 3.  I can rewrite Ramanujan's example as:
x^3 + y^3 = z^3 + 1  (where he had x,y,z = 10, 9, 12)
Fermat's equation is:
x^3 + y^3 = z^3   -- and this has no solutions for integers.

That's interesting -- such a small change, and we go from having no solutions, no matter how large we make x,y,z, to having ... how many?  Well, that's a question.  My version here is a more specialized version of so-called 'taxicab numbers' (named in honor of the above story).  You can see some more about them at Durango Bill's.  But I like mine better because of the connection to Fermat's Last Theorem. 

It seems common that answers in mathematics are either 0, 1, or infinity.  Fermat's equation (for n > 2) has 0 solutions.  We already have 1 for my 'Fermat-Ramanujan' equation.  If there's another, not that this is a proof, probably there are an infinity.  So I set my computer to some brute-force searching, and indeed there are more.  Not many.  It found 92 for z going from 12 (the smallest that has a solution) to 2,000,000 (which was pushing the limit of the computer; z^3 at that point is 8,000,000,000,000,000,000).  That suggests that there are an infinity of solutions to the Fermat-Ramanujan equation (a name I just invented, as far as I know), by that rule of thumb.

Challenges:
Can you find some? 
Can you do it by a more elegant method than having a computer pound away?
Can you prove that there _are_ (or are _not_) an infinity of solutions?

Read more »

Saturn's Hurricane

Many articles and blog posts about Saturn's north polar hurricane.  I'll point you to the NASA press release for some viewing and discussion of the Saturn side of things.

No complete answer here, but I'll raise the flag that one of the movies and set of papers I watched and read in graduate school was under the title "The Range and Unity of Planetary Circulations".  Additional flag: my bachelor's degree was in applied mathematics.  We argued that if you understood the mathematics behind things, you could rapidly move from one area where a certain math applied to any other area where it applied.  Geophysical Fluid Dynamics is one such.  Notwithstanding the name, it applies to any area where you have a fluid on a rotating body -- whether it's Saturn, the earth, the Sun, or Venus.

Coincidentally, I've been playing with a fluid dynamic model, nominally of the earth, but it could be Saturn just as well, and have a movie which I'm still trying to figure out how to share.  This movie has some characteristics which look a lot like the press release.

Century storms

"That's two straight years we've had a 'storm of the century'; those weather guys are idiots!"  Not long after I moved to the Washington, DC area, this happened, and the quote is real.  I'm not sure that the storms involved really were 'storm of the century' events -- events that if we had a long enough record, we'd see happen about 10 times per 1000 years -- but it's something to think of a little quantitatively, particularly in light of my normally abnormal note.

Let's suppose that we're building a house and would like it to last 30 years.  Well, to be specific, let's say we'd like a 99% chance of it lasting that long.  Obviously it has to be able to survive events that we'd expect to happen once per year.  And we can probably ignore things that we'd expect only once in a million years.  But what about a once in 100 year event?  The name misleads us in to thinking that the next time such an event would happen is 100 years after the last time.  While natural reading, it's wrong mathematics.  We could easily be in the unlucky 30 years that sees a 100 year event.  We could even see it twice.  But is there less than a 1% chance of having one 100 year event in a span of 30 years?  That's our design requirement.  If it can be expected more often than that, our house design is not reliable enough.  We need something better. And we'll need to get quantitative.

Read more »

Parts per million

One of the sillier arguments against climate change, especially human-affected climate change, is to claim that since CO2 is a trace gas, it can't have any significant effect.  It's now about 395 parts per million in the atmosphere.

It happens that I'm taking some medicine at the moment.  100 mg of one, and 10 mg of another.  To make the math easy, let's say I weigh 100 kg.  mg is milligram, so there are 1000 of them per gram.  1000 grams is 1 kg.

That first medication represents 1 part per million.  The second is a mere 0.1 parts per million.  The body is a complex system.  Small amounts of things can be very important.  Climate is also a complex system.

Let's continue a little in this vein.  Daily nutrition requirements, which I'll take from http://en.wikipedia.org/wiki/Reference_Daily_Intake , include:

• 1000 mg Calcium --> 10 ppm
• 1000 mg Phosphorous --> 10 ppm
• 60 mg Vitamin C --> 0.6 ppm
• 15 mg Zinc --> 0.15 ppm
• 2 mg Manganese --> 0.02 ppm
• 80 micrograms Vitamin K --> 0.008 ppm
• 6 micrograms Vitamin B-12 --> 0.0006 ppm

If you'd like to avoid scurvy, rickets, pellagra, and a host of other illnesses, you have to treat concentrations far, far lower than 400 ppm as being important.

Is climate a random walk?

Let's pick up again the discussion between Tamino and me.  He has objected to my use of cumulative sums on the grounds that cumulative sums of random numbers have bad behavior statistically.  He's correct about that statistical point, naturally, which means caution is needed regarding the statistical part of my post on finding a climate normal.

But how concerned should we be as climate scientists?  Crucial to that concern is that climate be, to a fair degree, experiencing random variation.  It isn't, strictly.  As Tamino mentioned, there is certainly a trend in more recent years -- not purely random variation.  His criticism is more one against the cumulative sums method.  As Jim Bouldin mentioned in the comments recently, we do routinely transform variables in order to study topics of scientific interest.  In his case, plants.

I'll start by showing you an illustration of why Tamino is concerned.  This is a plot of the cumulative sum -- of a purely random variable, with uniformly random numbers in the range plus or minus 1 degree.  This is actually about 10 times too large for climate.  So the late period value of 40 really means 4 degrees.

Each tick mark is 1 month, but assumes that each month's random number is completely independent of each other.  That isn't the case, as Tamino has documented.  This curve manages to accumulate its 4 degrees in about 200 years.

Er, it accumulates that climate wandering in only 200 years!.  That and the smooth curves, suggest why we cannot take climate to be random wandering over long periods.  Volcanoes suggest why we cannot do so for short periods either.  Intermediate periods might be ok physically.

Read more »

How not to compute trends

Did you know that scientists are lying about the trend in sea ice extent?  That's the conclusion if you apply the popular trend analysis technique those who claim that the earth has cooled, or 'not warmed', since 1998, or 2005.  The probable reason you don't hear about this is that the 'lie' would be that scientists are grossly underestimating how drastic the trend to less ice is if you believe that method.

The method used to claim that there's a cooling trend, or no warming trend, is to cherry-pick a too-recent start year that is exceptionally high and compute the difference between that and a particular recent year (any one, repeat the 'no cooling trend for the last decade' for years afterwards, even if more recent years are warmer).

So I'll take a recent year that had large sea ice extent -- 1996, and compute the trend between there and a recent year that had a low extent -- 2007.  Here's the straight line computed that way, plotted against the observations between 1996 and present.  These data are September ice extents from the NSIDC:

And from eye-inspection of it, it even looks like the average error is about 0.  Sometimes high, sometimes low.  This trend is for ice pack extent to lose about 330,000 km^2 per year, against the about 78,000 km^2 that is computed for a linear trend by climate scientists.  Clearly climate scientists are trying to hide the decline!

I've already done some things more honest than what the 'no warming since 1998' folks do, not least is, I showed you the trend line and the data.  But this is far from sufficient to have a reasonable trend analysis.

Read more »

Reconsidering forecasts and wagers

Comments in two different threads suggest that there's some good room for more discussion.  Yesterday, I noted that if someone would only take a bet at 50 to 1 odds, he wasn't very confident about his side of the bet.  Certainly not the 'just as likely to warm as cool' that was the statement which prompted the bet.  (Nobody here, by the way.)  As M commented, and I assume that he mean in terms of real money, he'd want 5:1 odds even on an even money situation.  I wouldn't mind that myself (again, I don't bet real money, but in examining the mathematics of expectation, that's how it goes).

For illustration of a concept that was in my mind, but not in the prior post, I'll pick up with Alastair's comments about his predictions and preferences for odds.  The thing is, the odds that you're willing to accept also describe what you think is really the case.  At least it's much closer than in the case that your lunch money is riding on the bet and you're already hungry.  Our situation here is betting something that doesn't exist (quatloos) and presuming that we can make a lot of bets, so that we can average out the wins and losses over time -- converging towards the mathematical situation.

I make use of this in group settings when lunch place selection is being discussed.  In a group of, say, 6, all will claim to have no preference between places A and B.  I pull out a coin and say, fine, heads it's A, tails it's B.  An amazing number of people suddenly develop a preference for one or the other.

So it goes with Alastair's estimate of 3.9 million km^2 against mine of 4.4.  His original estimate for uncertainty was 0.1 million km^2, while mine was 0.5 million.  I'll add a curve for him, where the uncertainty is increased to match mine; that'll be the 'Alastair-2' curve:

Read more »

Whiteboard on the end of global warming

The Whiteboard also has a series (now looks to be finished) on whether global warming has 'stopped'.  That's what prompted the series I linked to last week.  The question is pursued statistically, following the lead of Tamino at Open Mind.  The brief summary of the series is 'no'.  But it is well worth looking in to the series for the how and why we can say this, and how strongly we can say it.

Did Global Warming Stop After 1998?
Did Global Warming Stop After 2000?
Did Global Warming Stop in 1940?
Did Global Warming Stop After 2007?

For looking at the converse, global cooling:
Did Global Cooling Stop in 1970?

These are more mathematical takes than my old What cooling trend?  They come to the same conclusion, so those who'd like more math behind their conclusions can get it.  Since I did that post over a year ago, it's probably time for an update.  One of these days ...

Wrestling with data

I'll suggest those who haven't been, join me in keeping an eye on a series of posts that Ron Broberg is doing over at The Whiteboard.  As befits a whiteboard, he's showing a lot of the details that get cleaned out of most final publications, even on blogs.  The topic at hand is looking at the temperature records since 1880 and testing ideas on fitting curves to the data.  The series is now to #9 and it's apparent that there will be several more:


http://rhinohide.wordpress.com/2011/01/07/lines-sines-and-curve-fitting-1-oh-my/ (Starts out more on the issue of testing ideas on what we can conclude about temperature trends)
http://rhinohide.wordpress.com/2011/01/08/lines-sines-and-curve-fitting-2-r/ (try fitting the sine and then a line)
http://rhinohide.wordpress.com/2011/01/09/lines-sines-and-curve-fitting-3-double-down/ (Try fitting 2 sine waves)
http://rhinohide.wordpress.com/2011/01/10/lines-sines-and-curve-fittings-4-walk-and-chew-gum/ (Simultaneous line and sine fit.)
http://rhinohide.wordpress.com/2011/01/12/lines-sines-and-curve-fitting-5-a-growth/ (Trying an exponential curve)

http://rhinohide.wordpress.com/2011/01/14/lines-sines-and-curve-fitting-6-backcast-and-forecast/
http://rhinohide.wordpress.com/2011/01/15/lines-sines-and-curve-fitting-7-normal/  (Testing Normality 1)
http://rhinohide.wordpress.com/2011/01/16/lines-sines-and-curve-fitting-8-dagostino/ (Testing Normality 2)
http://rhinohide.wordpress.com/2011/01/17/lines-sines-and-curve-fitting-9-girma/

A sine is a standard oscillation.  It would be a pure tone (rather flute-like) in music.  For a bit more about oscillations and data series, and the language of time series analysis, take a look at my Introduction to Time Series Analysis.

Do I have to be good in math to be good at science?

The short answer: No.  Before the longer answer, here's the full comment/question from the teacher:

Question:  Do I have to be good in math to be good at science?

The reason I ask this is that many scientifically inclined students I know are not going to pursue science as they get older because they perceive themselves to be less than stellar in math. This is a shame. And a waste.  Please stress to the young, hormonally-infused people who read your blog (not the adults-one hopes they've figured it out already) that science is a process, just like running, and eating and IMing on the phone. And math is a tool -- a really great, useful tool -- that's part of the process.

Splitting hairs? I don't think so. I've got a few 5th graders who I've shared your blog with, and they were enjoying themselves until they hit the math and freaked out. Nooooooo, I say, fear not the many zeros and exponents. It all makes sense as you practice it (even I can say that).

So ... address my question. My 5th graders will thank you. And, of course, so will I!

I whole-heartedly second everything the teacher said.  The 'enjoying themselves until they hit the math and freaked out' is also not limited to the 5th graders. I've heard from some distinctly older folks about this too.  It's why I'll be making some changes to my posting practices.

For the students, there are two different questions to think about.  One is, what does it mean to be 'good at math', and the other is 'what am I doing when I'm doing science?'.  I'll share some of my thoughts and invite students and teachers to share theirs as well.  Questions, as always, welcome.

Read more »

CO2 and temperature for 800,000 years

First, the figure gives the answer on CO2 and temperature over the last 800,000 years:

Here we have each value of CO2 plotted against the temperature deviation from reference values -- with the temperature being for 1000 years before the corresponding CO2 value. The correlation for this is about 0.89 (R), meaning that you could explain 79 percent (0.89*0.89) of the variation (R^2) in CO2 by looking at the temperature. As is mentioned in my post Does CO2 correlate with temperatures, where we saw equally high correlation between temperature and CO2 (even higher if you give CO2 a 20-30 year lead on temperature), and quite a few times in the comments, correlation is not causation. Could be that both temperature and CO2 are pushed around by something else. More about that in a moment.

It's common to see the claim that temperature 'leads' CO2. Loosely speaking, this means that temperatures generally change before CO2 does, and that there is a consistent pattern to the connection -- if temperature rises, CO2 does as well. This is true; it leads by about 800 years [Caillon and others, 2003]. The amount of lead also seems to depend on whether you're in a glacial period or (as we are now) an interglacial. The correlation between temperature and CO2 is lower (by a very small amount) if you take their values for the same time (drops to 0.88). And it drops by more if we take the temperatures for 2000 years before the CO2 value, to 0.87, indicating from this very simple approach that the lead is between 0 and 1000 years, probably closer to 1000 -- as is found from the more serious approach in the reference.

So there's some interesting science to do, to understand why that lead exists, why it is many hundred years (rather than a few dozen years, or a few thousand), and why the lead time depends on whether you're in a glacial period or an interglacial period. Oddly, to me, most of the time that people mention this lead relationship, they are not referring to any of this.

Read more »

Assessing predictions

It's a little premature to make a detailed assessment of the predictions for September's average extent as the final numbers aren't in. They will be soon, but my focus is actually over on the question of how to go about doing the comparisons. Earlier, I talked about testing ideas, but there, the concern was more one of how to find something that you could meaningfully test. Here, with the September's average extent, we already have a well-defined, meaningful thing to look at.

Our concern now is to decide how to compare the observed September average extent with the climatological extent, and a prediction. While mine wasn't the best guess in the June summary at ARCUS, it was mine, so I know what the author had in mind.

Let's say that the true number will be 5.25 million km^2. My prediction was 4.92. The useless approach is to look at the two figures, see that they're different, and declare that my prediction was worthless. Now it might be, but you don't know that from just the fact that the prediction and the observation were different. Another part of my prediction was to note that the standard deviation to the prediction was 0.47 million km^2. That is a measure of the 'weather' involved in sea ice extents -- the September average extent has that much variation just because weather happens. Consequently, even if I were absolutely correct -- about the mean (most likely value) and the standard deviation, I'd expect my prediction to be 'wrong' most of the time. 'Wrong' in that useless sense that the observation differed by some observable amount from my prediction. The more useful approach is to allow for the fact that the predicted value really represents a distribution of possibilities -- while 4.92 is the most likely value from my prediction, 5.25 is still quite possible.

We also like to have a 'null forecaster' to compare with. The 'null forecaster' is a particularly simple forecaster, one with no brains to speak of, and very little memory. You always want your prediction to do better than the null forecaster. Otherwise, people could do as well or better with far less effort than you're putting in. The first 'null forecaster' we reach to is climatology -- predict that things will be the way they 'usually' are. Lately, for sea ice, we've been seeing figures which are wildly different from any earlier observations, so we have to do more to decide what we mean by 'climatology' for sea ice. I noticed that the 50's, 60's, and 70's up to the start of the satellite era had as much or somewhat more ice than the early part of the satellite era (see Chapman and Walsh's data set at the NSIDC). My 'climatological' value for the purpose of making my prediction was 7.38 million km^2, the average of about the first 15 years of the satellite era. A 30 year average including the last 15 years of the pre-satellite era would be about that or a little higher. Again, that figure is part of a distribution, since even before the recent trend, there were years with more or less (than climatology) ice covers.

It may be a surprise, but we also should consider the natural variability in looking at the observed value for the month. Since we're really looking towards climate, we have in mind that if the weather this summer were warmer, there'd be less September ice. And if it were colder, or different wind patterns, there would have been more ice this September. Again, the spread is the 0.47 (at least that's my estimate for the figure).

I'll make the assumption (because otherwise we don't know what to do) that the ranges form a nice bell curve, also known as 'normal distribution', also known as 'Gaussian distribution'. We can then plot each distribution -- from the observed, the prediction, and what climatology might say. They're in the figure:



This is one that makes a lot of sense immediately from the graphic. The Observed and Prediction curves overlap each other substantially, while the curves for Observed and Climatology are so far from each other that there's only the tiniest overlap (near 6.4). That tiny overlap occurs for an area where the curves are extremely low -- meaning that neither the observation nor the climatology is likely to produce a value near 6.4, and it gets worse if (as happened) what you saw was 5.25.

The comparison of predictions gets harder if the predictions have different standard deviations. I could, for instance, have decided that although the natural variability was .47, I was not confident about my prediction method, so taken twice as large a variability (for my prediction -- the natural variability for the observation and for the climatology is what it is and not subject to change by me). Obviously, that prediction would be worse than the one I made. Or at least it would be given the observed amount. If we'd really seen 4.25 instead of 5.25, I would have been better off with a less narrow prediction -- the curve would be flatter, but lower. I'll leave that more complicated situation for a later note.

For now, though, we can look at the people who said that the sea ice pack had 'recovered' (which would mean 'got back to climatology') and see that they were horribly wrong. Far more so than any of the serious predictions in the sea ice outlook (June report, I confess I haven't read all of the later reports). The 'sea ice has recovered' folks are as wrong as a prediction of 3.1 million km^2 would have been. Lowest June prediction by far was a 3.2, but the authors noted that it was an 'aggressive' prediction -- they'd skewed everything towards making the model come up with a low number. Their 'moderate' prediction was for a little over 4.7. Shift my yellow triangle curve 0.2 to the left and you have what theirs looks like -- still pretty close.

To go back to my prediction, it was far better than the null forecaster (climatology), so not 'worthless'. Or at least not by that measure. If the variability were small, however, then the curves would have narrow spikes. If the variability were 0.047, ten times smaller than it is, the curves would be near zero once you were more than a couple tenths away from the prediction. Then the distribution for my prediction would show almost no overlap with the observation and its distribution. That would be, if not worthless (at least it was closer than climatology), at least hard to consider having done much good.

Introductory Time Series Analysis

As often happens, this note is prompted by someone doing things that look wrong. Time series analysis is something which has been an interest of mine since I earned my Master's (looking at tidal signals in current meter data). And there are some elements which I think are eminently understandable without diving in to the gory details. So here is my shot at a very short, only marginally mathematical, introduction to time series analysis.

A time series is just a series of observations (of something, anything) through time. The monthly averaged global mean temperatures that I use for demonstrating principles of climate are time series. Monthly Southern Oscillation Index is another time series. Not coincidentally, those are the ones examined in the paper, and are what I'll consider here. But there are innumerable other time series -- daily close of the stock exchange, daily temperature, your weight day by day, and so on.

Much of the language for talking about time series is fairly ordinary. But there are a few terms I'd like to be sure we're using the same. Most important is 'period'. The period of something in your time series is the length of time from peak to peak (or trough to trough). Consider hourly temperatures in your area. They peak each day, around, say 3 PM. The period is then 24 hours. If we take a longer view, and consider daily high temperature, then the temperatures peak each year -- a period of 1 year. If we think of the brightness of the moon, it has a period (full moon to full moon) of 29.53 days. And so on. We could also ask how often the peak occurred per year (or other time of interest). This is the frequency -- how frequently do you hit the peaks. For hourly temperatures, the frequency is 365 per year (or 365 cycles per year). The frequency of the lunar cycle is 12.369 cycles per year. And the seasonal cycle has a frequency of 1 cycle per year.

In talking about periods and frequencies, and scientists tend to use the two terms interchangeably since the value of one can always be converted to a value of the other, we sometimes also hear about long/short periods, or high/low frequency. In the periods, it means what ordinary english would lead you to think -- long periods are periods that take a long time from peak to peak, and short periods are fast from peak to peak. You still have to know what is 'long' for the system to read any given paper correctly. If it's a geologist, they could mean 400 million years when they say 'long period' (the period for continental drift cycling), where a meteorologist might mean 40 years. High frequencies correspond to short periods (since the period is short, the thing happens often -- at high frequency). Low frequencies have long periods. The similarity of term with music is no accident. Low frequency sound (low pitch) has a long period, while high frequency sound has a short period.

So when you hit anything relating to time series, if you know about music, think in terms of frequencies and pitches. Another term that comes up in time series analysis, with a slightly different meaning than in music, is 'harmonic'. Unlike music, with its fifths, thirds, etc., in time series we work more simply. There is the base period/frequency. And then there are the integral multiples of that base frequency. The annual cycle's harmonics are 1 (the base period), 2 (6 month period, 2 cycles per year), 3 (4 month period, 3 cycles per year), 4 (you get the idea), and higher. In practice much of our weather time series can be captured by the first 4 harmonics of the annual cycle. (That's interesting in its own right -- it means that there is relatively little happening at periods of 5, 7, 8, 9 months, even though there's a lot at 4, 6, 12 months.)

As with music, we also are interested in how loud the frequency is. Our measure there is called amplitude. It is half the distance from peak to trough. Where I live, our peak summer high temperatures are about 90 F (32 C), and in the winter, our lowest highs are about ... call it -10 C (14 F). The range is 42 C, so the amplitude of our seasonal cycle is 21 C.

Again following music, there's usually more than one frequency being played at a time. This is certainly true for weather! Many different things happening all the time. In music, the description of all the notes the band or orchestra are playing at a time is called the score. For time series analysis, it is the spectrum. A little more involved in time series because we can look at different spectra (1 spectrum, 2 or more spectra) -- the amplitude spectrum, and the 'power' spectrum. Most work is actually done with the power spectrum, but the amplitude spectrum is easier to understand so I'll stay with that.

One of the things we do in looking at climate time series is average the data -- construct a moving average, for instance. The moving average says to take the first (some number, let's say 12) months of data and average them together. Then step forward (move) 1 month, and average the next 12. Repeat until you're at the last 12 months of data. As I've suggested for understanding global climate, you want quite a bit more than 12 months of data in your averaging. But we can also try to understand weather. A 12 month average will clobber most of what is happening shorter than 12 month periods (but not absolutely all of it, a point even scientists seem to forget -- it only completely clobbers the 12 month period and its harmonics), and let us look at what is happening on periods longer than 12 months (but some of that, too, gets damped). In musical terms, averaging suppresses the high notes, while leaving the bass line relatively unaffected.

Suppose what we really want is to suppress the bass line and enhance the treble -- suppress the climate frequencies and focus on the weather. Rather than averaging, which is a smoothing operation that suppresses the high frequencies, we would take differences. Say take the difference between months 12 months apart. We can think of the temperature as being a certain amount of weather, plus a certain amount of climate. The climate will be nearly the same 12 months apart, so when we do the subtraction, it is cancelled out and we have only the difference in weather those two months. Differencing is a sharpening operation that suppresses the bass line and enhances the treble. It is, however, an extremely biased operation -- not only does it suppress some and enhance others, but the degree of enhancement is proportional to the frequency. Unlike the averaging operation, which leaves low enough frequencies unchanged, the differencing affects all frequencies and does so strongly.

Changing time series data is called a filtering. The averaging and differencing operations are different filters. There are many, many more that we could use. Any time we do use a filter, though, we should be careful that it isn't creating problems for us. This is one reason I try always to work with the most nearly original data possible -- no filtering has been done that could obscure the effects I'm trying to work with.

If you're a more visual person than a music person, the spectrum (guess where we stole that word from!) also has some color traits. If the amplitudes are all about the same, regardless of the frequency, then we call it a 'white' spectrum. (White light is approximately equal contributions of all colors.) Instead of period, for light we think of wavelength. Frequency is still frequency, but now it means how many times per meter you see the peaks. High frequency light is blue. Low frequency light is red. When we do the moving average, we are making the spectrum redder. When we do a differencing, we are making the spectrum bluer. Most climate-related time series have red spectra -- there are higher amplitudes at longer periods (wavelengths). Our year to year variations are on the order of 0.1 C, but the ice age cycles are 5 C, for instance.

Statiscal significance of 150 years of data over 4.5 billion years

The subject line is close to a recent search query that lead someone here, and echoes a comment that's not unusual in blog comment sections. The thing about it is, it's not a very strong question.

One part of the question's failure is that it isn't really a proper statistical question. Given limits of search strings, that's no surprise. But it does show up in comments (usually in the vein of assertion "150 years is too small a statistical sample of 4.5 billion years of climate.") where such limits don't apply.

The basis of any good question is to try to understand something. If what you are trying to understand is not statistical, then pursuing statistics is not going to help you. "What is your height?" is marginally statistical. If you measure the length of a short metal bar many times, which I did in freshman physics lab, you'll get slightly differing answers. So you may well answer statistically with your mean and a sample standard deviation. This is only marginally statistical, in that it is purely descriptive statistics and no hypothesis testing is involved.

Your question, however, could be non-statistical: Did it rain in my back yard last night? How much rain did my rain gauge capture? If so, you will be wasting your time if you chase after statistical tables. More at hand, someone raising statistics in a blog debate about a non-statistical question is wasting your time.

But suppose that what you're trying to understand truly does require statistical considerations beyond minor description. What would such a question look like? One could be "Given the population of voters in the USA, how many randomly selected voters would I need to ask a yes/no question in order to have a standard error in my sample of 3% or less as compared to asking the everybody?" or, for more climate-related flavor "How many satellite observations with a (known) standard error of measurement would I need to find an average sea surface temperature whose standard error would be less than 0.1 K?"

Required is a description of the 'population' -- all possible observations (the population of voters in the USA, satellite observations) -- of your 'sample' (ask some number, which we want to determine, of voters, or satellite observations -- and what statistic you are trying to estimate (% of voters who like your candidate, mean sea surface temperature).

So, back to the original question. Does it describe the population? No. 'data' certainly doesn't limit us to anything in particular. I'll guess that it is global mean temperature, of the earth (maybe it's Mars -- I've read interesting papers about Martian climatology), that is the point of interest. But we shouldn't have to guess, a good question is clear on what it is asking about. Does it describe a sampling method? Not really. 150 years, well, I'll guess that this means 'take the most recent 150 years'. Most importantly, however, does it describe what statistic one is trying to estimate? No. Again, I'll guess: that it is "Global mean temperature over the entire history of the earth."

Putting it all together, the statistical question more reasonably posed is something like "Does the last 150 years of global mean air temperatures provide a good estimate of the global mean air temperature through the history of the earth?" (better would be to define 'good', say 'standard error within 0.2 K')

That's the statistical side. But since we're not interested solely in statistical questions regarding climate, at least not most of us, we also have to ask, "Is this a physically meaningful question?" The person who did the search, I don't know what they have in mind. They could indeed have in mind a question for which the mean temperature of the planet throughout its history is exactly the right number to answer.

Usually, though, comments about the 150 years vs. 4.5 billion surface in debates about modern climate and modern climate change. I'll have to invite contributions on what relevance the planetary mean temperature from 4.5 billion years ago to ... oh, let's say 30 million years ago ... has to questions of current climate and climate change. Certainly one wants to know how climate has changed through all of time, and why. I'm one such person. But at hand is those who argue that the global mean over that entire period matters.

What's important regarding human responses to climate change is the climate on human time scales. The last 150 years handily covers me back through my great-grandparents. The next 150 years will cover my children, grandchildren, and possibly great-grandchildren. Even a 'mere' million years ago, much less 4.5 billion, there weren't any humans around to think about climate change at all. So the 4.5 billion years is, for 'what do we do now' questions, a spectacularly large red herring.

Human society infrastructure also dictates a much shorter term (than 4.5 billion years) concern. Almost every mile of paved road in the world is less than 150 years old. Almost every mile of railroad. Almost all port structures. Absolutely all air transport terminals are less than 150 years old. Absolutely all electrical distribution structure, all phone lines, and even more so all cell phone towers are less than 150 years old. Coal, oil, natural gas distribution networks, again, are almost entirely or entirely less than 150 years old. Many of the world's major cities are less than 150 years old (I'll count Chicago here, as the great fire in 1871 erased so much of the city).

The climate-related concern here is that all these things were constructed based on what climate was like around the time of their construction. That climate drove what the standards would be for, say, tolerance to flooding, tolerance to drought, high winds, high rain rates, high and low temperatures, and so on. If climate changes outside the range that the infrastructure was built for -- almost all of it globally being much less than 150 years old -- then there is a serious risk that the structure will fail when it encounters brand new climate conditions.

In this vein, then, comments about the 'we did fine in the medieval warm period' are a different flavor of red herring. Chicago, Sao Paulo, Melbourne, Johannesburg, ... didn't exist back then. They did not 'do fine' in the medieval warm period, they never encountered it. Even cities that did, such as London, Rome, Xi'an, did so with far smaller populations than today's. London, ca. 1100, had a population around 18,000, and is about 1000 times larger today (taking the metro area). Rome was about 20,000 around 1100 AD, vs. 200 times as large now. Xi'an was among the largest of cities in the world then, and may have been about 500,000 around 1100 AD. But almost 20 times that today. (mostly Wikipedia figures). While a Rome or London of 10-20,000 survived a medieval warm period ok, Rome and London of many millions have never see it before.

As usual, we can get far by asking two questions: "Is this question statistically meaningful?" "Is the question physically meaningful?" After looking at the former, regarding the original search string, and not infrequent comment, we see that it isn't a meaningful statistical question. After rephrasing it to something that is statistically good, we check to see whether it makes physical sense. Turns out, for the sort of thing I'm concerned about and others use it, that even the rephrased question isn't physically meaningful. Knowing that global mean temperature for the last 4.5 billion years was 20 C (I make up a number) as opposed to 15 C still doesn't tell us whether major modern cities, and their millions of residents, will be able to manage likely climate changes. Nor does it tell us what adaptations to take, much less how expensive it would be to make those adaptations. And it doesn't tell us what the costs, both dollars and lives, would be if there were no adaptation.

Binomial probabilities

This also goes under the name of 'Bernoulli trials'. The idea is that you have a circumstance in which one of two things will happen. And you're going to try out the process many times. This could be tossing coins, or dice, or a batter going to the plate (or wicket) many time, a basketball or hockey player taking a number of shots, and so on. After a bunch of trials, you then ask what the chances are that you got that many heads (or sixes, hits, baskets, ...) or more. Even better is when you try to figure out the chances before you start the trials.

But let's be concrete. I find it easier to think of a particular example first, and then think about more general cases. (Some people prefer the other way around; people are different. But they're not writing this blog :-) Consider tossing a coin 5 times. What are the chances that we get 3 heads?

There's only 1 way to get 5 heads -- get a head on the first throw, and the second, and the third, and the fourth, and the fifth. 'And' is an important word in probability, meaning that we should multiply the probability of the individual events involved. Since we're using a fair coin with a 1/2 chance of turning up heads, this means 1/2 * 1/2 * 1/2 * 1/2 * 1/2. So, 1/32 chance of turning up 5 heads in 5 throws.

But let's think about getting 4 heads in 5 throws. We have 5 different ways of getting that. I'll list them off with 'H' meaning a heads, and 'T' meaning tails. They are:
THHHH or HTHHH or HHTHH or HHHTH or HHHHT.
Each one of these has a 1/32 chance of happening. The other important probability word is present. 'or' means to add the chances. So there is a 5/32 chance of getting 4 heads in 5 throws of a fair coin. (Or for your team to win 4 games out of 5 between evenly matched teams, or for a player to hit 4 shots out of 5 when he has a 50% shooting percentage), and so on.

It gets more complicated with 3 heads, 2 tails. And a lot more complicated when it's 493 tosses of the coin. That's where we want the general formula to do the legwork for us. When we get to cases where one event is more than 50% likely, again, it becomes nice to have a more general formula. I've made up a little spread sheet in Open Document format, if there's interest.

For the case at hand, about 'Grumbine scientists', we're wondering what the chances are that there'd be 5 or more scientists in a group of 493 people. By the way, speaking of family odds, the Bernoullis had an extraordinary run of mathematicians (hence the name for this bit of probability) and mathematical physicists ('Bernoulli effect' in fluid dynamics). In the case of 'scientist', the coin is weighted heavily against coming up that way. I made up a probability of 99.9% that a given person (in the US) was not a scientist who had published in the scientific literature in the last 20 years. Don't know that this is correct, which we'll return to. Nor, as we've already discussed and had comments on, are we confident that the number 493 is right or even very close.

With only 1 person in 1000 qualifying, we wouldn't be surprised to see 0 of 493 turn out to be scientists. The actual calculation gives 61% of the time that we'd expect 0. 30% of the time, we'd expect only 1. Conversely, the chance of something happening is 100% minus the chance of it not happening. Having 2 or more scientists show up, then, is only about a 9% chance. We shouldn't worship at the altar of the 5% level, but it's a good rule of thumb for getting started. With a 9% chance, we're not very impressed to see 2 or more scientists in this group of 493. But chances of getting 2 scientists are 7.4%. Between that, and some rounding in the earlier figures, there's only a 1.4% chance of finding 3 or more scientists in a group of 493 people. And that beats the 5% requirement handily. It's 0.2% for 4 or more, and 0.016% for 5 or more. This gets to a level where, as a matter of the probability, we'd be pretty confident that something real was going on.

But there's a joker in this deck, and I'm it. There is a selection effect problem. Namely, this group contains me -- because I started looking at the subject on the grounds that I was in it. Any group that contains me is guaranteed to have at least 1 scientist, 1 left-handed person, 1 runner, and so on. If I'm the person selecting the group, then we have to not count me. That leaves us with only 4 identified Grumbine scientists for the purpose of our research. As that's still at the 0.2% level, we're still pretty confident that something real is going on. Or at least if not, it's a pretty surprising coincidence.

Suppose that the number I made up for fraction of people who are scientists is too low. Let's say it's 0.3% instead of 0.1%. Then our chances of getting 4 or more scientists in the sample of 493 rises to 6.3%, which would not pass our standard for 'probably not chance'. So it is important to get a good idea of the figure. Now I'm pretty sure that the true figure isn't as high as that. It would mean 1 million people in the US had published in the last 20 years, and that just doesn't seem plausible. Even 300,000 (the 0.1%) strikes me as high, but I was trying to err on the high side in the first place.

Some odds and ends:

• We had some difficulty in finding data to work with.
  
• Once found, we saw that the data had some serious quality control problems
• Our starting point included a selection bias problem
• In trying to evaluate our conclusion, we discovered that the conclusion depended on an assumption we'd made without much evidence (the 0.1%)

All of this is common. To do science, you have to be ready to go back over your whole process to verify that where each step was not extremely strong, it at least doesn't change your conclusions. If it could, you have to mention this. Something that happens, though, in science is that once it's been mentioned, issues don't necessarily get rehashed in every paper for ever after. Inexperienced readers of science sometimes complain, because of this, about scientists 'hiding' problems. It isn't hidden, it's there in the scientific literature. It's just that the scientific literature assumes you're all big kids and have done your homework in reading the previous work. Citations aren't there for decoration.

Partly, this set of notes is to illustrate the Central Skill of a Scientist note. Although it turned out that the original idea, of there being surprisingly many Grumbines in science, is probably acceptable, it could have turned out otherwise. At that point, move on to other ideas. Scientists have many ideas, which is one of the secondary skills. So moving on to others is not a big deal.

Then again, having passed this far, we are in a position to ask -- again -- "So what?". More politely, "What would be shown even if the idea were true?" If there really were some exceptional number of Grumbines (or Bernoullis or Darwins to name some much better known families) in science, what would that mean? Unfortunately, nothing particular. Maybe it is a sign that there's a genetic contribution to entering science. Maybe it's a sign that there are family environment features which, for some reason, are common in this crowd. And maybe it really is just chance. These are more reasons to not get too wedded to ideas.