29 September 2014

Multiple Working Hypotheses

In exploring Arctic ice minima I was not so much trying to reach conclusions as to find hypotheses for further testing and exploration.  Let's pick up the hypotheses side now, as I think it gets much too little attention in science education and science student practice.  In saying that, I'm projecting my bias, of course.

Part of that bias comes from having read and agreed with T. C. Chamberlin's Method of Multiple Hypotheses (1890).  Or at least liked my take on it.  It also has some correspondence to John Stuart Mill's ideas in On Liberty about a marketplace of ideas (1859), which I also liked.  The crux is, if we consider only one idea/hypothesis we are liable to be overly protective of it, or overly hostile to it.  Either way, we do not arrive at the best hypothesis for continued work.  Chances of us having started by selecting the best of all possible hypotheses, out of the infinity which could be generated, are essentially zero.

So, instead of starting with:
  • Observe
  • Make a hypothesis about those observations
  • Make a prediction from that hypothesis
  • Run an experiment to test the hypothesis
We try something more like:
  • Observe
  • Make multiple hypotheses that explain the observations
  • Examine the hypotheses for how/where/when they lead to different predictions
  • Run an experiment to distinguish between stronger and weaker hypotheses
A different take, or at least a different discussion, of the method of multiple working hypotheses is by L. Bruce Railsback

So let's apply the method to my prior post.  (Certainly there can be better examples, and I'd be delighted to hear from a teacher about their own example!  Please do comment, with links if you have examples)

I come up with a few years which seem to be of extra interest to try to understand:
  • 1996 (particularly high year)
  • 2007, 2012 (particularly low years)
And there are some trend lines or periods which seem to be interesting:
  • 1979-2006
  • 1979-1996
  • 1997-2006
  • 2007-2013
I also suggested that maybe there was more variability in 2007-2013 than earlier, but kind of rejected that on the basis of eyeball evaluation.  Well, eyeballs plus a pretty casual examination of the numbers.  Let's keep that one in mind.

What results is not as neat as we're often told (not a knock on the teachers -- you have to start somewhere, and it's a good idea to start with something simpler rather than something more complex).  We've got some different categories of result, or conclusions, or ... whatever you want to call them:
  • Go back and look at the meteorology and oceanography of 1996, 2007, 2012 to see why (or whether those things contributed) they were unusual years for Arctic sea ice.
  • There are 4 (6, well, many) trend lines to consider -- 
    • how did they each perform in estimating the ice cover for 2014? (we can't control the earth, so we have to wait for nature to tell us what the result of the experiment is)
    • can we reject any of them? (is the observation too far outside what could happen due to weather's usual changing?)
    • Which ones can we reject?  How confidently?
  • Is there any change in variability?  (Has somebody already done research on the topic?)
Plus, from the comments (EvanH:
  • Is the change in area/extent being caused by changes in ice thickness?
This one points to a value of  having more than one person involved in generating the multiple hypotheses.

As we go forward, also keep in mind the comment from MMM
I don't know… 2007-14 is such a short time period, calling it a "new normal" just because there's no trend, rather than assuming it is a continuation of a declining trend where 2007 jumped ahead of the curve… seems unlikely. Not impossible, but I'd want a physical reasoning rather than just eyeballing and random trend fitting…

2007-14 really is a short period.  Remember that I observed we want more like 30 years to find a climate trend.  7 years is pretty short!  So, in pursuing those prior hypotheses, we should keep in mind the concern that there's just not enough data.  Or, at least not enough to make conclusions based _only_ on those data.  Keep looking for physical processes.

This is a different reason to have more than one person involved.  Again, we all tend to be too fond of our own hypotheses.  It is helpful to have someone else in the group to point out the weaknesses, or additional questions that need to be answered, from or for our hypotheses. 


7 comments:

Evan H said...

If I may clarify: The hypothesis I suggested is that the change in area is being driven by warming, but the changes you identified in the rate of change are being driven by thickness.

(More accurately, I think there's a relatively steady or steadily-accelerating decrease in volume which is being driven by warming, and the "jerkiness" of the area measurements is an artifact caused by the use of a single cross-sectional measurement of area as a proxy for volume.)

Robert Grumbine said...

Definitely want clarity.

I'm going to try to connect the volume and area/extent in a way simple enough for discussion here. I might email the raw math to you if I can't find a good, but simpler, view.

Kevin O'Neill said...

Over at Neven's we were trying to parse out whether arctic sea ice losses were mainly due to summer or winter processes. After running some winter > summer, summer > winter correlations I decided years with above average summer losses were driving the process, writing:

"The intuitive answer is that the losses are driven by summer melt processes. The data seems to bear intuition out: The correlations between the preceding winter and summer melt are very low. The correlations between melt and the following freeze are significant (though barely at 95% and with a small sample size).

BUT .... arctic winter warming is 4 times larger than summer warming! Hmmm .... back to the drawing board.
"

I then noticed that the arctic OLR reanalysis data showed something interesting:
"Take a look at the Interpolated surface OLR for February 2013 and compare it to previous Februarys. One has to go back to at least 2004 to find anything similar. Probably 2002 or earlier. It's also pretty easy to show that the February OLR plots are very closely related to the summer losses. Rank them in order just by visual appearance and you'll come out very close to the same order as the actual losses."

The forum thread was back in Jan/Feb so the 2014 OLR data wasn't yet available and I've never gone back to see if it 'predicted' this summer's melt. At the time I intended to download the netcdf reanalysis files and do some actual math with them - but never got around to it :(

Fernando Leanme said...

Maybe you should focus on the Kara Sea? That´s fairly closed off and it has very large ice cover changes. I´d love to have temperature and salinity data for the Kara over the last 10 years. And I bet there´s an extensive data set because ExxonMobil and Rosneft have been drilling between Yamal and Novaya Zemlya. This means they had private data gathering expeditions and arrays. I would focus on that data set if you can get it.

Robert Grumbine said...

Kevin:
Sorry about the delay.

'Back to the drawing board' is one of the most important phrases, especially if you engage in multiple hypotheses!

I hope you do go back to the data and see how 2014 has turned out. For incentive, we can make your consideration a guest post here if you like.

Fernando:
While the Kara Sea is relatively cut off from the ocean and main ocean circulation, it is enormously influenced (for the same reason) by the Yenesei River. And that river is extremely variable.

I'll extend the same offer/incentive to you. How about you pursue some analysis of the Kara Sea's variability and a) show that it is indeed more variable than other sectors (Sea of Okhotsk, Labrador Sea, Barents Sea, for instance) and b) connect that variability to something (river outflow?) else. Write it up and make it a guest post.


Doug Cotton (not posted):
The comment section of my blog is not the place to pursue your argument with Jeff Condon/Jeff Id/Airvent. Doubly so as the subject of your argument has nothing to do with the topic here.

Kevin O'Neill said...

Robert - I went back and revisited the reanalysis - but the data only goes through Dec. 2013.

After a little digging I found this note on the NOAA website:

------------
This is the last planned update of the NOAA Interpolated OLR dataset. Production will resume when we receive funding to do so. Please email us if this impacts your research.

Temporal Coverage:
Monthly values for 1974/06 - 2013/12
Daily values for 1974/06 - 2013/12/31
------------

I'll have to look for an alternative dataset.

Michael 2 said...

"Observe, Make multiple hypotheses that explain the observations"

An excellent approach suitable for inductive reasoning.