Monday, October 26, 2009

Doing science, with sea ice

Every so often, I commit an act of science. Like most acts of science, you almost certainly never heard about it. Like many, however, life was eventually improved for some people somewhere. I'm rather pleased about that side of it.

What was at hand was, on one hand (it does help to have many hands if you're in science), a fairly straightforward piece of engineering. On the other hand, a bit of science. Remember that I think both engineering and science are good things, if different. Engineering is mainly aimed at 'apply what is known to achieve benefit for someone', while science is aimed at 'try to understand more about the universe'.

Back in 1993, I was at the National Meteorological Center (NMC), the part of the National Weather Service (in US -- NOAA) that develops the new weather forecast models or tries to make the old ones better. My area was sea ice. Now, one thing we sea ice, polar oceanography, polar meteorology people were entirely confident about was that sea ice mattered, a lot. For, well, everything, or at least enough. If we didn't think it mattered, we'd hardly be spending our time studying it. People outside our little community, including folks working on numerical weather prediction, didn't think sea ice mattered for much. And, if it did matter, surely it was only something that mattered for long time modeling -- climate scale forecasting. Surely the ice was already well enough represented to be good enough for weather prediction purposes.

Partisan as I was, and am, in favor of sea ice, I must confess that there were (and are) good reasons to believe that for short range forecasting, you didn't need very accurate representation of sea ice. It doesn't cover much of the surface area of the earth. And, while it might be very reflective, at the times that there is the most ice that is most reflective, there isn't much sun for the ice to reflect. I could have simply sat back in a wrangle with the weather folks, endlessly asserting that sea ice was important, and how much energy sea ice reflected was still important, and weather is chaotic so it had to matter, vs. endless repetitions of their counter-arguments. Perhaps you've seen that sort of thing happen a time or two on a blog or two.

Instead, time to do some science. Run the experiment and see what happens. This has the downsides that it requires my time, and I have to run the risk of the experiment showing that I was wrong -- that modest changes to how much of the sun's energy sea ice reflects really did not affect weather.



But that, seriously, is what makes it a good experiment to run. I wasn't certain, nor was anyone else, exactly how it would turn out. If I turned out to be wrong, then there's a contribution to our understanding of the universe -- indeed weather for a few days really doesn't care a lot about exactly how reflective sea ice is. It was previously assumed and expected that this was the case. But here, finally, would be evidence that the common assumption was correct. If I turned out to be right, and those small changes did matter to weather forecasts, then the contribution is that not only climate (which everyone agreed was sensitive to such things) but weather as well cared. Me being right or wrong is not where the science sat (much as I would prefer to be right, of course). Whether sea ice reflectivity (albedo) mattered for even short time scales is where the science was.

So I ran the experiments. That was a plural because you need more than a single forecast to decide whether a change is for the better. Weather is chaotic, which means, among other things, that things just happen. You could be a little better in the forecast for no reason of skill, but just because the chaos ('the butterflies' we often call it) kicked things over better. I ran experiments -- 5 day forecasts starting on different days. I chose 40 days, from March through June (10 days per month). The 10 days per month, for months spanning the seasonal transition from spring to summer (northern hemisphere) or fall to winter (southern hemisphere) were a large enough collection that the butterflies couldn't swamp the results, and we'd be able to see how much sun-dependence there was.

To represent the reflectivity of ice, I used work done elsewhere (Ross and Walsh, 1987 -- I was working in 1993) as my estimated improved version. That was to replace a representation that dated back to 1964. As usual, we hope that the additional observations that 20 more years of science had would make for a better representation. But, you still have to test it.

As it turned out, the new reflectivity representation really did improve the weather forecasts. See A sea ice albedo experiment with the NMC Medium Range Forecast Model for the gory details.

So the short term good news was both that my prediction was shown to be right, and, not quite as short term, I got to publish an article making an addition to what we understood about how weather works. As it worked out, it was longer before the last leg of good news happened. But it did -- my suggestion for change was incorporated in the NMC's official medium range forecast model.

There was a post script to the story. After the change went in to the operational weather model, I heard from marine forecasters. They had noticed that storms heading up towards the Arctic, particularly those heading past Iceland, were being forecast better by the model. This made their jobs easier. Not that they ever entirely trust the model, but it makes their job easier, and they can do it better, if they are working with the last 30 km of correction, rather than something like 300 km range. (I make up the numbers, the important reality being that the better the model is in providing the first guess, the more accurate the forecaster refinements are.) I looked back to my experimental runs and realized that most of the skill improvement was from the occasional (one or two out of 10) forecast that was very much better at predicting a storm's path.

Never have written that up, though it's been something I've relayed to other people in the 15 years since then.

In the note here, I've focused on the engineering side. The science details are in the paper. But there are a few good illustrations about how science works:
  • There was no drama 
  • A modest improvement was made
  • Interesting things were learned afterwards
The lack of drama is perhaps the most different from what most people think science is like.  The reality is that several thousand papers were published that year (1994) in meteorology/oceanography/glaciology.  Of those, maybe a few dozen were really 'dramatic'.  Call it 1%.  Most of science, the 99%, is an incremental business -- verifying that things really do work out the way that we expect (least exciting), or that models/representations/hypotheses/... are more limited than we hoped (a little more exciting -- how do we handle what happens beyond those limits?!), or that reality is opposite what we expected (interesting!).   For me, the result was what I expected.  For the weather prediction folks, it was against what they expected; so it was more interesting for them than me. 

I'll note this article is the outgrowth of a comment, and my reply, over at A Few Things Ill-Considered (neither having much to do with the main post, the initial comment is at #6, my reply at #8).

8 comments:

matt andrews said...

Thanks, that's a good read.

(BTW, it was Cory's blog, not Tim's...)

Penguindreams said...

Oops. Thanks for the correction. I've updated the text.

skanky said...

There's been some speculation* in the UK that the recent wet summers (well, Julys more specifically) may be influenced by the fact that there's less ice in the Arctic (the direct cause is the jet stream latitude - but that's the same thing as storm tracks).

I personally haven't looked at the correlation (I'm slightly sceptical that there is one, but also think that four years might be a tad short ;) )but 2005 onwards had some very high rainfall totals in summer in various parts of the UK.

This post is very interesting on a number of levels, but it does raise a question as to whether you think there *might* be something in it, based on how much your changes affected the model?



*I'm sure there's been some research, but I'd expect it's too soon to see it - I did do a quick GS search which didn't turn up anything promising).

Penguindreams said...

Always a help if you can provide a link to the speculation. Is this tabloid speculation? Science magazine speculation? Something you've seen in professional scientific literature?

My first reaction is like your first -- 4 years is short to be thinking you've found a climate relationship from data. It's conceivable that someone already had a hypothesis that such a correspondence would exist if sea ice declined. That would require less data for support. I'd still be puzzled at only 4 years sufficing, but not by as much.

Given what I saw -- which, remember, was only 40 forecasts, of only 5 days length -- I would not be surprised if changes in the ice pack were translated in to shifts in the jet stream and storm tracks. One thing I did establish in my research was that the changes in sea ice reflectivity affected the entire depth of the atmosphere. So jet stream effects make sense to me.

But making sense to me isn't enough. Someone will have to sit down and do some experiments focused on this question and see whether the sea ice has the right size effect, and in the right way.

skanky said...

"Always a help if you can provide a link to the speculation. Is this tabloid speculation? Science magazine speculation? Something you've seen in professional scientific literature?"

Sorry yes. The speculation was on meteorology discussion boards. Difficult to link to as they were normally passing comments, questions etc. amongst discussion on the high rainfall and floods, rather than on the possible link specifically, IIRC.

I hadn't paid it a lot of attention, until I read this post.

"One thing I did establish in my research was that the changes in sea ice reflectivity affected the entire depth of the atmosphere. So jet stream effects make sense to me."

That's what I was after really, thanks, so now...

"Someone will have to sit down and do some experiments focused on this question and see whether the sea ice has the right size effect, and in the right way."

...it's something I'll now keep an eye out for.

crandles said...

"8 ) Jara Imbers’ extension of Neil Massey’s experiment. Neil has been running an atmosphere-only model driven with prescribed sea-surface-temperatures which allows higher resolution and shorter work-units. Jara is extending at least a few decades into the future now that we have a range of simulated surface temperature changes from the various coupled experiments (both our own and other peoples) to draw on. This isn’t entirely straightforward, to put it mildly, mostly because of complications with how to make the sea-ice consistent with the surface temperatures (in the coupled models, sea-ice is computed, so while it might be wrong, it is at least self-consistent), but it does provide a way of getting substantially higher atmospheric resolution than is possible in the coupled experiments, which is valuable for impact studies. The software for this experiment already exists (it is identical to Neil’s), but we are working on the driver files."

from http://climateprediction.net/board/viewtopic.php?f=36&t=5927&start=72

further details of experiment
http://climateprediction.net/content/validation-and-attribution-experiment

Just wondering what your reaction to this is. Would the ice issue mean it is doomed to failure to represent storm tracks sensibly?

Penguindreams said...

crandles:
I guess this is another illustration of the saying "All models are wrong, some models are useful." The model I worked on also had the sea ice fixed -- all ice was 3 meters thick, and 100% compact, and did not move, grow, or shrink for the length of the forecast.

So definitely I think you can do something useful with a model that doesn't have what I'd consider good sea ice.

For, say, tropical concerns (and note that half the globe is within 30 degrees of the equator), as long as their ice is not hideously absurd (removing all ice year round, or extending it down to the middle of the Pacific year round), they'll be ok. As the interest moves towards the poles (only 1/6th of the earth's surface is poleward of 60 degrees), the quality of the results will degrade. But it will degrade only to the extent that having sea ice respond interactively matters to the question being asked. Sea ice has a very strong seasonal cycle, such that once you have that right in the model, you've got a very large fraction of the sea ice effects on the atmosphere more or less right as well.

As you go out to climate time scales, it is less reasonable to keep running with the same annual cycle. So for longer term scenarios, you need to do something better with the ice. And, as I found in my work, for very short range forecasts, you also need to do better.

Given the two links you provide, it looks like the experiment is being done in awareness that specifying the sea ice and sea surface temperature (the latter is a bigger issue as it affects 70% of the globe, vs. the 6% or so of sea ice) limits the kinds of conclusions they can draw. The validation and attribution link focuses on a period for attribution where the sea surface temperature and sea ice cover are known, and on a particular question (storminess over the UK) that isn't exceptionally sensitive to sea ice. (It is sensitive -- in particular to Hudson Bay ice cover, I'm told, but not enough to make me concerned.)

crandles said...

Thank you for the discussion. I tend to think trust them, they are experts, but it is nice to get other expert views.