22 July 2011

Reconsidering forecasts and wagers

Comments in two different threads suggest that there's some good room for more discussion.  Yesterday, I noted that if someone would only take a bet at 50 to 1 odds, he wasn't very confident about his side of the bet.  Certainly not the 'just as likely to warm as cool' that was the statement which prompted the bet.  (Nobody here, by the way.)  As M commented, and I assume that he mean in terms of real money, he'd want 5:1 odds even on an even money situation.  I wouldn't mind that myself (again, I don't bet real money, but in examining the mathematics of expectation, that's how it goes).

For illustration of a concept that was in my mind, but not in the prior post, I'll pick up with Alastair's comments about his predictions and preferences for odds.  The thing is, the odds that you're willing to accept also describe what you think is really the case.  At least it's much closer than in the case that your lunch money is riding on the bet and you're already hungry.  Our situation here is betting something that doesn't exist (quatloos) and presuming that we can make a lot of bets, so that we can average out the wins and losses over time -- converging towards the mathematical situation.

I make use of this in group settings when lunch place selection is being discussed.  In a group of, say, 6, all will claim to have no preference between places A and B.  I pull out a coin and say, fine, heads it's A, tails it's B.  An amazing number of people suddenly develop a preference for one or the other.

So it goes with Alastair's estimate of 3.9 million km^2 against mine of 4.4.  His original estimate for uncertainty was 0.1 million km^2, while mine was 0.5 million.  I'll add a curve for him, where the uncertainty is increased to match mine; that'll be the 'Alastair-2' curve:
Here begins the fun and games of the mathematics.  My expectation, if I thought my statistical guess were seriously good, which I don't (hence the 3 different methods for estimating the September cover) is the blue curve.  Alastair's original is the orangish -- the one with the very narrow peak.  And the curve for less confidence is the yellow -- same peak, but broader spread.

To the extent that we believe any of our estimates, and estimates of uncertainty, the probability that we're predicting for a given range of extents is the area under the curve between those.  You see essentially no area under Alastair's original curve for extents over 4.2 million km^2.  That represents a high confidence in a new record.  My statistical guess shows some area under the blue curve for extents under 4.2, meaning that I wouldn't be shocked if a new record were set this year.  I would, on the other hand, be shocked if the ice extent were over 5.5 million km^2.

In his second comment, Alastair mentioned a good science point -- he had changed his forecast method because it hadn't worked in previous years.  That's key to doing science.  Being right is the goal, and the way you learn how to be right is to change your methods as reality disagrees with your expectation.  Mistakes are fine -- learning from them is where you're doing the science.

Alastair also mentioned that he thinks there's a 50% chance of a new record this year.  The old record was 4.3 million km^2.  His original estimates give essentially 100% chance for that.  My modification shows an 86% chance for his less confident estimate.  To get it down to 50%, we have to increase the uncertainty (the standard deviation of the curve) to 4!  That's ... extraordinary, to say the least.  The difference between highest and lowest years is not even that large.  The other way to get the probability of a new record to 50% is to make the center of his expectation to be 4.3 million km^2 -- practically the same as my prediction of 4.4.

Taking my own numbers, the guess translates to a 37.5% chance of a new record this year.  Let's look at that.  Does my intuition complain when it sees that number?  Not really.  If it were to be 98%, or even 86%, I'd be concerned.  But somewhat less than half ... doesn't seem outrageous.  2% would also be unreasonable.  We set the record in only 30 years, which means a 3% chance in each year.  And there's a downward trend in the extent, so the chances of a new record should be increasing year by year.

It looks like I misplaced a decimal point in figuring things the other day.  Here's a table of areas (probabilities) of various outcomes with respect to my curve and the two I'm using for Alastair:

OutcomeBobAlastair-1Alastair-2
Extent below 4.150.250.99990.77
Extent below 4.300.3751.00.86
Extent between 3.8 and 4.00.0800.8380.223

So our expectations for the ice cover being below 4.15 are really more like 4:1, Alastair.  Does that sound more attractive?  3:1 for the less confident version of your prediction.  For setting a new record, your 50% chance translates to  4:3 odds between us -- you win 3 quatloos if it is a new record, pay 4 if it isn't.

5 comments:

Alastair said...

Bob,
You say you do not bet money, which rather explains why you just don't get it.

What I learnt from my last bets with you and W. two years ago was that getting good odds does not mean that you automatically win. For instance, on a roulette wheel there are 35 numbers and two zero slots. The odds of winning are 1:37 but the payout is only 1:35. That is how the house makes a profit. Now suppose some naive person decided to set up a new gambling house and offers payouts of 40:1 to attract clients. If I went there and bet my shirt on 13, my lucky number, I would be getting goods odds but would almost certainly lose my shirt.

That was what happened when I bet with you that there would be a new record minimum two years after the previous one. I had thought that the odds of a record one year after the the first were evens so that the following year those odds were a bargain. So my judgement that I was getting good odds may have been correct, but I lost my money/quatloos.

Of course you will say I should have doubled up my bet the following year but that would have lost too. So I would have to double up again this year but you (and W.) are not giving those same advantageous odds now for a new minimum!

Does this mean that that teh ice is not retreating and there will never be a new minimum - no. Does thai mean I would be foolish to bet on it- yes.

This year your spread of possibilities pretty well matches mine, so I am hardly going to bet against that. If you compare the first two curves, my predictions overlaps about an eighth of yours. (I leave you to do the maths :-). So you should be offering me 8:1, or 10:1 to make it worth my while, as I originally proposed.

You need to make up your mind quick because I am having second thoughts now that the IJIS extent curveis flattening out :-(

Cheers, Alastair.

Robert Grumbine said...

No problem Alastair. To continue your roulette example, the US roulette wheel has 37 slots, but pays off only 35:1. The difference, as you suggest is why the house always wins in the long run.

On the other hand, with the break even odds, the statistical expectations, being 37:1, they're not paying out only 5:1.

Once we've figured out what the statistical expectations are, we can then see what kind of hedging to the odds make sense to either of us. If I compute the statistics being 1:1, I'm hardly going to offer you 10:1!

Anyhow, no worries. My purpose here is illustration of the mathematics, not accumulating the largest pile of quatloos.

Anonymous said...

Actually, there is another reason to avoid 50/50 1:1 bets. They are almost never "fair". While I disagree with Alastair's logic, he is approaching what I think is a decent point: a "fair" 1:1 bet should be somewhere between my 50/50 point and the 50/50 point of the person I'm betting with.

If I believe the most likely sea ice extent in September is 5 million km^2, and you believe it is 4 million km^2, then a 1:1 bet at 5 million is not at all fair. A 1:1 bet at 4.5 million would be fair, and so I am within my rights to turn down the 1:1 bet at 5 million (even if it is just for quatloos).

-M

Alastair said...

OK, 'll bet you 50Q at 10:1 that this year's summer Arctic sea ice extent falls below 4.0M sq km and does not drop below 3.9 sq km as recorded on the IJIS site.

Robert Grumbine said...

Since JAXA/IJIS gives higher figures than NSIDC, that sounds like a good bet for me. If IJIS is above 4.0 or below 3.9 million km^2 extent, you owe me 50Q. If it's between 4.0 and 3.9, inclusive, I owe you 500Q.