I'm bad at econometrics. I've got a couple of simple questions, that ought to have a simple answer. Noah Smith's post (HT Mark Thoma) reminded me of it. There are probably other students at the back of the econometrics class who don't know the answer either, so I'm going to ask for all of us.
A perfect weather forecaster would only ever say "P(t)=1.0" or "P(t)=0.0", and would always be right. So R(t)=P(t) for all t.
But if both forecasters are imperfect, how do we use the data to tell us which forecaster was better? Or how good each one is on a scale with pure guess at one end and a crystal ball at the other end? How can we estimate the degree to which a forecaster was over-confident or under-confident in his own forecasting ability? What about bias?
Simple intuitive answers preferred, if possible. Thanks.