I can figure out how some things work. I can't figure out how other things work. But there are some things I can't figure out how it's even possible for them to work. Using macroeconomic models to make macroeconomic forecasts falls into that third category of things. I can't see how it's even possible to do it. And yet people claim to do it all the time.
This is really simple stuff. Either I'm making some awfully simple mistake, or all the people who claim to be making model-based macroeconomic forecasts are just blowing smoke. They must have thought about this problem. I'm laying bare my ignorance (I never claimed to know much econometrics anyway). Someone's got to ask this stupid question; it might as well be me.
Start with a structural model. It has a system of equations S. Each equation relates one endogenous variable y to other endogenous variables Y, and exogenous variables X. So Y=S(Y,X). We now solve that system to get the reduced form system of equations R, with all the Y variables on the left hand side, and all the X variables on the right hand side. So Y=R(X).
And let's just assume that we know for sure that our model is the absolute truth about how the Y variables are determined, and that we know with absolute certainty the exact numerical values of all the parameters in R. So if we know X, the model can tell us Y. So far so good.
How does this model give us any help whatsoever to forecast future values of Y?
Let me lay out the timing more explicitly.
Suppose it is a very simple macroeconomic model, with no lags. So Y at time t is determined by X at time t. Y(t)=R(X(t)).
If Statistics Canada tells us what X(t) is, we can use the model to tell us what Y(t) is. But that's unlikely to be of much help. If Statistics Canada can tell us what X(t) is, they can usually also tell us what Y(t) is. We don't need the model. If we are trying to forecast what Y will be next year, so t is 2011, we will need to know what X will be in 2011. Statistics Canada can't help us. They won't do the survey until next year. So the model can't help us either. The only case where the model would be of any use is if Statistics Canada has a one month lag in collecting data on all variables in X, and a two month lag in collecting data on Y. In this case, we could use the model to "forecast" last month's Y.
Suppose instead that the model has a consistent one period lag, so Y(t)=R(X(t-1)). And suppose that Statistics Canada reports all data on X immediately. Now we can use the model for genuine forecasting of the future. Statistics Canada tells us what X is today, and we use the model to tell us what Y will be one period in the future. But the model can tell us nothing about what Y will be two periods in the future, because Statistics Canada can't tell us what X will be one period in the future. And yet I keep hearing about model-based forecasts for one, two, three, four, etc., periods in the future.
Even worse, I have never heard of a single macroeconomic model that has the lag structure Y(t)=R(X(t-1)). And even worser, I can't even imagine that any reasonable macroeconomic model could possibly have that lag structure. Doesn't any contemporaneous X variable have any effect? Because if it does, then the model looks like Y(t)=R(X(t-1),X(t)), and we are right back where we started. We can't forecast next year's Y unless Statistics Canada tells us next year's X, and they can't.
Here is my guess about what people who claim to make model-based forecasts are actually doing. They have a model Y(t)=R(X(t)). They then make a non model-based forecast of X(t+1). They then substitute that forecast for X(t+1) into the model, and solve for Y(t+1).
Obviously, even if the model is exact and true, the forecast of Y(t+1) is only as good as the non model-based forecast of X(t+1). Garbage X(t+1) in, garbage Y(t+1) out.
Less obviously, I can think of no reason to believe that this two-stage procedure would give us any better forecasts of Y(t+1) than a simple one-stage non model-based forecast of Y(t+1). And that's even if you knew the model is true and its parameters exactly right, which of course it never is. Instead of sticking a curvy ruler on the time-series for X, extrapolating out, and plugging the forecast for X(t+1) into the model, why not just stick a curvy ruler on the time-series for Y, and forget about the model? (For "curvy ruler" read "Vector-Auto-Regression", if you like).
Macroeconomic models help us understand stuff. I can see how they help us make conditional forecasts, like if we increase x by $1, how much will y increase, ceteris X paribus? I can even see how they help us make sure that forecasts of Y(t+1) and X(t+1) are internally consistent, so that if you are wrong on some variables you will be wrong on all variables. But I can't see how they help us make forecasts.
So, what am I missing? Or did everybody already know this, and know that model-based macroeconomic forecasts are really just smoke?