Suppose you believe in theory X. You want to do research in theory X. But all the interesting ideas in theory X have already been well-researched. You can't think of anything interesting and new to say about theory X. There might be some unresolved problems in theory X that need to be tackled. But you can't think of any way to tackle them.
Then theory Y suddenly appears. Theory Y is new, and has lots of unexplored areas that you have the skills to work on. You don't believe in theory Y, but you could make a useful contribution to theory Y in your research.
What would you do?
If you are an ambitious economist, especially an ambitious young economist who needs a thesis topic or publications, you would have a very strong incentive to do your research on theory Y. Even if you weren't ambitious, and just needed something to work on to stay busy, or needed to publish anything to stay respectable, you would have an incentive to work on theory Y.
And it's not necessarily always a bad thing that economists might have an incentive to do research developing theories they don't personally believe in.
We believe theory X is better than theory Y, but we don't know for certain that X is better than Y. Other people may think the opposite. We might be wrong. Making a contribution to theory Y would be better, in expected value terms, than making no contribution to theory X.
Even if we are absolutely convinced that theory X is better than theory Y, maybe the only way to convince others that theory Y is a dead end is to join them in developing theory Y to its conclusion. So that everyone else learns it is a dead end, and we can post a big sign at the entrance to theory Y telling future economists that they shouldn't waste their time exploring this path, because we have already explored it thoroughly and it leads nowhere.
And who knows, we might even pick up a few clues and techniques along the way to the dead end that could be useful in following other, more promising paths.
What we research, and what we believe, aren't necessarily the same thing. What gets published in the journals is a survey of what we are currently researching. It isn't an accurate survey of what we currently believe. The whole point of a journal is not to publish what everybody already believes. The journals are a map of where we are currently exploring for gold. They are not a map of existing gold deposits. They are not a map of where we think gold might be found in places we can't currently explore.
It's not necessarily wrong to believe in theory X but work on theory Y. But there are three dangers we need to be aware of.
The first danger is that others, seeing us working on theory Y, might mistakenly think we believe in theory Y. They don't realise that the journals are not a survey of what we believe. Or, if you treat them as a survey, they are a very biased survey.
The second danger is that we start to believe in the theories we are working on, just because we are working on them, and making progress. We want to believe that our work and progress in theory Y is useful, and will lead to discovering truth. If we can only hammer nails, we really want to believe that all nails need hammering.
The third danger is that we mislead our students. Graduate students need to be taught about "cutting edge" research. Because that's probably where they too will most profitably be swinging machetes in the near future. We will teach them a lot about theory Y. They will probably be taught by professors who themselves are working on theory Y. Those professors may have succumbed to the second danger. The third danger is that the students may think that theory Y is believed to be best, just because they are being taught it.
I think that real business cycle theory is theory Y.
Update: in response to Frances' comment. What I meant to say is that Real Business cyle theory has been one theory Y over that last few years. It won't always be theory Y in future. There have been other theory Y's in the past. Outside of macroeconomics, there are other theory Y's right now.