Confident forecasters highly sought and notoriously wrong

One of the biggest challenges in the financial business is that once you are seen as an “expert” people start asking you to forecast the future. Worse, whenever you are asked to speak at a conference, panel or media spot, most often the interviewers want you to bring your “buy ideas” and by that they usually mean specifically equity buy ideas. If you say you aren't keen on equity valuations and have no screaming buys on your list at a given time, the general response is one of disappointment that you are so bearish. You start feeling like the smelly kid at the high school dance.
The most sought after speakers are typically the one's that are the most confident, bullish and sure of their predictions. Even though in reality, these are the worst possible people to listen to. The guys that are telling you without a blink that oil will be $200 by the end of 2011, or that gold will be $2,000 in 6 months, or there is no downturn in sight and the Dow will be 14,000 within the next year…all very dangerous. I am reminded of Galbraith's infamous quip, “there are only two kinds of forecasters those who don't know what is going to happen, and those who don't know they don't know…”
Another excellent quote on this comes from the Big Picture blog today:
“One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision.”
-Bertrand Russell
For the record, I am painfully aware of the danger and complexity of the future and especially when it comes to intensely emotional things like publicly traded financial markets. I have learned over the years that those who are most useful in this area are those who have trained themselves to watch for key, objective markers that suggest upside and most importantly– downside risks. The bulk of financial participants and pundits are fixated only on the upside of their particular investment product, idea or thesis. This wilful blindness makes them dangerous. If we miss some upside opportunity it can be frustrating no doubt, but where we fail to see and plan for downside risks it is frequently devastating.
This all ties in nicely with a clip I saw on BNN last night (I'll watch anything to get through my heart-thumping 45 minutes on the Elliptical machine).
Author Dan Gardner was explaining his new book Future Babble which outlines a recent study on what attributes make the best and worse financial forecasters.
It is an interesting discussion, according to Gardner the more confident and far in the future one puts their predictions the least likely they are to be correct. The most important link to accuracy is one's “style of thinking”. It's not that there is no value to making forecasts; we have to make an effort to foresee the future. But the rare useful forecasters are the one's who know their limitations and are comfortable with complexity and uncertainty even though this is not what the average listener wants to hear.
Watch the clip here.“In 2008 the price of oil blew past $140 a barrel and was predicted to be on its way to $200. It plummeted to about $30. From the price of oil and gold to US homes, economists, pundits and market makers have been notoriously wrong when it comes to predicting the future. Are we doomed to repeat the mistakes of the past when we try to predict the future, BNN speaks to Dan Gardner, author of Future Babble.”