Knowing the Limits of Financial Models

Checking the weather? Guess what — you’re using a model. While models can be useful for gaining insights for making good decisions, they are inherently incomplete simplifications of reality.

As disappointed investors increasingly move away from traditional active management investing and hedge funds, products like ETFs based on factor models have become a popular investing alternative for driving sales by firms and advisors competing in a crowded marketplace. Often nowadays promoted as “smart beta strategies,” factor model-version products are based on underlying scientific financial models. But they also have serious limitations that few investors looking at past performance alone know about when making critical wealth planning decisions.

To help shed light on financial models for planning investment strategy, let’s start by examining an everyday example: a daily weather forecast. Using data on current and past weather conditions, a meteorologist makes numerous assumptions and then attempts to estimate what the weather will be the next day or week. The weatherman’s model may help you decide if you should bring an umbrella when you leave the house in the morning. However, as anyone knows who has been caught without an umbrella in an unpredicted rain shower, reality often behaves differently than the weatherman’s model.

In investment management strategy, models are used to gain insights to inform us about important investment decisions. Financial researchers frequently look for new models to help better answer questions like, “What drives returns?” These models are often touted as being complex and sophisticated and incite academic debates about who has a better model. Investors who are evaluating alternative investment approaches mimicking those models can benefit from a better understanding that no model, just like the weather, can explain the reality of markets. Hence, investors should be wary of many highly promoted “quantitative” approaches whose model version requires a high degree of trust.

Very simply, the risks of a security or a portfolio of securities, just like the weather, cannot be completely “explained” by any model — even one that has become well-accepted, but variously adapted. And even then, few models promoted by the financial industry are understood by them as academic models by those who developed them.

Minding the Judgment Gap

As with weather forecasts, even if the model is respected, investment models rely on multiple inputs. Instead of things like barometric pressure or wind conditions, investment models involve variables like the expected return or volatility of many different securities. For example, using these sorts of inputs, some popular models may recommend an “optimal” mix of securities based on the expected interaction of asset classes or securities with one another over time. Users should be extremely cautious. The saying “garbage in, garbage out” definitely applies to inputs used for “optimization” models. In other words, a model’s output can only be as good as its input.

Also, poor assumptions built into the model obviously will lead to poor decisions. However, even with sound underlying assumptions, a user who places too much faith in inherently imprecise inputs, can expose themselves to extreme model results and consequently a disappointing experience if they act blindly on those results.

Given these limitations, bringing financial research into practice requires an informed advisor if not an informed client — and someone having a very acute awareness of the model’s limitations to identify when and how a particular model is appropriate. Since no model perfectly represents reality, you don’t ask, “Is this model true or false?” (to which the answer is always false), you should ask, “How does this model help me better understand the world, to make a better decisions?” and, “In what ways can the model be wrong, to limit my potential risks?”

When evaluating investment approaches relying on financial models, you must establish how confident you can be in your financial advisor’s ability to test and implement ideas garnered from model he presents. This step requires your judgment and trust in the skill and experience of your advisor and the management solution he recommends.

The transparency offered by some approaches, such as conventional index funds, requires a low level of trust because their model is quite simple: it’s very easy to evaluate whether a particular index fund has matched the return of a specific index: Either Vanguard’s S&P 500 Index matches closely the actual Standard & Poor 500 index of large US stocks or it does not. The tradeoff with this level of mechanical transparency is that it may sacrifice potential higher returns, as it prioritizes matching the index over any possible benefits from management discretion.

But even if a model is highly reliable, does not mean it’s practical for wealth management. In the case of an index fund replicating the S&P 500 index from January 2000 to December 2009, an index manager may brag to the public about how very close his actual return was to the actual index within basis points, but how many retired investors relying on their portfolio for income would have considered return for ten long years of nearly minus 1 percent a year a successful experience — or even stuck around that long?

For more opaque and complex quantitative strategies like those used by hedge funds, the requisite level of trust needed is vastly higher. You (or your advisor) should clearly understand the model’s inputs and assumptions and limitations, how well as how the model is implemented in practice. Finally, have a method to evaluate outcomes periodically.

Rigorous attention must be paid to how any financial model is implemented. To quote Nobel laureate Robert Merton, successful use of a model is “10% inspiration and 90% perspiration.” In other words, having a good idea is just the beginning. Most of the work is implementing the ideas contained in the model and making it work in the real world. Typically there is a huge gap between theory and practice — as so many disappointed hedge fund and ETF investors over the years have sadly learned.


In the end, the difference between judiciously using a model to guide your decisions and blindly following it can be hazardous to your wealth. Cutting through the marketing hype around the “latest and greatest” factor models and identifying which one to use may mean realizing your values, goal and dreams for yourself and your family — or not.

Taking action

The importance of having an customized wealth management framework cannot be overemphasized. A professional retirement specialist working with a network of experts in finance, accounting, tax and law who works with successful families like yours, can help you plan, protect and pass on your wealth. A professional wealth management process for smart decision-making will help put you in control of your future, gain financial freedom, and give you peace of mind. By avoiding costly mistakes from what you don’t know, and from what you don’t know that you don’t know, you chances increase of an abundant retirement, and leaving a legacy for family, community, and causes you care deeply about.

For more information about financial models and the limitations, see Paul Byron Hill, “Informed Strategy: Models and the Art of Science” (Planning Perspectives, 3rd Quarter 2017)