Predicting Stock Returns for Fun and Profit

By in Economics, Guest Post, Investing, Money Management, Stocks | 9 comments

Can You Predict Future Stock Market Returns?

Guest article by Rob Bennett

Have you ever tried your hand at predicting stock returns?

See what Rob Bennett has learned with his research about predicting stock returns.

I have been studying how to predict stock returns effectively for nine years. The purpose of this article is to put forward a brief summary of what I have learned.

predict the stock market

1) Short-Term Predictions Never Work.

I do not believe that it is possible to predict where stock returns are going to be in six months or in a year or in two years. I have run into some smart people who disagree. What I can say with certainty is that, even if those people are right, it takes a lot of work to predict stock returns in the short-term; more work than most middle-class investors are willing to put to the task.

2) Long-Term Predictions Always Work.

We have 140 years of stock-return data available to us and a mountain of academic research reporting on that data. That research and that data shows that long-term predictions (predictions of where stock prices will be in 10 years or 20 years) always work if performed in a reasonable manner. This is the best-kept secret of stock investing. (Barb’s comment- No prediction, no matter it’s basis is 100% accurate.)

3) Investor Emotion Cannot Be Predicted.

There’s much more demand for predicting stock returns in the short-term than there is for long-term predictions. So, if there were some means of making effective short-term predictions, I would very much like to learn about them. However, I have come to the conclusion that short-term predictions will never work because the primary influence on stock prices in the short term is investor emotion. Emotion is by definition an irrational phenomena. So it cannot be predicted.

4) The Greater the Influence of the Economic Realities on Stock Prices, the More Predictable Returns Become.

The obvious question is — Given how hard it is to make good short-term predictions, how can it be that long-term predictions always work? I believe it is because investor emotion is not the primary influence on stock prices in the long term. After the passage of about 10 years, the economic realities become the dominant influence. The economic realities are known. And the economic realities are rational phenomena. So long-term predictions lend themselves to prediction by those willing to take the time to look up a few numbers.

5) It Is Far Easier to Make Effective Predictions Than Most People Imagine.

My guess is that most people don’t bother trying to make long-term predictions because they assume it would take a lot of work to pull them off. Nothing could be farther from the truth. Every factor that affects the price of a broad stock index is reflected in the price of that index. So you don’t need to worry about inflation or productivity or consumer confidence or anything else.

There’s only one exception to that general rule. Overvaluation and undervaluation are never reflected in the nominal (not adjusted for inflation) market price because these two factors are by definition indicative of mispricings. Look at the nominal price of an index, make an adjustment for the overvaluation or undervaluation that will be disappearing over the course of the next 10 years (it is an “Iron Law” of stock investing that prices are always in the process of reverting to the mean, according to Vanguard Founder John Bogle) and you have a ballpark number for where stock prices will stand in 10 years.

Barbara’s comment; As the average stock market returns during the past few years have been close to 20%, it’s not unreasonable to expect future stock market returns in the low single digits. Since the long term mean, or overage stock market return is approximately 9 percent and recent returns have been double the average, going forward it’s likely we’ll experience a few below average return years.

6) Long-Term Predictions Can Never Be Precise.

We know that from any valuation level stock prices will be headed in the direction of fair value over the course of the next 10 years. But that 10-year number will also be influenced to some extent by the investor emotion that is dominant at that time. Remember, investor emotion is never predictable. So we can never make precise return predictions, even in the long term.

7) Our Prediction Abilities Are Real But Limited.

So what can we do? We can identify a range of possible returns that will apply in 10 years and assign rough probabilities to particular points along the range. For example, the P/E 10 value (the price of the S&P 500 over the average of its last 10 years of earnings) is today “26″ (SeptemberAugust, 2014). A regression analysis of the historical data from the stock-return predictor calculator, indicates the following future return probabilities:

stock market valuation tool

Barbara’s update: According to Bennett’s Stock Return Predictor tool, during the next 20 years, the most likely “real’ (before inflation) stock market returs would be 2.72 percent with a best case scenario of 6.72 percent and a worst possible picture of -1.28 percent.

Since a real return doesn’t take into account inflation, you can tack on the average inflation rate of about 3.00 percent and see what the nominal return would be. Using an approximate nominal return, even the worst case 20 year return scenario, would be a positive return of 1.72 percent[3.00 + (-1.28)]. And the best case return for the upcoming 20 years would be 10.88 percent (including 3 percent inflation).

8) Knowing This Much Provides a Huge Edge.

The long-term benefit obtained by knowing this much is counter-intuitively large. Investors who change their stock allocations in response to big valuation shifts sooner or later nearly always go ahead of those who do not do so. The edge is often small. But that edge grows and grows over the years as a result of the magic of compounding returns. Over the course of an investing lifetime, investors who employ long-term predictions to inform their stock allocation choices can realistically expect to be able to retire five to ten years sooner than those investors who elect not to do so.

Rob Bennett often writes about stock market investment research and the tricky tricks used to trick you. 

Chime in, do you think stock market returns are predictable or not?

A version of this article was previously published.


  1. I definitely think it is possible to predict long term prices. Short term prices are harder to predict because there is more volatility in the short term. You can do it with fundamental analysis or technical analysis. I prefer a mix of both. I use technical analysis for a general idea of the direction and then fundamental analysis to figure out what the catalyst will be that creates the move.

    Financial Success for Young Adults

    September 19, 2011

  2. I’m not very good at either long-term or short-term, but I do try to take a whack at long-term trends.

    101 Centavos

    September 19, 2011

  3. You may want to take a look at this study by Associate Professor Wade Pfau, 101 Centavos:

    He finds that: “[Long-term] market timing strategies provide comparable returns as a 100 percent stocks Buy-and-Hold strategy but with substantially less risk. Meanwhile, [long-term] market timing provides comparable risks and the same average asset allocation as a 50/50 fixed allocation strategy, but with much higher returns.”

    I’ve done statistical tests showing that switching to Valuation-Informed Indexing would permit the typical middle-class investor to be able to retire five to ten years sooner than he could following a Buy-and-Hold strategy. It can take 10 years to go ahead. But once you go ahead, the benefits of compounding returns kick in and the differential can grow very large over the course of an investing lifetime.


    Rob Bennett

    September 20, 2011

  4. These are really practical tips. I think the long term approach to anything is the best route to go, including when it comes to investing. I also agree that emotions can play a large roll in the outcome. I think it is much easier to research a trend over the long term than to try to guess which way something is going to go in the short term.

    Miss T @ Prairie Eco-Thrifter

    September 20, 2011

  5. The long term prediction is always more precise since there is a lot of (and complete) data to be used in the prediction process. In contrary, the shorter the time frame, less precise the prediction result.


    September 20, 2011

  6. I think you can and can’t predict everything. I can be a know-it-all and say the market always goes up, but does it for sure?

    Robert @ My Multiple Incomes

    September 21, 2011

  7. you can and can’t predict everything.

    This is a good way of putting it.

    What people need to do to make sense of this is to add a time element. Some people stay fully invested in stocks at all times and do so on the thinking that stocks always go up in the long run. There is a sense in which this is really so. The question they are failing to examine is — How long is it going to be before I earn a return from stocks equal to what I could have earned in a far safer asset class?

    In 2000, Treasury Inflation-Protected Securities were paying 4 percent real. Stocks were priced to pay an annualized 10-year return of a negative 1 percent real. So those who went with stocks were giving up 5 percentage points of return per year for 10 years running. Do the math and that comes to a total loss of your initial portfolio value of 50 percent. Make a decision that causes you to lose 50 percent of your accumulated savings of a lifetime and you delay your retirement by many years, perhaps decades. And you also lose the compounding returns you would have had on those returns for many years into the future.

    Now —

    Will those who invested in stocks in 2000 ever come out ahead? They will. The same data that shows that the likely 10-year return was a negative 1 percent real also shows that the likely 30-year return was 5 percent real. At the end of 30 years, the stock investor goes ahead of the TIPS investor even for stocks purchased at the prices that applied in 2000.

    But there is no law that says that the TIPS investor needs to stay in TIPS after the valuations for stocks improve. The best approach is to lower your stock allocation a bit when prices are insanely high and stocks are insanely risky and then increase it a bit when stocks are again selling at the prices from which they have never in history delivered a poor long-term return. Investors who are open to adjusting their allocations a bit to take the price being charged for stocks into consideration have the best of all worlds.

    The trouble with Buy-and-Hold is that you lost 30 years of compounding on the amount by which you were down all those years. So you never really get even with the TIPS investor. Also, the losses suffered by the stock investor may at times have been so great that he was tempted to get out of stocks entirely, which is the worst thing you can do.

    The Valuation-Informed Indexer might have gone to a 20 percent stock allocation in 2000. That way he limited his losses during the time when stocks offered a poor long-term value proposition but kept something in stocks to cover the possibility that we might see a few years of great returns despite the high prices (remember, short-term returns are entirely unpredictable). This investor is covered for both possibilities and he he keeping his risk profile roughly constant (the risk of investing in stocks is far greater when prices are high, so to remain at the same risk level you chose when prices were reasonable you need to lower your stock allocation).

    I have a calculator at my web site that lets you test as many random but realistic 30-year return sequences as you like. You enter a stock allocation, the calculator chooses a return for the year that is realistic given what we have seen in the historical record, and then you have the option of choosing a new allocation or sticking with the same one. At the end of the 30 years, you can compare the portfolio amount that results from following a Valuation-Informed Indexing strategy with what you would have had if you had followed a Buy-and-Hold strategy. I have run hundreds of tests and I beat the numbers for Buy-and-Hold in about 90 percent of the tests. There have been cases in which my portfolio size ends up being double the size of the Buy-and-Hold portfolio. There is academic research showing that Valuation-Informed Indexing ALWAYS beats Buy-and-Hold on a risk-adjusted basis in the long-term.

    The reason why the results are so good is that Valuation-Informed Indexing puts the odds on your side. With Buy-and-Hold, you are going with a high stock allocation both when that is a high probability bet and when that is very, very low probability bet. With Valuation-Informed Indexing, the probabilities always favor you.

    It is true that we cannot predict everything. We cannot predict the short term at all. And we cannot predict the long-term precisely. We can never know what returns sequence is going to pop up. So there is always a range of possibilities that you need to consider, not one definite result that you know is going to turn up.

    That said, there are circumstances where ALL or just about all possibilities call for the same allocation. When stocks are selling at the prices they were selling at in 2000, there are no good long-term possibilities for stock investors unless you are willing to wait 30 years to see a good return. At 10 years out, every realistic possibility is bad for stock investors at those prices.

    So, while I don’t want to oversell this and say that you can know your exact stock return in advance, I do think that the power here to enhance your return while diminishing your risk is hard to overstate. This is a very, very big deal. I see it as the most important breakthrough in our understanding of how stock investing works achieved in my lifetime.

    A statistical test shows that engaging in long-term timing cuts the risk of stock investing by 78 percent. And it greatly enhances returns as well. My estimate is that any middle-class person who elected to this investing approach could expect to be able to retire 5 to 10 years sooner than would have been possible following a Buy-and-Hold strategy.

    We cannot predict everything but we can predict some things. And I think we should be making more of an effort to let people know about the things we really can predict effectively. The finance industry does not want people knowing about this because they like it if we feel that we cannot predict anything and thus we just need to remain fully invested in stocks at all times. I believe that we should be telling the accurate story, which is that some things really cannot be predicted and some things really can be predicted.

    Here is the link for the calculator I mentioned above in case anyone wants to give it a whirl:


    Rob Bennett

    September 22, 2011

  8. Great Post Rob. Some really interesting insights. I also don’t believe in short term predictions as there are too many factors which change the dynamics of the market. I always take a long term approach and protect myself with options.


  1. Historical stock and bond returns - […] Financial educators frequently use the historical return data to help you figure out what returns to expect in the…
  2. The Magic of Compounding-Get Rich While You Sleep - […] fund or ETF and forget about it. Fast forward 20, 30, or 40 years, it is highly likely that…
  3. How to Prepare for Inflation - […] I am not great at predicting the future. But with the current inflation rate at -0.1% for the 12…

Post a Reply

Your email address will not be published. Required fields are marked *