Monday, January 27, 2014

Prices and Returns

Warning: this will only be interesting to academic finance people.

One of the fun things about teaching is that it forces me to look back at old ideas and refine them. Last week, I needed a problem set for my MBA class. It occurred to me, why not have them do for returns what Shiller did for dividends?

Here it is

At each date \( t \) I plot the return and final terms of the Campbell-Shiller identity

\( p_t - d_t = \sum_{\tau=t}^T \rho^{\tau-t-1} \Delta d_{\tau} - \sum_{\tau=t}^T \rho^{\tau-t-1} r_{\tau} + \rho^{T-\tau} \left( p_T-d_T \right) \)

where p = log price, d = log dividend, r = return, \(\rho = 0.96\)

In words, plot at each date the actual price-dividend ratio, the corresponding ex-post dividends, the corresponding ex-post return, and ex-post terminal price. (There is no expectation on the right hand side.) Shiller plots the price, dividend and terminal price term, (see here). I'm just adding the return term.

What does this mean? Shiller's plots contrast actual prices with what prices would be if clairvoyant investors knew what actual dividends would be and discounted at a constant rate. It's a total bust. Here, we're looking at, what would prices be if clairvoyant investors knew what actual returns were going to be, but thought dividends would never change. As you can see, actual prices almost exactly mirror these "ex-post rationally discounted" prices!

A second, deeper, meaning. It is more conventional to make this decomposition using expected values,

\( p_t - d_t = E_t \left[ \sum_{\tau=t}^T \rho^{\tau-t-1} \Delta d_{\tau} - \sum_{\tau=t}^T \rho^{\tau-t-1} r_{\tau} + \rho^{T-\tau} \left( p_T-d_T \right) \right] \)

For \(E_t\) we use regression based forecasts, for example by running long-run returns on dividend yields and a vector of other variables. If you use dividend yields as the forecasting variable then each term is just a number times dividend yield at time t. To be specific, if you run

\( \sum_{j=1}^k \rho^{j-1} r_{t+j} = a + b^r \times (p_t-d_t)+\varepsilon^r \)

Then the three terms are

\( p_t - d_t = b^d \times (p_t-d_t) - b^r \times (p_t-d_t) + b^{pd} \times (p_t-d_t) \)

Since \(b^r \approx -1 \) (i.e. the regression coefficient of long run returns on d-p has a coefficient of about +1) \( b^d \approx 0 , \ b^{pd}\approx 0 \) we see that the discount rate term accounts for all price volatility. Plotting the terms is pretty boring: 

Yes, there are separate red and blue lines. Price-dividend ratios do not forecast dividend growth so the green line is flat. Price dividend ratios do forecast returns, just enough to account for the volatility of prices.

Now, to the point:  What if we add more variables to forecast returns and dividend growth? Investors surely use lots of information.  That would surely change our understanding of the sources of price volatility, no?  In "Discount rates" I tried Lettau and Ludvigson's cay variable. It did a great job of forecasting short run returns. But it decays quickly, and doesn't change this long run picture much at all.

Ok, but surely there are other variables out there that can forecast returns and dividend growth, that could upend the whole picture, no?

My top picture answers that question. Even if you can perfectly forecast returns, you will not substantially change the decomposition of price-dividend ratio volatility. The ex-post values are a sort of upper bound for how much things can ever change, no matter how much more information we stick in the VAR. And the answer is, no matter how we change short-run return forecasts, no matter what information set we use, the decomposition of price volatility will still say the vast majority of price-dividend ratio variation comes from expected returns. (And, likewise, Shiller's plots for ex-post dividends say that no matter how many variables you try, dividend forecasts will not explain much price-dividend ratio volatility.)

You may either pity or admire my MBAs who put up with this sort of thing on a weekly basis. If you want more details or documentation, it's problem set 3 here. Now, back to writing Problem set 5.


  1. I am so glad to see that your class is still the #1 learning experience you can have at Booth!

    1. Thanks for the support, and I wish it were true. Well, no, it's good to work where my colleagues teach such great classes.

  2. So the class is still MATLAB focused then? Any plans to support other packages?

    1. Students can use any package they want, matlab, r, python, gauss, octave, etc. One masochist did it all in excel/visual basic.

    2. Matlab's pretty but too slow for industry, imho. Student's should investigate python/Cython or, the new kid on the block, Julia.

  3. This chart is awesome, and made me think a lot. Thank you, John!
    Maybe we should make a similar picture for corporate bonds, which have terminal price of one (so we can erase your red curve for \rho^T p_T - d_T).

    In "Discount Rates", you are focusing on long-run forecasting coefficients on D/P ratio.
    In multivariate regressions, it is tricky as D/P ratio gets orthogonalized by other variables.
    If you put a new variable which forecasts returns incredibly well, so that the return forecasting coefficient on D/P becomes zero, then is it right to conclude that the orthogonalized variance of D/P entirely corresponds to dividend growth? I don't think so. In the end, it seems to me that we have to look at the volatility ratio, vol(expected long-run dividend growth)/vol(D/P). We want to understand the information in D/P, not orthogonalized D/P. So why do we care so much about long-run forecasting coefficients? Of course, in univariate regressions, the long-run coefficient is equal to the volatility ratio, but it is a very special case.

    1. Hi Yoshio. Tough questions as always.

      I think government bonds would have an almost perfect plot. Variation in prices corresponds 100% to variation in return from purchase to maturity. Corporate bonds are more interesting -- you've got the data.

      That means we think of stock price-dividend ratios as exactly the same thing as a bond. Then actual prices and returns also move when essentially random-walk dividends also move.

      Your multivariate question is trickier. We still have p-d = (diividend growth term) - (return term), and that holds ex post.. suppose z_t = return term , i.e we find a perfect long-run return forecasting varialble. Now the multiple regression of long run return on pd and z will have a zero coefficient on pd, but pd and z are so correlated that a plot of pd and expected long run returns will be highly correlated -- it will be exactly my plot.

      Volatility isn't a good measure, because then you have covariance terms. I'm still thinking about how to generalize the interpretation of single regression forecast coefficients as a variance decomposition to multiple regression.

      No answer, just a great question.

  4. There are a number of qualifications to be made here.

    First, the finding that dividend growth is unpredictable by the dividend-price ratio is very much dependent on sample period and on how dividends are reinvested, see e.g.

    Second, outside the US dividend growth is strongly predictable by the dividend-price ratio in many countries, see e.g.

    Third, a periodically collapsing rational bubble with constant expected returns generate exactly the predictive results in Cochrane's Dog That Did Not Bark paper (RFS 2008), see . Thus, in principle the price movements and return predictability found may be due to bubbles.

  5. Wow. Great comment. Thanks.

    I digest things as follows. Dividends do seem to be forecastable about one year ahead. Fama and French used price/next year's dividends to capture this fact. Reinvested dividends are also a sneaky way to do the same thing. For example, fall 2008, trailing dividends did not fall, but reinvested dividends fell a lot as the market crashed -- and a lot of the market crash was lowered expectations of 2009 dividends. But a full model would show, I think, about a year's worth of dividend growth predictability and flat after that.

    An important item on the agenda is do to this right; get a monthly model going that overcomes the seasonality in dividends. But, don't forget, use variables that obey the identity! I have seen papers that use nonreinvested dividends at annual frequency -- but then returns don't obey R_t+1 = (P_t+1 + D_t+1)/D_t. It's easy to forecast returns like that!

    Of course agents see more than we do. I thought actual reality would have many other variables that forecast both returns and dividend growth, offsetting so that dp doesn't change. Dividend growth really would be forecastable, and returns even more forecastable. I failed to find that with cay in "discount rates," but was still hoping. These ex-post decompositions though suggest that might never be true.

    The bubble observation is very interesting. In population of course, a model with constant expected returns and constant expected dividend growth would show that dp forecasts neither returns nor dividend growth, and is driven entirely by forecasts of rho^k dp_t+k. I think the point of the paper is that in "short" samples, one may well mistakenly see return forecastability that is not there. Since under a bubble dp has a unit root, all sorts of bad small sample properties of the regressions may be expected.

    Still, it falls in the category of you can't really reject that the moon isn't made of green cheese sorts of explanations. Our point estimates are, it's all expected returns, none dividend growth, and none bubble term. One might not be able to reject the view that it's all dividend growth or all bubble, but our point estimates are where they are.

    Thanks again for the great comment and pointing me to these interesting papers

  6. Dear John,

    How do you reconcile the results from this blog with the results from a recent paper in the RFS called "What Drives Stock Price Movements?" (

    They show that stocks DO have an important expected cashflow component (dividends, if you will) to them.


  7. Before I Bitly this to the absolutely everything about absolutely nothing folder, an observation and a question.

    Anyone who has ever been in an American courtroom knows that shareholder have absolutely no real protection from corporate officers, directors, or the Carl I's of this World and that over some period of time most all shareholder value will be stolen (Buffett, the exception, doesn't pay dividends, as I recall).

    Thus, the result seems obvious to me.

    If that is the case, nice math and programming skills, but where can you go with this?

    1. I'm completely confused as to the connection between your observation and the result. Which result do you mean?

  8. A better question would be how did John Davidson find this blog? lol...

  9. I was really hoping for an answer on my question.

    1. I haven't read the paper yet. It looks interesting, will report back when I get a chance. If anyone else reads it and reconciles it with the above graph let me know

  10. Seems to me this is what your charts show---cheating

  11. Dear Mr. Cochrane, thank you for your time. I have just a small minor question: You use a log-linear approximated model of Campbell-Shiller, but still in The Dog That Didn't Bark you show results in levels. Why do you do this? Or why do other researchers do this? Why don't you stick to solely using log returns on log dividend yields? Kind regards

  12. I'm not sure I follow. It seems to assume that the betas calculated within the original model remain static (1920 - 2010). What happens when the beta's are then applied to data moving forward 2011 - 2022? Does the model still show that price to dividend predict these returns?


Comments are welcome. Keep it short, polite, and on topic.

Thanks to a few abusers I am now moderating comments. I welcome thoughtful disagreement. I will block comments with insulting or abusive language. I'm also blocking totally inane comments. Try to make some sense. I am much more likely to allow critical comments if you have the honesty and courage to use your real name.