Wall Street Quants Shouldn't Confuse Luck With Skill

Johannes Kepler is remembered for his seminal contributions to astronomy. However, like most 16th-century astronomers, he supported himself in part by making astrological predictions. His mentor, Michael Mäestlin, taught him to always prophesy disaster. If something bad happens, you’re celebrated for being right. If not, you’re celebrated for preventing catastrophe. Making optimistic predictions either makes you look foolish if bad things happen or be forgotten if nothing bad happens.

Which brings us to February 2016, when Rob Arnott and his colleagues at investment firm Research Affiliates, or RA, published a paper titled “How Can ‘Smart Beta’ Go Horribly Wrong?” This month the team declared victory with “Revisiting Our ‘Horribly Wrong’ Paper: That Was Then, This Is Now.” For investors, “smart beta” means getting exposure to a market selectively rather than simply buying all assets in that market. Popular factors are size (buy stocks with small market capitalization), value (buy stocks with low ratios of price to value metrics like book value or cash flow) and momentum (buy assets that have gone up in price recently). The term “factor investing” is similar except managers also short assets with the opposite characteristics, such as buying stocks with small capitalization and shorting stocks with large capitalization.

The 2016 paper claimed that four specific factors—size, quality, momentum and low beta—had very low value characteristics relative to history. In other words, to get exposure to those factors you had to buy stocks with poor value characteristics, such as high price to book value ratios, and that in similar periods in the past those factors had underperformed. The impression given by the paper’s title and some of its rhetoric was that factors would crash in the near future, but the actual recommendations were more modest: investors should readjust expectations for lower factor returns from non-value factors and increase emphasis on value.

The victory claim is based on this chart. The horizontal axis shows the over (positive numbers) or under (negative numbers) valuation of five factors in four markets in 2016 versus their performance from March 2016 to September 2022. The dotted line suggests overvaluation in 2016 led to negative returns over the subsequent 6.5 years.

The first thing to notice is the pattern is entirely driven by the low beta factor. Here is the chart with that factor removed. Now there is no obvious trend, so the claim of a general factor breakdown is false. Moreover, the lowest performance is from the value factor—the one that RA suggested investors overweight. Finally, while some factors in some markets would have disappointed investors over the period, none can be described as horribly wrong.

To delve a bit deeper into factor performance, I’m going to switch from RA’s factors to Fama-French factors. These are the academic gold standard, with transparent and simple methodology. Dartmouth Professor Kenneth French publishes their values regularly on his website. Three of the RA factors have direct analogs in Fama-French: size, value and momentum. All three basically broke even over the full seven years from the beginning of 2016 to the end of 2022. Momentum might have been said to crash in 2017, down 20%, but it recovered rapidly. Value and size had early pandemic lows that might be called crash levels, but they also recovered quickly, and RA cannot claim credit for having predicted the pandemic.

The two Fama-French factors that RA omitted enjoyed steady strong performance: conservative up 19% and robust up 42%. For the two non-Fama-French factors used by RA, I want to compare them with transparent factors published regularly. These come from AQR Capital Management, a firm that has been critical of the 2016 paper. (I worked at AQR for 10 years. Following some fresh criticism from AQR co-founder Cliff Asness, Arnott wrote in an email to Bloomberg News last week that he is proud of the 2016 paper.) I don’t do this to suggest AQR factors are better than RA’s, only to demonstrate that details in factor construction make a great difference in analysis. In these kinds of debates it’s best for everyone to use the same public data with disclosed methodology.

The AQR quality factor was up 29% over the period, and the AQR low-beta factor was up 38%. So, the poor performance of RA’s low-beta factor was due to its construction, while the good performance of the quality factor is confirmed by both versions.

A larger issue with the 2023 paper is that it evaluates factors by their total return. Factors are not meant to be held in isolation, but to be added to market portfolios. Their value is as much in reducing risk as in increasing return. An investor 100% in the stock market from 2016 to 2022 got a 12.2% average annual return with 16.8% annualized volatility, for a Sharpe ratio (average annual return above Treasury bills divided by annualized volatility) of 0.56. An investor 65% in the stock market with 5% in each of the five Fama-French factors plus the two AQR factors comparable to RA’s, got a 9.2% annual return with 10.9% volatility, for a 0.60 Sharpe. This is a disappointing result compared with average history for factor investing—a small improvement in Sharpe ratio—but hardly a crash or horribly wrong.

The main takeaways from the 2016 paper—the predicted factor crash and the advice to overweight value—were flawed. The larger point that investors should time factors based on valuations is not supported by the evidence of the last seven years but has not been definitively disproven. In fact, there is mild evidence that modest tactical shifts in factor allocation can help a little. The only 2016 prediction that stands up is the RA low-beta factor, but not low-beta factors in general, would lose money.

RA now recommends reversing the 2016 advice and taking advantage of undervalued factors. I have no idea if this will prove to a good or bad bet, but the outcome of the 2016 predictions don’t give me any confidence. And RA may be making an error in ignoring Mäestlin’s ancient wisdom. 

Aaron Brown is a former managing director and head of financial market research at AQR Capital Management. He is author of The Poker Face of Wall Street. He may have a stake in the areas he writes about.

This article was provided by Bloomberg News.