If you need more proof that index funds are the best way to invest in the stock market - here it is

By Mark R. Rank

Darts, monkeys and Wall Street: Your stock market success might just be dumb luck

A Russian circus monkey named Lusha was able to select an investment portfolio that beat 94% of the country's investment funds.

Trying to outguess the stock market has been an active sport for decades, if not centuries. Determining when and where the market and individual stocks are heading in the short run has been the gold standard for many professional and amateur investors alike. For anyone who can do so on a consistent basis, considerable money is to be had. Yet the vast majority end up losing. Why? The reason can be found in the random factor.

It turns out that flipping a coin or throwing darts to predict short-term movement of stocks often produces results on par with professional investors. Economist and author Burton Malkiel is credited with popularizing the phrase "a random walk." As he writes, "A random walk is one in which future steps or directions cannot be predicted on the basis of past history. When the term is applied to the stock market, it means that short-run changes in stock prices are unpredictable." The original analogy pertained to a "drunken man staggering around an empty field. He is not rational, but he's not predictable either."

To illustrate this principle, a number of amusing examples can be found. During the 1990s, the Wall Street Journal ran a dartboard contest pitting the throwing of four darts against a professional stock investor each month. The four darts were thrown at financial pages hung on the wall in order to determine which stocks would be included in the portfolio, while the professional used their expertise to pick four stocks for their portfolio. Each month the darts were pitted against a new expert. After a decade of darts versus investors, no clear winner was declared.

Even more impressive, a Russian circus monkey named Lusha was able to select an investment portfolio that beat 94% of the country's investment funds. In 2009, Lusha was presented with 30 blocks representing different companies, and was asked to pick eight of the blocks for investment. Her portfolio grew by almost 300% over the next year.

Such a performance is nothing new. In 1933, Alfred Cowles published a study analyzing a number of printed financial services and every purchase and sale made by 20 leading fire insurance companies over a four year period. His conclusion was that the "best of a series of random forecasts made by drawing cards from an appropriate deck was just as good as the best of a series of actual forecasts, and that the results achieved by the insurance companies 'could have been achieved through a purely random selection of stocks.'"

Lucky strike

These examples illustrate in a light-hearted way the fact that there is considerable randomness in how the stock market behaves on a short-term basis. Research has shown that investors are much better off with an indexed fund rather than an actively managed portfolio. Yet the belief in being able to predict short-term gains and losses persists.

As Malkiel observes: "Human nature likes order; people find it hard to accept the notion of randomness. No matter what the laws of chance might tell us, we search for patterns among random events wherever they might occur - not only in the stock market but even in interpreting sporting phenomena."

You might object to this line of argument by pointing to a select number of investors who have beaten the market for several consecutive years running and ask, "Isn't this evidence that some investors with superior skill can actually come out ahead?"

Here again, the explanation is likely to be the random factor. Given the large numbers of professional investors, by simple chance some of them will have a lucky streak of doing well. For instance, if 1,000 people flip a coin 10 times, it's likely that a few will get eight heads or tails in a row simply by chance. Similarly, it is quite possible, and in fact likely, that some professional investors will experience a hot or cold streak of six or seven years with respect to their choices. As Malkiel writes, "With large numbers of investment managers, chance will - and does - explain some extraordinary performances."

For example, suppose we want to know whether an investor can pick on a weekly basis whether the stock market will go up or down at the end of the week. In this experiment, we designate five weeks of correct choosing as our measure of a savvy investor. Simply by pure chance, at least 1 in 32 individuals will probably be correct. This has nothing to do with skill or knowledge, but with the fact each week there's a 50/50 chance of getting the prediction correct. Multiplying this across the course of five weeks (0.5 x 0.5 x 0.5 x 0.5 x 0.5) puts the odds at 0.03125 - or 1/32. Consequently, regardless of skill, one in 32 individuals will likely have the correct predictions for each of those five weeks.

Another way of seeing this is with a particular scam that has been used to hoodwink amateur investors into handing over their money. It goes like this: I obtain a list of 1,024 potential investors and send them an email or text saying that I have developed a new foolproof way of predicting short-term changes in the stock market and in specific stocks.

But I realize that you are probably skeptical about such a claim. Therefore, in order to prove my ability to accurately forecast, each Monday morning for the next 10 weeks I will provide you with my prediction based on an exclusive algorithm as to whether a particular stock will be up or down at the closing bell on that Friday.

On the first Monday, I send out emails to 512 individuals giving my prediction that the stock will be up for that week, while the other 512 individuals receive emails that the stock will be down. For the next week, I send out 512 emails to the group that had the correct prediction, and for 256 of those individuals my email says that the stock will be up, and for the other 256 I predict the stock will be down. I continue this process through to week nine, where I now have two individuals who believe I have been right on all nine weeks. One gets the prediction the stock will be up, the other that the stock will be down.

I then contact the person in which my predictions have been correct all 10 weeks in a row. My email says that you can now clearly see that the algorithm is completely accurate, and that by having this advanced information, you will achieve an enormous windfall in the market. If you would like my stock prediction for this coming week, it will only cost you $5,000. Of course, this person does not know that it took 1,023 people to arrive at this point.

The moral of this story? The random walk theory suggests that investing in the stock market is best done through an index fund. Such a fund will mirror the overall patterns in the market. Because the long-term trajectory of the stock market has been upward, the investor whose portfolio is indexed to say, the Standard and Poor's 500 Index SPX, should do fine over the long horizon. But for the investor who chooses to play the market, buyer beware.

Mark R. Rank is the Herbert S. Hadley Professor of Social Welfare at Washington University in St. Louis. He is the author of "The Random Factor: How Chance and Luck Profoundly Shape Our Lives and the World Around Us" (University of California Press, 2024).

Plus: Index funds are ruining the stock market

Also read: Shouldn't more people say 'to hell with the adviser' - get some index funds and call it a day? What am I missing here?

-Mark R. Rank

This content was created by MarketWatch, which is operated by Dow Jones & Co. MarketWatch is published independently from Dow Jones Newswires and The Wall Street Journal.

(END) Dow Jones Newswires

06-18-24 1401ET

Copyright (c) 2024 Dow Jones & Company, Inc.