Can AI Predict Future Stock Returns?

Machine learning is a field of inquiry devoted to understanding and building methods that leverage data to improve performance without human intervention. It is a part of artificial intelligence. Machine-learning algorithms build a model based on sample data, known as training data, to make predictions or decisions without being explicitly programmed to do so. These algorithms are used in a wide variety of applications, such as in medicine, email filtering, speech recognition, and computer vision, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.

While algorithmic trading has been widely applied for a long time to investing, AI could also be used to make decisions in the earlier stages of portfolio formation. The potential advantages of AI include:

• Superior computational power to analyze mass data in a short period of time.
• Avoidance of cognitive biases to which humans are susceptible; AI is more rational.

Can AI replace humans at predicting stock returns or enable them to make better predictions?

AI vs. Human Analysts

Sean Cao, Wei Jiang, Junbo Wang, and Baozhong Yang, authors of “From Man vs. Machine to Man + Machine: The Art and AI of Stock Analyses,” published in the October 2024 issue of the Journal of Financial Economics, examined how AI performs compared with human analysts in predicting stock returns. Their goal was to determine:

• Under what circumstances do human analysts retain their advantage over AI?
• What is the impact of combining human analysts with AI on stock prediction accuracy?
• What are the implications of these findings for the broader application of AI in skilled professions and decision-making processes?

The authors built their own AI model to predict 12-month stock returns (inferred from 12-month target prices) and compared them with analyst forecasts made at the same time on the same stocks. They collected firm-level, industry-level, and macroeconomic variables, as well as textual information from firms’ disclosures, news, and social media (updated to right before the time of an analyst forecast), as inputs, deliberately excluding information from analyst forecasts themselves so that the AI model did not benefit from analyst insights. Their sample of analyst forecasts was built from the Thomson Reuters Institutional Brokers Estimate System analyst database. After merging IBES with CRSP and Compustat data, their final sample consists of 1,153,565 12-month target price forecasts on 6,315 firms issued by 11,890 analysts from 861 brokerage firms, and 5,885,063 first-quarter to fourth-quarter earnings predictions on 8,062 firms issued by 14,363 analysts from 926 brokerage firms. Their model spanned the period 2001-18. The authors found:

• An AI analyst trained to digest corporate disclosures, industry trends, and macroeconomic indicators surpasses most analysts (54.5%) in stock return predictions. The machine’s advantage could arise from either its superior ability to process information or its immunity from predictable human biases due to incentives or psychological traits.
• Relative to analysts, the AI model was able to generate superior returns, or alpha, on the order of 50 to 72 basis points monthly, statistically significant at the 1% level in almost all cases.
• The AI model comfortably beat the analysts in the low-skill quantiles and was virtually even with analysts (analyst beat ratio of 49.3%) who demonstrated superior performance in each of the past five years, an excellence only achieved by just 7.3% of all analysts.
• Macro variables and firm returns contributed the most (27.6% and 24.4%, respectively) to the AI model’s prowess, followed by firm characteristic variables (22%) and textual information (9.3%)—highlighting the importance of qualitative information. Information from earnings had the lowest share (2%).
• Humans beat AI when institutional knowledge is crucial—for example, involving intangible assets and financial distress. For example, human analysts perform better when predicting the returns for smaller and less liquid firms. They also perform better in the presence of intangibles, in industries or companies experiencing rapid changes or high competitive dynamics, and in cases when firms are facing higher distress risk or undergoing significant financial stress.
• AI wins when information is transparent but voluminous.
• After adding analyst forecasts to the information set of the machine-learning models underlying their AI analyst, the resulting model outperformed 54.8% of the forecasts of the AI-only model, while also reducing extreme errors.
• Analysts and the AI model were about equally likely to make extreme errors (9.3% and 7.8%, respectively, using the 90th percentile threshold)—the combined model avoided approximately 90% of the extreme errors made by human analysts and 40% of those made by AI alone.
• The combined model leverages the complementary strengths of both humans and AI. AI excels in processing large volumes of data and identifying patterns, while human analysts add nuanced understanding and contextual insights, resulting in a more robust and reliable forecasting tool.
• Analysts catch up with machines after “alternative data” becomes available if their employers build AI capabilities.
• Documented synergies between humans and machines inform how humans can leverage their advantage for better adaptation to the growing AI prowess.

Augmented by AI

Perhaps the most interesting finding was that while the model outperformed analysts in predicting returns, when it came to predicting earnings, analysts beat machine with a probability 69.2%. However, the combined analysts and AI model outperformed 55% of the analyst forecasts.

Their findings led the authors to conclude: “Overall, this study supports the hypothesis that analyst capabilities could be augmented by AI and, more importantly, that analysts’ work possesses incremental value to and synergies with AI modeling, especially in unusual and fast-evolving situations.”

They added: “While the future of AI remains uncertain, the parts of human skills that are incremental to AI, as we document, allow for promising Man + Machine collaboration and augmentation.”

Empirical Research on the Performance of AI Funds

There is one study we can review on the live performance of AI-powered mutual funds.

Rui Chen and Jinjuan Ren, authors of the study “Do AI-Powered Mutual Funds Perform Better?” published in the August 2022 issue of Finance Research Letters, evaluated the performance of AI-powered mutual funds. Their data sample was from the CRSP Survivor-Bias-Free US Mutual Fund Database and covered the 26-month period of November 2017-December 2019. They labeled AI-powered funds as those that use machine-learning technologies to actively select stocks in portfolio choice; quantitative funds as those that use fixed rules and numerical methods to generate computer-driven models and make investment decisions; and discretionary funds as those traditional funds that select stocks and make investment decisions mainly through human judgment. Following is a summary of their findings:

• The performance of AI-powered mutual funds was statistically indistinguishable from the aggregate market in 25 out of the 26 months in the sample period.
• AI-powered funds did not generate significant risk-adjusted returns and showed only marginally superior stock-selection skills (only by equal-weight) and no market-timing skills.
• AI-powered funds did outperform their human-managed peers because of lower turnover—31% versus 72%—resulting in lower transactions costs and marginally superior stock-selection skills.
• AI-powered funds held fewer stocks (149 versus 197)—thus, their portfolios were more concentrated.
• AI-powered funds avoided some prevalent behavioral biases (such as the disposition effect).

Investor Takeaways

Cao, Jiang, Wang, and Yang showed that stock predictions can be improved by integrating the strengths of AI and human analysts, leading to better outcomes than relying on either alone. They demonstrated that AI and human analysts complement each other—AI excels in processing large volumes of data and identifying patterns, while humans provide contextual understanding, intuition, and nuanced insights. Recognizing these synergies helps in designing systems where both AI and human inputs are maximized. However, there isn’t any evidence to date that AI-powered funds outperform on a risk-adjusted basis.

Perhaps the most important takeaway is that AI models help to avoid human bias, making forecasts more accurate—which should lead to markets becoming more efficient, reducing the opportunity to generate alpha through security selection.

Larry Swedroe is the author or co-author of 18 books on investing, including his latest Enrich Your Future.