Complex machine learning models created in the academic world don’t always yield results in the real world. But in 1982, Spyros Makridakis began a series of competitions to observe forecasts in real life and judge their accuracy. Famously, he discovered that statistically sophisticated and complex methods don’t necessarily provide more accurate forecasts than simple ones. Today, this competition continues as the M competitions, with the latest iteration called the M6 Financial Forecasting Competition. We believe that this competition has the potential to play a transformative role in the field of financial market prediction, similar to the impact of the ImageNet competition in the field of computer vision.
Since 2010, ImageNet has hosted the prestigious ImageNet Large Scale Visual Recognition Challenge (ILSVRC), an image classification competition that provides a benchmark dataset and evaluation system for researchers and practitioners to test and compare different image-classification algorithms. A historical moment occurred in 2012, when AlexNet achieved a good ILSVRC classification top-5 error rate of 16%. Breakthroughs and innovations developed in response to the challenging problem presented by the competition led the world to realize that what was once considered an extremely hard problem in computer vision was in fact solvable.
The M competitions are designed to create breakthroughs and innovations in response to challenging, real-world problems. Specifically, M6 is designed to promote advancements in forecasting financial prices and investment-portfolio allocation. The M competition also has a reputation for fostering innovation, as demonstrated by the M4 competition, which resulted in the development of a highly innovative hybrid ES-RNN model.
This year, the OCI Forecasting team demonstrated that by using a limited amount of historical price data from the past 6-8 months in combination with advanced machine learning algorithms on a single compute VM, it was possible to achieve strong performance and secure a second place in the M6 competition. Based on this result, we believe that leveraging the full historical data of financial markets from multiple regions and using state-of-the-art algorithms on large-scale cloud clusters could bring major breakthroughs and challenge the assumptions of the efficient market hypothesis (EMH).
Background for the M6 competition
The four most dangerous words in investing are: “This time it’s different.” – Sir John Templeton
History repeats itself, and investors shouldn’t let themselves be blinded by the hype of the moment and make decisions that ignore historical trends and patterns. To make informed investment decisions, the crucial question to ask is which specific aspect of history is currently being mirrored in the present.
Prediction of financial markets is a challenging task because of their nonstationary and complex nature and EMH. EMH states that financial markets are efficient and that it’s impossible to consistently achieve abnormal returns through the use of any publicly available information, including historical data. Trying to predict financial markets using historical data and models can be difficult because the market is constantly changing and adapting to new information. Additionally, many factors, such as economic, political, and social events, can influence financial markets, making it hard to predict their behavior. But, the use of new and advanced approaches, such as reinforcement learning (RL) with human feedback, trained on massive stock market datasets may change this forever.
In today’s rapidly advancing AI landscape, breakthroughs are breaking well-established domains such as search, and the ML community is eager to get to newer domains. The M6 competition could bring attention to the field of financial-market prediction and potentially challenge the belief in the EMH. Although it might initially shake the notion that markets are always perfectly efficient, after these prediction algorithms become more widely available, it’s likely that the EMH will be reestablished.
How to construct quantitative equity portfolios
Several methods can be used to construct quantitative equity portfolios for a universe of stocks:
- Fundamental analysis analyzes the financial and operational characteristics of individual companies to identify those that are likely to outperform their peers. This analysis might involve evaluating factors such as a company’s financial health, growth prospects, and competitive position.
- Technical analysis uses past price and volume data to identify patterns and trends that might indicate future price movements. Technical analysts use various tools and techniques, such as chart patterns and indicators, to make their predictions.
- Factor investing identifies and targets specific characteristics, or factors, that have been shown to be predictive of future returns. Some common factors used in factor investing are value, momentum, and quality.
- Machine learning uses algorithms and computer models to analyze large amounts of data to identify patterns and make predictions. Machine learning can be used to identify factors that are predictive of future returns and to construct portfolios that are optimized for those factors.
- Risk-based methods involve constructing portfolios that are designed to optimize for specific risk-related objectives, such as maximizing return for a given level of risk or minimizing risk for a given level of return. Techniques such as risk budgeting or risk parity might be used.
These and many other methods can be used to construct quantitative equity portfolios, and the best approach depends on the specific goals and constraints of the portfolio.
The OCI Forecasting team’s M6 competition submission
The OCI Forecasting team decided to respect the spirit of the competition by using only the price data provided by the organizers, instead of incorporating any external or additional data, which is permissible within the rules of the competition.
At first, it might seem that we were at a disadvantage compared to other participants in the competition, who have access to high-quality data and can use more sophisticated methods to construct their portfolios. But good performance can be achieved by using only price data, which is the most fundamental data available. Price data can provide valuable insights into the underlying value and performance of a company. Other ancillary data sources might provide additional information, but they’re ultimately derived from or related to the price data, and it’s possible to achieve good results without them.
The OCI Forecasting team accomplished a significant achievement by securing second place in the Q3 of the M6 competition. Unlike traditional analysts, who prioritize optimizing Ranked Probability Score (RPS) and then Information Ratio (IR), the OCI team approached these objectives separately.
To attain a low RPS, the OCI team used a combination of Bayesian optimization and OCI Forecasting algorithms to calculate the ranked probabilities of all 100 securities. The process involved converting price data to return data, dividing it into deciles based on weekly returns, applying forecasting algorithms, and using Bayesian optimization to determine the 200 parameters that optimize the correlation between RPS and forecasts. This ultimately resulted in the creation of the submission file.
The team achieved high IR by identifying a period in history with a similar market regime, creating multiple portfolios optimized for different tenures through Pareto optimization applied to various portfolio performance objectives, and combining portfolios with the highest consistency from different tenures.
The OCI Forecasting team
The winning competition entrants from Oracle, including myself and Akshay Gupta, are applied scientists in the OCI Forecastingservice, a limited-availability AI service that delivers univariate and multivariate time-series forecasts through statistical, machine learning, and deep learning algorithms. Our service makes it possible for those without data science experience to take advantage of automatic data preprocessing, automated best-model selection, and explainability and confidence intervals, and to predict key business metrics. And of course, we apply our deep research and findings to improve our service whenever we can.