Ensemble Strategy for Algorithmic Trading Using Deep Reinforcement Learning

Objective
Trading strategies are crucial in investment companies as they guide decision-making processes and optimize returns. However, designing a profitable strategy within the complex and dynamic stock market environment poses significant challenges. The intricacies of market behavior and the multitude of influencing factors necessitate advanced modelling techniques. The growing availability of extensive data sets and increased computational power have facilitated the use of agent-based models, which have become essential tools for understanding economic and financial systems. The Tehran Stock Exchange often requires rapid adaptation due to severe volatility, regulatory changes, and sudden economic shifts. The choice to implement an ensemble strategy, consisting of deep reinforcement learning agents, arises from the unique challenges and opportunities of the Tehran Stock Exchange. Unlike traditional supervised learning models that make predictions solely based on historical data, agent-based models offer an adaptive approach that can respond to market changes in real-time. Another reason for selecting this strategy is its capacity to perform complex portfolio management operations. Combining multiple deep reinforcement learning agents, each with distinct strengths, the ensemble approach can leverage diverse strategies to optimize trades, manage risk, and enhance decision-making across different market conditions. Therefore, this research proposes an Ensemble strategy for algorithmic trading, leveraging deep reinforcement learning to optimize stock trading strategies that maximize returns while minimizing investment risk.
 
Methods
This study implements an ensemble trading strategy by modelling the stock market and employing five distinct deep reinforcement learning algorithms. This ensemble strategy synthesizes each algorithm's strengths and best features, making it adaptable to various market conditions. To achieve this, Data from stocks listed in the price index of the top 50 companies on the Tehran Stock Exchange are utilized to train and test these algorithms. The performance of the trading agent, using different reinforcement learning algorithms, is subsequently evaluated and compared against the benchmark index and a traditional minimum-variance portfolio allocation strategy. The comparative analysis helps thoroughly assess the effectiveness of the ensemble approach in real-world trading scenarios.
 
Results
From June 29, 2022, to January 20, 2024, the research implemented various trading models to gauge their performance. The ensemble strategy demonstrated a significant annual return of 47.13%, a cumulative return of 78.47%, and a risk-adjusted return of 1.56. These results indicate a superior performance over individual deep reinforcement learning algorithms, the benchmark price index of the 50 Tehran Stock Exchange companies, and the traditional minimum-variance portfolio allocation strategy. Among the individual algorithms, the Soft Actor-Critic (SAC) algorithm recorded the highest returns, with an annual return of 29.89% and a cumulative return of 47.89%. However, its higher annual volatility of 44.22% suggested weaker risk management. Conversely, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm achieved a more balanced outcome with a risk-adjusted return of 0.92, highlighting its effective risk management alongside respectable returns. Therefore, the findings indicate that the ensemble strategy can effectively create a trading strategy that outperforms deep reinforcement learning algorithms, the price index of the top 50 companies on the Tehran Stock Exchange, and the minimum variance portfolio allocation strategy.
 
Conclusion
The Ensemble strategy offers a robust and adaptive framework for dynamic stock portfolio management by combining the strengths of multiple deep reinforcement learning algorithms. It is a reliable trading strategy that enhances returns and effectively manages investment risks. Future improvements to this strategy also involve further integrating fundamental and macroeconomic indicators to refine its predictive accuracy. Additionally, incorporating legal and regulatory constraints into the stock market modeling process, as well as considering market participants beyond investors, could improve the realism and performance of the model. This holistic approach would provide a more comprehensive understanding of market dynamics, potentially leading to more stable and robust trading outcomes.