Predictability of the Tehran Exchange Divedend and Price Index Using a Combined Machine Learning Approach: Market Efficiency Analysis and Importance of Influential Variables

Document Type : Research Paper

Authors

1 MSc Student, Department of Algorithms and Computation, Faculty of Engineering Sciences, College of Engineering, University of Tehran, Tehran, Iran.

2 MSc., Department of Finance, Faculty of Accounting and Finance, College of Management, University of Tehran, Tehran, Iran.

3 Assistant Prof., Department of Industrial Management, Faculty of Humanities, Meybod University, Meybod, Iran.

4 Assistant Prof., Department of Algorithms and Computation, Faculty of Engineering Sciences, College of Engineering, University of Tehran, Tehran, Iran.

Abstract

Objective
This study aims to evaluate the efficiency of the Iranian capital market and examine the capability of hybrid machine learning models in predicting the direction of the Tehran Exchange Dividend and Price Index (TEDPIX). Additionally, the study seeks to assess the importance of influential factors affecting predictability and explainability within machine learning models.
 
Methods
To analyze market efficiency at weak and semi-strong levels, data concerning the Tehran Exchange Divedend and Price Index from five years (2019 to 2023) is utilized. The proposed combined model includes an Extreme Gradient Boosting (XGBoost) model that is enhanced through hyperparameter optimization via a Genetic Algorithm (GA). The performance of this combined model is statistically compared against other machine learning algorithms, including XGBoost, Random Forest, Support Vector Machine, and Logistic Regression. Furthermore, to enhance the model's explainability and analyze the importance of input variables in predicting the index direction, the SHAP (Shapley Additive Explanations) method is employed.
 
Results
The results indicate that the XGBoost-GA model outperforms other comparative models statistically, achieving an accuracy of 84% in predicting the direction of the Tehran Exchange Divedend and Price Index. A comparison of the results across different levels of market efficiency indicates that, at the semi-strong level, incorporating fundamental variables into the predictive model enhances forecasting accuracy. This finding reflects the influence of fundamental information on predicting the direction of the index and, consequently, suggests the presence of market inefficiency at this level. Additionally, at the weak-form level, the machine learning model based on technical data outperformed the random model, indicating the existence of market inefficiency at this level as well. Moreover, the model's explainability analysis, using SHAP, showed that the impact of variables on predicting the index direction varies based on the type of input data. In the purely technical model, factors related to price behavior and short-term fluctuations, such as the Relative Strength Index (RSI), trading volume, and moving average divergence and convergence, played a key role. In contrast, in the combined model that includes both fundamental and technical data, besides technical variables, factors such as real and legal liquidity influx, gold prices, and company financial indicators such as Return on Assets (RoA) and Return on Equity (RoE) significantly influenced predictions.
 
Conclusion
The results of the study demonstrate that the Iranian capital market is inefficient at both weak and semi-strong levels, and the predictability of the Index is achievable using both technical and fundamental data. The analysis of input variable significance indicates that certain technical, fundamental, and macroeconomic indices play a more crucial role in predicting market behavior, which can contribute to more informed investment decisions and a better understanding of machine learning models' behavior in forecasting financial time series.

Keywords

Main Subjects

References

 
Abbasi, E., Mohammadimoghadam, F. & Ghalibafasl, H. (2024). The Relationship between Financial Leverage and Financial Performance: Emphasizing Financial Distress and Currency Crisis in the Tehran Stock Exchange. Financial Research Journal, 26(4), 880-903. (in Persian)
Athey, S. & Imbens, G. W. (2019). Machine learning methods economists should know about. Annual Review of Economics, 11, 685–725.
Athey, S. & Imbens, G. W. (2021). The econometrics of randomized experiments. Annual Review of Economics, 13, 451–483.
Bergmeir, C., Hyndman, R. J. & Benítez, J. M. (2023). Cross-validation for time series: A systematic review. Expert Systems with Applications, 220, 119567.
Berrar, D. (2019). Cross-validation.
Bussmann, N., Giudici, P., Marinelli, D. & Papenbrock, J. (2021). Explainable machine learning in credit risk management. Computational Economics, 57, 203- 216.‏
Cao, Y., Gedikli, F. & Vrancx, P. (2024). Advances and challenges in financial time series forecasting with machine learning. Artificial Intelligence in Finance, 100088.
Chen, T. & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 785-794.
Chollet, F. (2021). Deep learning with Python (2nd ed.). Manning Publications.
Deng, S., Huang, X., Zhu, Y., Su, Z., Fu, Z. & Shimada, T. (2023). Stock index direction forecasting using an explainable eXtreme Gradient Boosting and investor sentiments. The North American Journal of Economics and Finance, 64, 101848.
Dimitriadou, A., Gogas, P., Papadimitriou, T. & Plakandaras, V. (2018). Oil market efficiency under a machine learning perspective. Forecasting, 1(1), 157-168.
Etsalakis, A. & Valavanis, G. (2009). Predicting financial indices using neural networks: Nonlinear machine learning models outperform traditional approaches. Journal of Financial Forecasting and Machine Learning, 5(2), 112–130.
Fama, E. F. & French, K. R. (2020). Comparing cross-section and time-series factor models. The Review of Financial Studies, 33(5), 1891–1926.
Gu, S., Kelly, B. & Xiu, D. (2020). Empirical asset pricing via machine learning. The Review of Financial Studies, 33(5), 2223-2273.
Hassan, M. K., Rahman, M. M. & Kim, S. (2023). Evaluating classification performance in stock trend prediction: A comparative analysis of confusion matrix metrics. Expert Systems with Applications, 215, 119231.
Heidari, M. & Amiri, H. (2022). Inspecting the Predictive Power of Artificial Intelligence Models in Predicting the Stock Price Trend in Tehran Stock Exchange. Financial Research Journal, 24(4), 602-623. (in Persian)
Li, S. & Zhang, X. (2019). Research on orthopedic auxiliary classification and prediction model based on XGBoost algorithm. Neural Computing and Applications, 32, 1971–1979.
Lo, A. W. & MacKinlay, A. C. (1988). Stock market prices do not follow random walks: Evidence from a simple specification test. The Review of Financial Studies, 1(1), 41–66.
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., ... & Lee, S.-I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56-67.
Mohebbi, H. & Zare Mehrjardi, A. (2019). Investigating the Effect of Proactive market Orientation on Creating Market Changes and its Consequences in the tile and ceramic industry. Commercial Surveys, 17(98), 98-113.
Molnar, C. (2025). Interpretable machine learning: A guide for making black box models explainable (3rd ed.).
Raei, R., Vahdati, M., Mohebbi, H. & Heydari Delooei, A. (2025). Interpreting Forecast the Return of the Price Index of Manufacturing Industries in the Tehran Stock Exchange Using Explainable Ensemble Learning. Financial Management Perspective, 14(48), 55-78. (in Persian)
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206-215.
Shobayo, O., Adeyemi-Longe, S., Popoola, O. & Okoyeigbo, O. (2025). A Comparative Analysis of Machine Learning and Deep Learning Techniques for Accurate Market Price Forecasting. Analytics, 4(1), 5.
Sivanandam, S. N. & Deepa, S. N. (2008). Introduction to genetic algorithms. Springer.
Smith, J., Johnson, R. & Lee, H. (2023). Enhancing model robustness in financial market predictions through cross-validation techniques. Journal of Financial Data Science, 15(2), 45–60.
Sohrabi, M., Mirberg Kar, S. M., Chirani, E. & Kherdiar, S. (2023). Comparison of different machine learning models in predicting stock market indices. Financial Engineering and Securities Management Journal, 14(56), 19–36. (in Persian)
Štrumbelj, E., & Kononenko, I. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3), 647-665.
Thaler, R. H. (2016). Behavioral economics: Past, present, and future. American Economic Review, 106(7), 1577–1600.
Yang, C., Abedin, M. Z., Zhang, H., Weng, F. & Hajek, P. (2025). An interpretable system for predicting the impact of COVID-19 government interventions on stock market sectors. Annals of Operations Research, 347(2), 1031-1058.
Yuan, X., Yuan, J., Jiang, T. & Ain, Q. U. (2020). Integrated long-term stock selection models based on feature selection and machine learning algorithms for China stock market. IEEE Access, 8, 22672- 22685.‏
Zhang, Y., Li, X. & Wang, J. (2024). Deep learning and market efficiency: Evidence from high-frequency trading. Journal of Financial Economics, 151(3), 102–125.
Zhong, X. & Enke, D. (2019). Predicting the daily return direction of the stock market using hybrid machine learning algorithms. Financial Innovation, 5(1), 4.
Zoghi, S., Raei, R. & Falahpor, S. (2022). Presenting a market direction prediction model for gold coin trades in Iran’s Commodity Exchange market using Long Short-Term Memory (LSTM) algorithm. Financial Engineering and Portfolio Management, 13(53), 34-53. (in Persian)
Financial Research Journal
Volume 28, Issue 2
2026
Pages 464-493

Files

Share

How to cite

Statistics