The Evaluation Problem in Cryptocurrency Price Forecasting with Machine Learning and Deep Learning: A Problem-Centric Systematic Review of 48 Studies (2018–2025)

Ansari Saleh Ahmar; Abdul Rahman

doi:10.61255/jeemba.v4i1.1307

Authors

Ansari Saleh Ahmar Universitas Negeri Makassar, Indonesia
Abdul Rahman Universitas Negeri Makassar, Indonesia

DOI:

https://doi.org/10.61255/jeemba.v4i1.1307

Keywords:

Cryptocurrency Forecasting, Deep Learning, Bitcoin, Evaluation Failure Modes, Regime-Stratified Evaluation

Abstract

Purpose – Cryptocurrency price forecasting with machine learning (ML) and deep learning (DL) has produced 48 Scopus-indexed journal articles since 2018, yet the same LSTM architecture applied to Bitcoin daily closing prices yields mean absolute percentage errors ranging from 1.7% to 4.8% across papers in this corpus. This review examines why the literature fails to accumulate knowledge despite growing output and identifies the evaluation practices responsible for that failure.

Design/methodology/approach – A PRISMA 2020 compliant search of Scopus retrieved 48 peer-reviewed English-language articles on ML and DL applications to cryptocurrency price prediction published between 2018 and 2025. All articles were retained after dual-reviewer screening (κ = 0.86) and Mixed Methods Appraisal Tool quality appraisal at the ≥10/16 threshold. Structured data extraction covered architecture type, target coin, forecast horizon, evaluation metric, and train/test split specification.

Finding/Results – Five evaluation failure modes affect 39 of 48 articles: calendar concealment (47.9%), split inconsistency (37.5%), normalisation silence (33.3%), baseline heterogeneity (25.0%), and single-regime evaluation (100%). CNN-LSTM hybrids outperform standalone LSTM in 9 of 12 studies that test both, yet neither this finding nor the 6× Transformer growth ratio can be verified across studies because evaluation conditions are not shared.

Originality/Value – This is the first PRISMA 2020 compliant systematic review of cryptocurrency ML forecasting. It introduces a five-mode evaluation failure taxonomy and proposes a regime-stratified evaluation design prescribing three mandatory calendar-anchored test periods — the 2021 bull run, the 2022 FTX collapse, and the 2024 institutional entry period — as the minimum standard for deployment-relevant performance claims.

Abstract views: 22 ,

PDF downloads: 19

Downloads

Download data is not yet available.

References

Ahamed, S. A., & Ravi, C. (2021). Prediction of cryptocurrency prices using LSTM and ANN. International Journal of Swarm Intelligence and Evolutionary Computation, 10(6), 1–8. doi: 10.4018/IJSIR.2021040102

Altan, A., Karasu, S., & Bekiros, S. (2019). Digital currency forecasting with chaotic meta-heuristic signal processing. Chaos, Solitons and Fractals, 126, 325–336. doi: 10.1016/j.chaos.2019.07.011

Ammer, M. A., & Aldhyani, T. H. H. (2022). Deep learning algorithm to predict cryptocurrency fluctuation prices. Electronics, 11(15), 2349. doi: 10.3390/electronics11152349

Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. doi: 10.1109/TPAMI.2013.50

Farooq, A., Irfan Uddin, M., Adnan, M., Alamer, A., Almutairi, S., & Ullah, Z. (2024). Bidirectional LSTM for cryptocurrency forecasting. Heliyon, 10(22), e40142. doi: 10.1016/j.heliyon.2024.e40142

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press. doi: 10.7551/mitpress/9780262035613.001.0001

Gurgul, V., Lessmann, S., & Härdle, W. K. (2025). Deep learning for intraday cryptocurrency forecasting with order book data. International Journal of Forecasting, 41(2), 601–621. doi: 10.1016/j.ijforecast.2025.02.007

Han, P., Chen, H., Rasool, A., Jiang, Q., & Yang, L. (2025). Decomposition-Transformer hybrid for multi-step cryptocurrency price prediction. Expert Systems with Applications, 265, 125515. doi: 10.1016/j.eswa.2024.125515

Hitam, N. A., & Ismail, A. R. (2018). Comparative performance of ML algorithms for cryptocurrency forecasting. Indonesian Journal of Electrical Engineering and Computer Science, 11(3), 1121–1128. doi: 10.11591/ijeecs.v11.i3.pp1121-1128

Hong, Q. N., Pluye, P., Fabregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, C., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., & Vedel, I. (2018). Mixed Methods Appraisal Tool (MMAT) version 2018. Education for Information, 34(4), 285–291. doi: 10.3233/EFI-180221

Kehinde, T. O., Adedokun, O. J., Joseph, A., Kabir, Y., & Aliyu, K. (2025). Helformer: Attention-based deep learning for cryptocurrency prediction. Journal of Big Data, 12(1), 1–28. doi: 10.1186/s40537-025-01135-4

Ladhari, A., & Boubaker, H. (2024). Fractional integration and deep learning for cryptocurrency forecasting. Forecasting, 6(2), 16. doi: 10.3390/forecast6020016

Lahmiri, S. (2020). Minute-ahead stock return forecasting based on singular spectrum analysis and support vector regression. Applied Mathematics and Computation, 370, 124951. doi: 10.1016/j.amc.2019.124951

Lahmiri, S., & Bekiros, S. (2019). Cryptocurrency forecasting with deep learning chaotic neural networks. Chaos, Solitons and Fractals, 118, 35–40. doi: 10.1016/j.chaos.2018.11.014

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. doi: 10.2307/2529310

Li, Y., Jiang, S., Li, X., & Wang, S. (2022). Hybrid data decomposition-based deep learning for Bitcoin prediction. Financial Innovation, 8, 43. doi: 10.1186/s40854-022-00336-7

Livieris, I. E., Kiriakidou, N., Stavroyiannis, S., & Pintelas, P. (2021). An advanced CNN-LSTM model for cryptocurrency forecasting. Electronics, 10(3), 287. doi: 10.3390/electronics10030287

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4, 1. doi: 10.1186/2046-4053-4-1

Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system. Retrieved from https://bitcoin.org/bitcoin.pdf

Nazareth, N., & Reddy, Y. V. R. (2023). Financial applications of machine learning: A literature review. Expert Systems with Applications, 219, 119640. doi: 10.1016/j.eswa.2023.119640

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Gluud, C., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., … Moher, D. (2021). The PRISMA 2020 statement. BMJ, 372, n71. doi: 10.1136/bmj.n71

Pantachang, K., Tansuchat, R., & Yamaka, W. (2022). Bayesian deep learning for Bitcoin price prediction. Axioms, 11(10), 527. doi: 10.3390/axioms11100527

Pečiulis, T., Ahmad, N., Menegaki, A. N., & Biržinytė, I. (2024). Transformer model for multi-step Bitcoin price forecasting. Forecasting, 6(3), 3114. doi: 10.1002/for.3114

Rafi, M., Mirza, Q. A. K., Sohail, M. I., & Aliasghary, M. (2023). Enhancing cryptocurrency price forecasting accuracy via feature selection. IEEE Access, 11, 62940–62956. doi: 10.1109/ACCESS.2023.3287888

Sebastiao, H., & Godinho, P. (2021). Forecasting and trading cryptocurrencies with machine learning. Financial Innovation, 7, 3. doi: 10.1186/s40854-020-00217-x

Shamshad, H., Ullah, F., Ullah, A., & Kebande, V. R. (2023). Forecasting and trading stable cryptocurrencies with machine learning. IEEE Access, 11, 117369–117387. doi: 10.1109/ACCESS.2023.3327440

Syed, S., Talha, S. M., Iqbal, A., & Ahmad, N. (2024). Ensemble machine learning for Bitcoin direction prediction. AI, 5(4), 136. doi: 10.3390/ai5040136

Wang, M., Braslavski, P., & Ignatov, D. I. (2025). Temporal Fusion Transformer for multi-step cryptocurrency forecasting. Forecasting, 7(3), 48. doi: 10.3390/forecast7030048

Yasir, M., Attique, M., Latif, K., & Chaudhary, G. M. (2023). Deep learning for cryptocurrency forecasting using social media sentiment. Journal of Enterprise Information Management, 36(2), 522–547. doi: 10.1108/JEIM-02-2020-0077

Younas, R., Raza Ur Rehman, H. M., & Choi, G. S. (2025). CNN-GRU with sentiment features for multi-coin cryptocurrency prediction. Journal of Big Data, 12(1), 91. doi: 10.1186/s40537-025-01291-7