{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T23:11:34Z","timestamp":1782515494839,"version":"3.54.5"},"reference-count":36,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In this paper, we propose a novel approach to optimize parameters for strategies in automated trading systems. Based on the framework of Reinforcement learning, our work includes the development of a learning environment, state representation, reward function, and learning algorithm for the cryptocurrency market. Considering two simple objective functions, cumulative return and Sharpe ratio, the results showed that Deep Reinforcement Learning approach with Double Deep Q-Network setting and the Bayesian Optimization approach can provide positive average returns. Among the settings being studied, Double Deep Q-Network setting with Sharpe ratio as reward function is the best Q-learning trading system. With a daily trading goal, the system shows outperformed results in terms of cumulative return, volatility and execution time when compared with the Bayesian Optimization approach. This helps traders to make quick and efficient decisions with the latest information from the market. In long-term trading, Bayesian Optimization is a method of parameter optimization that brings higher profits. Deep Reinforcement Learning provides solutions to the high-dimensional problem of Bayesian Optimization in upcoming studies such as optimizing portfolios with multiple assets and diverse trading strategies.<\/jats:p>","DOI":"10.3390\/a16010023","type":"journal-article","created":{"date-parts":[[2023,1,2]],"date-time":"2023-01-02T02:44:03Z","timestamp":1672627443000},"page":"23","update-policy":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Optimizing Automated Trading Systems with Deep Reinforcement Learning"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0003-2093-9093","authenticated-orcid":false,"given":"Minh","family":"Tran","sequence":"first","affiliation":[{"name":"John von Neumann Institute, Vietnam National University, Ho Chi Minh City 70000, Vietnam"},{"name":"CHArt Laboratory EA 4004, EPHE, PSL Research University, 75014 Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Duc","family":"Pham-Hi","sequence":"additional","affiliation":[{"name":"John von Neumann Institute, Vietnam National University, Ho Chi Minh City 70000, Vietnam"},{"name":"Financial Engineering Department, ECE Paris Graduate School of Engineering, 75015 Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Bui","sequence":"additional","affiliation":[{"name":"CHArt Laboratory EA 4004, EPHE, PSL Research University, 75014 Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,1]]},"reference":[{"key":"ref_1","unstructured":"Chan, E.P. (2021). Quantitative Trading: How to Build Your Own Algorithmic Trading Business, John Wiley & Sons."},{"key":"ref_2","unstructured":"Xiong, Z., Liu, X.Y., Zhong, S., Yang, H., and Walid, A. (2018). Practical deep reinforcement learning approach for stock trading. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lucarelli, G., and Borrotti, M. (2019, January 24\u201326). A deep reinforcement learning approach for automated cryptocurrency trading. Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations, Crete, Greece.","DOI":"10.1007\/978-3-030-19823-7_20"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, Y., Liu, Q., Zhao, H., Pan, Z., and Liu, C. (2020, January 7\u201312). Adaptive quantitative trading: An imitative deep reinforcement learning approach. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i02.5587"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1016\/j.neucom.2021.04.005","article-title":"A parallel multi-module deep reinforcement learning algorithm for stock trading","volume":"449","author":"Ma","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_6","unstructured":"Pricope, T.V. (2021). Deep reinforcement learning in quantitative algorithmic trading: A review. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Millea, A. (2021). Deep reinforcement learning for trading\u2014A critical survey. Data, 6.","DOI":"10.20944\/preprints202111.0044.v1"},{"key":"ref_8","first-page":"41","article-title":"Multi-objective optimization of technical stock market indicators using gas","volume":"68","author":"Fayek","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_9","first-page":"2951","article-title":"Practical bayesian optimization of machine learning algorithms","volume":"25","author":"Snoek","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1016\/j.jebo.2004.07.022","article-title":"Technical trading in the Santa Fe Institute artificial stock market revisited","volume":"61","author":"Ehrentreich","year":"2006","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bigiotti, A., and Navarra, A. (2018, January 19\u201321). Optimizing automated trading systems. Proceedings of the The 2018 International Conference on Digital Science, Budva, Montenegro.","DOI":"10.1007\/978-3-030-02351-5_30"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"10","DOI":"10.3905\/jfds.2019.1.021","article-title":"Machine learning in asset management\u2014Part 1: Portfolio construction\u2014Trading strategies","volume":"2","author":"Snow","year":"2020","journal-title":"J. Financ. Data Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pardo, R. (2011). The Evaluation and Optimization of Trading Strategies, John Wiley & Sons.","DOI":"10.1002\/9781119196969"},{"key":"ref_14","first-page":"26","article-title":"Hyperparameter optimization for machine learning models based on Bayesian optimization","volume":"17","author":"Wu","year":"2019","journal-title":"J. Electron. Sci. Technol."},{"key":"ref_15","unstructured":"Bergstra, J., and Bengio, Y. (2012). Random search for hyper-parameter optimization. J. Mach. Learn. Res., 13."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","article-title":"A simplex method for function minimization","volume":"7","author":"Nelder","year":"1965","journal-title":"Comput. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1126\/science.220.4598.671","article-title":"Optimization by simulated annealing","volume":"220","author":"Kirkpatrick","year":"1983","journal-title":"Science"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Powell, M.J. (1994). A direct search optimization method that models the objective and constraint functions by linear interpolation. Advances in Optimization and Numerical Analysis, Springer.","DOI":"10.1007\/978-94-015-8330-5_4"},{"key":"ref_19","unstructured":"Fu, W., Nair, V., and Menzies, T. (2016). Why is differential evolution better than grid search for tuning defect predictors?. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/BF00120661","article-title":"Bayesian methods in global optimization","volume":"1","year":"1991","journal-title":"J. Glob. Optim."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1023\/A:1012771025575","article-title":"A taxonomy of global optimization methods based on response surfaces","volume":"21","author":"Jones","year":"2001","journal-title":"J. Glob. Optim."},{"key":"ref_22","unstructured":"Ni, J., Cao, L., and Zhang, C. (2008). Evolutionary optimization of trading strategies. Applications of Data Mining in E-Business and Finance, IOS Press."},{"key":"ref_23","first-page":"1","article-title":"Applications of data mining in e-business and finance: Introduction","volume":"177","year":"2008","journal-title":"Appl. Data Min. E-Bus. Financ."},{"key":"ref_24","unstructured":"Jomaa, H.S., Grabocka, J., and Schmidt-Thieme, L. (2019). Hyp-rl: Hyperparameter optimization by reinforcement learning. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107119","DOI":"10.1016\/j.knosys.2021.107119","article-title":"Technical analysis strategy optimization using a machine learning approach in stock market indices","volume":"225","author":"Ayala","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Fern\u00e1ndez-Blanco, P., Bodas-Sagi, D.J., Soltero, F.J., and Hidalgo, J.I. (2008, January 10\u201314). Technical market indicators optimization using evolutionary algorithms. Proceedings of the 10th Annual Conference Companion on Genetic and Evolutionary Computation, Lille, France.","DOI":"10.1145\/1388969.1388989"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"114632","DOI":"10.1016\/j.eswa.2021.114632","article-title":"An application of deep reinforcement learning to algorithmic trading","volume":"173","author":"Ernst","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.asoc.2013.09.011","article-title":"The trading on the mutual funds by gene expression programming with Sortino ratio","volume":"15","author":"Chen","year":"2014","journal-title":"Appl. Soft Comput."},{"key":"ref_29","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_30","unstructured":"Wilder, J.W. (1978). New Concepts in Technical Trading Systems, Trend Research."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"83105","DOI":"10.1109\/ACCESS.2021.3085085","article-title":"Evaluation of deep learning models for multi-step ahead time series prediction","volume":"9","author":"Chandra","year":"2021","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gen, M., and Cheng, R. (1999). Genetic Algorithms and Engineering Optimization, John Wiley & Sons.","DOI":"10.1002\/9780470172261"},{"key":"ref_34","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the Icml, Atlanta, GA, USA."},{"key":"ref_35","first-page":"26","article-title":"Neural networks for machine learning","volume":"138","author":"Tieleman","year":"2012","journal-title":"Coursera (Lecture-Rmsprop)"},{"key":"ref_36","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/www.mdpi.com\/1999-4893\/16\/1\/23\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T17:55:16Z","timestamp":1760118916000},"score":1,"resource":{"primary":{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/www.mdpi.com\/1999-4893\/16\/1\/23"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,1]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["a16010023"],"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.3390\/a16010023","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,1]]}}}