Skip to main content
SpringerLink
Log in

Discrete-Event Simulation-Based Q-Learning Algorithm Applied to Financial Leverage Effect

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Discrete-event modeling and simulation and machine learning are two frameworks suited for system of systems modeling which when combined can give a powerful tool for system optimization and decision making. One of the less explored application domains is finance, where this combination can propose a driven tool to investor. This paper presents a discrete-event specification as a universal framework to implement a machine learning algorithm into a modular and hierarchical environment. This approach has been validated on a financial leverage effect based on a Markov decision-making policy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Barbieri E, Capocchi L, Santucci J. Devs modeling and simulation based on markov decision process of financial leverage effect in the eu development programs. In: 2017 Winter simulation conference (WSC), 2017; pp 4558–4560. https://doi.org/10.1109/WSC.2017.8248203

  2. Bernard P, Zeigler HP, Kim TG. Theory of modeling and simulation. 2nd ed. Cambridge: Academic Press; 2000.

    Google Scholar 

  3. Bertsekas DP. Dynamic programming: deterministic and stochastic models. Upper Saddle River: Prentice-Hall Inc; 1987.

    MATH  Google Scholar 

  4. Bromuri S. Dynamic heuristic acceleration of linearly approximated sarsa(\(\lambda\)): using ant colony optimization to learn heuristics dynamically. J Heuristic. 2019;25(6):901–32. https://doi.org/10.1007/s10732-019-09408-x.

    Article  Google Scholar 

  5. Capocchi L, Santucci JF, Poggi B, Nicolai C. DEVSimPy: A collaborative python software for modeling and simulation of DEVS systems. In: Proceedings of 20th IEEE international workshops on enabling technologies, 2011; pp 170–175. https://doi.org/10.1109/WETICE.2011.31

  6. Even-Dar E, Mansour Y. Learning rates for q-learning. J Mach Learn Res 2004; 5:1–25. http://dl.acm.org/citation.cfm?id=1005332.1005333

  7. Floyd MW, Wainer GA. Creation of devs models using imitation learning. In: Proceedings of the 2010 summer computer simulation conference, society for computer simulation international, San Diego, CA, USA, SCSC ’10, 2010; pp. 334–341. http://dl.acm.org/citation.cfm?id=1999416.1999459

  8. Hayashi S, Prasasti N, Kanamori K, Ohwada H. Improving behavior prediction accuracy by using machine learning for agent-based simulation. In: Nguyen NT, Trawiński B, Fujita H, Hong TP, editors. Intelligent information and database systems. Heidelberg: Springer; 2016. p. 280–9.

    Chapter  Google Scholar 

  9. Huang W, Nakamori Y, Wang SY. Forecasting stock market movement direction with support vector machine. Comput Oper Res. 2005;32(10):2513–22. https://doi.org/10.1016/j.cor.2004.03.016. http://www.sciencedirect.com/science/article/pii/S0305054804000681, applications of Neural Networks.

  10. Kazem A, Sharifi E, Hussain FK, Saberi M, Hussain OK. Support vector regression with chaos-based firefly algorithm for stock market price forecasting. Applied Soft Computing. 2013;13(2):947–58. https://doi.org/10.1016/j.asoc.2012.09.024. http://www.sciencedirect.com/science/article/pii/S1568494612004449.

  11. Lake BM, Ullman TD, Tenenbaum JB, Gershman SJ. Building machines that learn and think like people. 2016; CoRR arXiv:1604.00289

  12. Lin W, Hu Y, Tsai C. Machine learning in financial crisis prediction: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 2012;42(4):421–436. https://doi.org/10.1109/TSMCC.2011.2170420

  13. Meraji S, Tropper C. A machine learning approach for optimizing parallel logic simulation. In: 2010 39th International Conference on Parallel Processing, 2010; pp 545–554. https://doi.org/10.1109/ICPP.2010.62

  14. Mulvey JM. Machine learning and financial planning. IEEE Potential. 2017;36(6):8–13. https://doi.org/10.1109/MPOT.2017.2737200.

    Article  Google Scholar 

  15. Nielsen NR. Application of artificial intelligence techniques to simulation. Springer, New York, 1991; pp 1–19. https://doi.org/10.1007/978-1-4612-3040-3_1

  16. Patel J, Shah S, Thakkar P, Kotecha K. Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Exp Syst Appl. 2015;42(1):259–68. https://doi.org/10.1016/j.eswa.2014.07.040. http://www.sciencedirect.com/science/article/pii/S0957417414004473.

  17. Puterman ML. Markov Decision processes: discrete stochastic dynamic programming. 1st ed. New York: Wiley; 1994.

    Book  Google Scholar 

  18. Rachelson E, Quesnel G, Garcia F, Fabiani P. A simulation-based approach for solving generalized semi-markov decision processes. In: Proceedings of the 2008 Conference on ECAI 2008: 18th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2008; pp 583–587. http://dl.acm.org/citation.cfm?id=1567281.1567408.

  19. Russell S, Norvig P. Artificial intelligence: a modern approach. 3rd ed. Upper Saddle River: Prentice Hall Press; 2009.

    MATH  Google Scholar 

  20. Saadawi H, Wainer G, Pliego G. Devs execution acceleration with machine learning. In: 2016 Symposium on Theory of Modeling and Simulation (TMS-DEVS), 2016; pp 1–6. https://doi.org/10.23919/TMS.2016.7918816

  21. Seo C, Zeigler BP, Kim D. Devs markov modeling and simulation: formal definition and implementation. In: Proceedings of the theory of modeling and simulation symposium, society for computer simulation international, San Diego, TMS ’18, 2018; pp 1:1–1:12. http://dl.acm.org/citation.cfm?id=3213187.3213188

  22. Sutton RS, Barto AG. Introduction to reinforcement learning. 1st ed. Cambridge: MIT Press; 1998.

    MATH  Google Scholar 

  23. Toma S. Detection and identication methodology for multiple faults in complex systems using discrete-events and neural networks: applied to the wind turbines diagnosis. Theses, University of Corsica, 2014a. https://hal.archives-ouvertes.fr/tel-01141844

  24. Toma S. Detection methodology and identify multiple faults in complex systems from discrete events and neural networks: applications for wind turbines. Theses, Université Pascal Paoli, 2014b. https://tel.archives-ouvertes.fr/tel-01127073

  25. Van Tendeloo Y, Vangheluwe H. The modular architecture of the Python(P)DEVS simulation kernel (WIP). In: Proceedings of the Symposium on Theory of Modeling & Simulation - DEVS Integrative, Society for Computer Simulation International, San Diego, DEVS ’14, 2014; pp 14:1–14:6. http://dl.acm.org/citation.cfm?id=2665008.2665022

  26. Wallis L, Paich M. Integrating artifical intelligence with anylogic simulation. In: 2017 Winter Simulation Conference (WSC), 2017; pp 4449–4449. https://doi.org/10.1109/WSC.2017.8248156

  27. Watkins CJCH. Learning from delayed rewards. PhD thesis, King’s College, Cambridge, 1989.

  28. Yeh CY, Huang CW, Lee SJ. A multiple-kernel support vector regression approach for stock market price forecasting. Exp Syst Appl. 2011;38(3):2177–86. https://doi.org/10.1016/j.eswa.2010.08.004, http://www.sciencedirect.com/science/article/pii/S0957417410007876.

  29. Yu H, Mahmood AR, Sutton RS (2017) On generalized bellman equations and temporal-difference learning. In: Advances in Artificial Intelligence - 30th Canadian Conference on Artificial Intelligence, Canadian AI 2017, Edmonton, AB, Canada, May 16–19, 2017, Proceedings, pp 3–14. https://doi.org/10.1007/978-3-319-57351-9_1

  30. Zeigler BP. Theory of modeling and simulation. Cambridge: Academic Press; 1976.

    MATH  Google Scholar 

  31. Zeigler BP, Sarjoughian HS. System entity structure basics. Guide to modeling and simulation of systems, simulation foundations, methods and applications. London: Springer; 2013. p. 27–37.

    Google Scholar 

  32. Zeigler BP, Muzy A, Kofman E. Theory of Modeling and Simulation, third edition. Academic Press, New York 2019. https://doi.org/10.1016/B978-0-12-813370-5.00003-1

Download references

Author information

Authors and Affiliations

  1. UMR CNRS 6134, University of Corsica, Campus Grimaldi, 20250, Corte, France

    E. Barbieri, L. Capocchi & J. F Santucci

Authors
  1. E. Barbieri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Capocchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. F Santucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. Capocchi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barbieri, E., Capocchi, L. & Santucci, J.F. Discrete-Event Simulation-Based Q-Learning Algorithm Applied to Financial Leverage Effect. SN COMPUT. SCI. 1, 50 (2020). https://doi.org/10.1007/s42979-019-0051-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-019-0051-7

Keywords

Search

Navigation