An Experimental Evaluation of Large Language Models in Supporting the DEX Multi-Criteria Decision-Making Process
Abstract
We experimentally assessed the capabilities of two mainstream artificial intelligence chatbots, ChatGPT and DeepSeek, to support the multi-criteria decision-making process. Specifically, we focused on using the method DEX (Decision EXpert) and investigated their performance in all stages of DEX model development and utilization. The results indicate that these tools may substantially contribute in the difficult stages of collecting and structuring decision criteria, and collecting data about decision alternatives. However, at the current stage of development, the support for the whole multi-criteria decision-making process is still lacking, mainly due to occasionally inconsistent and erroneous execution of methodological steps. To leverage the strengths of both approaches, we also propose a hybrid workflow for DEX model development that begins in the LLM and continues in the specialized DEXiWin software.References
Bana e Costa, C., De Corte, J.-M., Vansnick, J.-C. (2003): MACBETH (Overview of MACBETH multicriteria decision analysis approach). International Journal of Information Technology & Decision Making 11:02, 359–387. https://doi.org/10.1142/S0219622012400068.
Banerjee, S., Agarwal, A., Singla, S. (2024). LLMs will always hallucinate, and we need to live with this. https://doi.org/10.48550/arXiv.2409.05746.
Bansal, A. (2024): Comprehensive Study on LLM Agents. International Journal of All Research Education & Scientific Methods 12(8):2455-6211.
Bohanec, M., Rajkovič, V., Bratko, I., Zupan, B., Žnidaršič, M. (2013): DEX methodology: Three decades of qualitative multi-attribute modelling. Informatica 37, 49–54.
Bohanec, M. (2022): DEX (Decision EXpert): A qualitative hierarchical multi-criteria method. In: Multiple Criteria Decision Making (ed. Kulkarni, A.J.), Studies in Systems, Decision and Control 407, Singapore: Springer, https://doi.org/10.1007/978-981-16-7414-3_3.
Bohanec, M. (2024): DEXiWin: DEX Decision Modeling Software, User’s Manual, Version 1.2. Ljubljana: Institut Jožef Stefan, Delovno poročilo IJS DP-14747. https://dex.ijs.si/dexisuite/dexiwin.html.
Bohanec, M. (2025): Using large language models for creating, modifying and interpreting decision tables in multi-criteria method DEX. CECIIS 2025, 36th Central European Conference on Information and Intelligent Systems, Varaždin, Croatia, 651–658.
Bohanec, M., Rajkovič, U., Rajkovič, V (2025): Utilizing large language models for supporting multi-criteria decision modelling method DEX. IS 2025, Proc. of the 28th International Multiconference Information Society, Volume A, Ljubljana, Slovenia. 19–22.
Bray, T., Paoli, J., Sperberg, C.M., Maler, E., Yergeau, F. (2008): Extensible Markup Language (XML) 1.0 (Fifth Edition). https://www.w3.org/TR/REC-xml/.
Hämäläinen, R.P., Luoma, J., Saarinen, E. (Eds.). (2016): Behavioral Operational Research: A Capabilities Approach. Cham: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-25405-6.
Ishizaka, A., Nemery, P. (2013): Multi-criteria decision analysis: Methods and software. Chichester: Wiley. ISBN: 978-1-119-97407-9.
Kamath, U., Keenan, K., Somers, G., Sorenson, S. (2024): Large Language Models: A Deep Dive: Bridging Theory and Practice. Springer, 506p, ISBN-13 978-3031656460.
Kulkarni, A.J. (Ed.) (2022): Multiple Criteria Decision Making. Studies in Systems, Decision and Control 407, Singapore: Springer, https://doi.org/10.1007/978-981-16-7414-3.
Radovanović, S., Delibašić, B., Vukanović, S. (2024): Combining LLM and DIDEX method to predict Internal Migrations in Serbia. Proc. 24th International Conference on Group Decision and Negotiation & 10th International Conference on Decision Support System Technology (GDN ICDSST 2024), Vol. 1: Technology as a support tool (Eds. S.P. Duarte, P. Zaraté, A. Lobo, B. Delibašić, T. Wachowicz, M.C. Ferreira), University of Porto.
Saaty, T.L., Vargas, L.G. (2012): Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. Springer, US, New York, https://doi.org/10.1007/978-1-4614-3597-6.
Svoboda, I., Lande, D. (2024): Enhancing multi-criteria decision analysis with AI: Integrating analytic hierarchy process and GPT-4 for automated decision support. https://arxiv.org/abs/2402.07404.
Wang, H., Zhang, F., Mu, C. (2025). One for all: A general framework of LLMs-based multi-criteria decision making on human expert level. arXiv e-prints, 2502.15778. https://arxiv.org/abs/2502.15778.
Wang, L., Ma, C., Feng, X. et al. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 186345. https://doi.org/10.1007/s11704-024-40231-1.
Yuan, W., Liu, P., Gallé, M (2024): LLMCRIT: Teaching large language models to use criteria. Findings of the Association for Computational Linguistics: ACL 2024, 7929–7960, https://doi.org/10.18653/v1/2024.findings-acl.472.
DOI:
https://doi.org/10.31449/inf.v49i4.12884Downloads
Published
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
