Email this article 
Hide
Show all
LLM-Based Agents: Can They Transform Organizational Realities? An Experimental Verification Attempt
Abstract
Theoretical background: Large language models (LLMs) have transformed artificial intelligence (AI) and are increasingly considered for use in organizational management. However, they have limitations, especially in generalization and performance on topics they were not trained on. Despite these constraints, LLMs show potential to support multiple management-related activities, particularly when deployed as autonomous or collaborative agents.
Purpose of the article: The article aims to explain how LLMs can be applied to organizational management despite their current limitations. It focuses on the emerging concept of LLM-based autonomous agents that can reason, plan, and act in an environment. It also highlights the value of multi-agent collaboration coordinated by an orchestrator.
Research methods: The paper primarily takes a conceptual and exploratory approach by synthesizing existing ideas and emerging research on LLMs in management contexts. It discusses potential architectures, such as individual agents and orchestrated multi agent systems, rather than reporting a controlled experiment. The article identifies key organizational areas where these approaches could be applied.
Main findings: LLM-based agents can potentially perform complex organizational tasks by combining reasoning, planning, and action capabilities. Collaborative multi-agent solutions managed by an orchestrator may improve effectiveness compared to single-agent setups. The article concludes that more empirical research is needed to validate these solutions in real organizations and to assess their effects on efficiency and workplace adaptation.
Keywords
Full Text:
PDFReferences
Blei, D.M., Ng, A.Y., & Jordan, M.I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.
Chase, H. (2023). Langchain. https://docs.langchain.com/docs/
Eisenhardt, K.M. (1989). Agency theory: An assessment and review. The Academy of Management Review, 14(1), 57–74.
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N., Wiest, O., & Zhang, X. (2024). Large Language Model-based Multi-Agents: A Survey of Progress and Challenges. exarXiv:2402.01680v1 [cs.CL].
Händler, T. (2023). Balancing autonomy and alignment: A multi-dimensional taxonomy for autonomous LLM powered multi-agent architectures. arXiv preprint arxiv:2310.03659. https://doi.org/10.48550/arXiv.2310.03659
Hong, S., Zheng, X., Chen, J., Cheng, Y., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., Zhou, L., Ran, C., Schmidhuber, J., Wu, C., Xiao, L., Zhuge, M., & Wang, J.(2023). MetaGPT: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352.
Hussein, A., Gaber, M.M., Elyan, E., & Jayne, C.(2017). Imitation learning: A survey of learning methods. ACM Computing Surveys, 50(2), 1–35.
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38. https://doi.org/10.1145/3571730
Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., & McHardy, R. (2023). Challenges and applications of large language models. arXiv preprint arXiv:2307.10169.
Korzynski, P., Mazurek, G., Altmann, A., Ejdys, J., Kazlauskaite, R., Paliszkiewicz, J., Wach, K., & Ziemba, E. (2023). Generative artificial intelligence as a new context for management theories: analysis of ChatGPT. Central European Management Journal, 31(1), 3–13. https://doi.org/10.1108/CEMJ-02-2023-0091
Lan, L.L., & Heracleous, L. (2010). Rethinking agency theory: the view from law. Academy of Management Review, 35(2), 294–314.
Li, G., Hammoud, H.A.A.K., Itani, H., Khizbullin, D., & Ghanem, B. (2023). CAMEL: Communicative agents for “mind” exploration of large scale language model society. arXiv preprint arXiv:2303.17760.
Liu, Z., Zhang, Y., Li, P., Liu, Y., & Yang, D.(2023). Dynamic LLM-agent network: An LLM-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170.
Ma, Q., Xue, X., Zhou, D., Yu, X., Liu, D., Zhang, X., Zhao, Z., Shen, Y., Ji, P., Li, J., Wang, G., & Ma, W. (2024). Computational experiments meet large language model based agents: A survey and perspective. arXiv:2402.00262v1.
Mo, Y., Kontonatsios, G., & Ananiadou, S. (2015). Supporting systematic reviews using LDA-based document representations. Systematic Reviews, 4(172), 1–12. https://doi.org/10.1186/s13643-015-0117-0
Padgham, L., & Winikoff, M. (2005). Developing Intelligent Agent Systems: A Practical Guide. John Wiley & Sons.
Park, J.S., O’Brien, J.O., Cai, C.J., Morris, M.R., Liang, P., &Bernstein, M.S. (2023). Generative agents: Interactive simulacra of human behaviour. In The 36th Annual ACM Symposium on User Interface Software and Technology (UIST ‘23). Association for Computing Machinery, NY.
Qin, Y.S., & Hu, Y. (2023). Tool learning with foundation models. CoRR, abs/2304.08354.
Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation-augmentation paradox. Academy of Management Review, 46, 192–210.
Ross, S., & Morrison, G. (2004). Experimental research methods. In D.J. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (pp. 1021–1043). Lawrence Erlbaum Associates.
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., Scialom, & Toolformer, T. (2023). Language models can teach themselves to use tools. arXiv preprint. arXiv:2302.04761.
Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines, Journal of Business Research, 104, 333–339.
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W.X., Wei, Z., & Wen, J.-R. (2023). A survey on large language model-based autonomous agents. CoRR, abs/2308.11432.
Wei, J., Shuster, K., Szlam, A., Weston, J., Urbanek, J., & Komeili, M. (2023). Multi-party chat: Conversational agents in group settings with humans and models. CoRR, abs/2304.13835. https://doi.org/10.48550/arXiv.2304.13835
Wooldridge, M.J., & Jennings, N.R. (1995). Intelligent agents: Theory and practice. Knowledge Engineering Review, 10(2), 115–152.
Wu, Q., Bansal, G., Zhang, J., Wu, J., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A.H., White, R.W., Burger, D., & Wang, C. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv:2308.08155v2 [cs.AI].
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., Zheng, R., Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C., Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R., Cheng, W., Zhang, Q., Qin, W., Zheng, Y., Qiu, X., Huang, X., & Gui, T. (2023). The rise and potential of large language model based agents: A survey. arXiv:2309.07864.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). React: Synergizing reasoning and acting in language models. arXiv:2210.03629v3 [cs.CL].
Zhang, C., K. Yang, S. Hu, Wang, Z., Li, G., Sun, Y., Zhang, C., Zhang, Z., Liu, A., Zhu, S.-C., Chang, X., Zhang, J., Yin, F., Liang, Y., & Yang, Y. (2023). ProAgent: Building proactive cooperative AI with large language models. CoRR, abs/2308.11339.
Zhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., Wang, X., Qiao, Y., Zhang, Z., & Dai, J. (2023). Ghost in the Minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. arXiv preprint arXiv:2305.17144.
DOI: http://dx.doi.org/10.17951/h.2026.60.2.27-42
Date of publication: 2026-08-12 23:46:07
Date of submission: 2024-07-05 12:43:24
Statistics
Indicators
Refbacks
- There are currently no refbacks.
Copyright (c) 2026 Mariusz Hofman
This work is licensed under a Creative Commons Attribution 4.0 International License.