A Systematic Literature Review: A Comparison Of Available Approaches In Chatbot And Dialogue Manager Development

Arya Jayavardhana; Samuel Ady Sanjaya

doi:10.46729/ijstm.v4i6.983

Arya Jayavardhana Department of Information Systems, Faculty of Engineering and Informatics, Universitas Multimedia Nusantara, Tangerang Selatan, Banten, Indonesia
Samuel Ady Sanjaya Department of Information Systems, Faculty of Engineering and Informatics, Universitas Multimedia Nusantara, Tangerang Selatan, Banten, Indonesia

DOI: https://doi.org/10.46729/ijstm.v4i6.983

Keywords: ANN, chatbot, dialogue manager, DQN, LSTM and RNN

Abstract

The present study reviewed a number of articles chosen from a screening and selecting
process on the various different methods that can be used in the context of chatbot
development and dialogue managers. Since chatbots have seen a significant rise in
popularity and have played an important role in helping humans complete daily tasks,
this systematic literature review (SLR) aims to act as a guidance for future research.
During the process of analyzing and extracting data from the 13 articles chosen, it has
been identified that Artificial Neural Network (ANN), Ensemble Learning, Recurrent
Neural Network (RNN), and Long-Short Term Memory (LSTM) is among some of the
most popular algorithms used for developing a chatbot. Where all of these algorithms
are suitable for each unique use case where it offers different advantages when
implemented. Other than that, dialogue managers lean more towards the field of Deep
Reinforcement Learning (DRL), where Deep Q-Networks (DQN) and its variants such
as Double Deep-Q Networks (DDQN) and DDQN with Personalized Experience Replay
(DDQN-PER) is commonly used. All these variants have different averages on episodic
reward and dialogue length, along with different training time needed which indicates
the computational power needed. This SLR aims to identify the methods that can be
used and identify the best proven method to be applied in future research.

Downloads

Download data is not yet available.

References

M. Allouch, A. Azaria, and R. Azoulay, “Conversational Agents: Goals, Technologies, Vision and Challenges,”

Sensors, vol. 21, no. 24, p. 8448, Dec. 2021, doi: 10.3390/s21248448.

B. Mott, J. Lester, and K. Branting, “Conversational Agents,” 2004. doi: 10.1201/9780203507223.ch10.

A. Joseph and N. G. Eapen, “Conversational Agents and Chatbots,” in Machine Learning and Deep Learning in

Natural Language Processing, Boca Raton: CRC Press, 2023, pp. 17–52. doi: 10.1201/9781003296126-4.

H. N. Io and C. B. Lee, “Chatbots and conversational agents: A bibliometric analysis,” in 2017 IEEE

International Conference on Industrial Engineering and Engineering Management (IEEM), Dec. 2017, pp. 215–

doi: 10.1109/IEEM.2017.8289883.

A. S. Miner, A. Milstein, S. Schueller, R. Hegde, C. Mangurian, and E. Linos, “Smartphone-Based

Conversational Agents and Responses to Questions About Mental Health, Interpersonal Violence, and Physical

Health,” JAMA Intern. Med., vol. 176, no. 5, p. 619, May 2016, doi: 10.1001/jamainternmed.2016.0400.

J. Cassell, “Embodied conversational agents: Representation and intelligence in user interfaces,” AI Mag., vol.

, no. 4, pp. 67–83, 2001.

D.W.Massaro, M. M. Cohen, S. Daniel, and R. A. Cole, “Developing and Evaluating conversational Agents,” in

Human Performance and Ergonomics, Elsevier, 1999, pp. 173–194. doi: 10.1016/B978-012322735-5/50008-7.

B. Liu and C. Mei, “Lifelong Knowledge Learning in Rule-based Dialogue Systems,” Nov. 2020, doi:

48550/arXiv.2011.09811.

J. Singh, M. H. Joesph, and K. B. A. Jabbar, “Rule-based chabot for student enquiries,” J. Phys. Conf. Ser., vol.

, no. 1, p. 012060, May 2019, doi: 10.1088/1742-6596/1228/1/012060.

S. A. Thorat and V. Jadhav, “A Review on Implementation Issues of Rule-based Chatbot Systems,” SSRN

Electron. J., 2020, doi: 10.2139/ssrn.3567047.

B. E. Nishimoto and A. H. Reali Costa, “Dialogue Management with Deep Reinforcement Learning: Balancing

Exploration and Exploitation,” in 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), Oct. 2019,

pp. 449–454. doi: 10.1109/BRACIS.2019.00085.

M. J. Page et al., “The PRISMA 2020 statement: an updated guideline for reporting systematic reviews,” Syst.

Rev., vol. 10, no. 1, p. 89, Dec. 2021, doi: 10.1186/s13643-021-01626-4.

J. Yang, Y.-L. Chen, L. Y. Por, and C. S. Ku, “A Systematic Literature Review of Information Security in

Chatbots,” Appl. Sci., vol. 13, no. 11, p. 6355, May 2023, doi: 10.3390/app13116355.

B. Kitchenham and P. Brereton, “A systematic review of systematic review process research in software

engineering,” Inf. Softw. Technol., vol. 55, no. 12, pp. 2049–2075, Dec. 2013, doi: 10.1016/j.infsof.2013.07.010.

S. Ayanouz, B. A. Abdelhakim, and M. Benhmed, “A Smart Chatbot Architecture based NLP and Machine

Learning for Health Care Assistance,” in Proceedings of the 3rd International Conference on Networking,

Information Systems & Security, Mar. 2020, pp. 1–6. doi: 10.1145/3386723.3387897.

S. Dey and D. Shukla, “Analytical study on use of AI techniques in tourism sector for smarter customer

experience management,” in 2020 International Conference on Computer Science, Engineering and Applications

(ICCSEA), Mar. 2020, pp. 1–5. doi: 10.1109/ICCSEA49143.2020.9132925.

International Journal of Science, Technology & Management ISSN: 2722 - 4015

http://ijstm.inarah.co.id

A. S. Sreelakshmi, S. B. Abhinaya, A. Nair, and S. Jaya Nirmala, “A Question Answering and Quiz Generation

Chatbot for Education,” in 2019 Grace Hopper Celebration India (GHCI), Nov. 2019, pp. 1–6. doi:

1109/GHCI47972.2019.9071832.

N. I. Mohd Rahim, N. A. Iahad, A. F. Yusof, and M. A. Al-Sharafi, “AI-Based Chatbots Adoption Model for

Higher-Education Institutions: A Hybrid PLS-SEM-Neural Network Modelling Approach,” Sustainability, vol.

, no. 19, p. 12726, Oct. 2022, doi: 10.3390/su141912726.

H. Cuayáhuitl et al., “Ensemble-based deep reinforcement learning for chatbots,” Neurocomputing, vol. 366, pp.

–130, Nov. 2019, doi: 10.1016/j.neucom.2019.08.007.

M. Bali, S. Mohanty, D. S. Chatterjee, M. Sarma, and R. Puravankara, “Diabot: A Predictive Medical Chatbot

using Ensemble Learning,” Int. J. Recent Technol. Eng., vol. 8, no. 2, pp. 6334–6340, Jul. 2019, doi:

35940/ijrte.B2196.078219.

Sapna et al., “Recommendence and Fashionsence,” in Proceedings of the ACM India Joint International

Conference on Data Science and Management of Data, Jan. 2019, pp. 256–259. doi: 10.1145/3297001.3297035.

F. Patel, R. Thakore, I. Nandwani, and S. K. Bharti, “Combating Depression in Students using an Intelligent

ChatBot: A Cognitive Behavioral Therapy,” in 2019 IEEE 16th India Council International Conference

(INDICON), Dec. 2019, pp. 1–4. doi: 10.1109/INDICON47234.2019.9030346.

M.-C. Lee, S.-Y. Chiang, S.-C. Yeh, and T.-F. Wen, “Study on emotion recognition and companion Chatbot

using deep neural network,” Multimed. Tools Appl., vol. 79, no. 27–28, pp. 19629–19657, Jul. 2020, doi:

1007/s11042-020-08841-6.

Y. M. Çetinkaya, İ. H. Toroslu, and H. Davulcu, “Developing a Twitter bot that can join a discussion using stateof-the-art architectures,” Soc. Netw. Anal. Min., vol. 10, no. 1, p. 51, Dec. 2020, doi: 10.1007/s13278-020-

-4.

B. Arora, D. S. Chaudhary, M. Satsangi, M. Yadav, L. Singh, and P. S. Sudhish, “Agribot: A Natural Language

Generative Neural Networks Engine for Agricultural Applications,” in 2020 International Conference on

Contemporary Computing and Applications (IC3A), Feb. 2020, pp. 28–33. doi:

1109/IC3A48958.2020.233263.

Y.-T. Wan, C.-C. Chiu, K.-W. Liang, and P.-C. Chang, “Midoriko Chatbot: LSTM-Based Emotional 3D

Avatar,” in 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), Oct. 2019, pp. 937–940. doi:

1109/GCCE46687.2019.9015303.

K. Pathak and A. Arya, “A Metaphorical Study Of Variants Of Recurrent Neural Network Models For A

Context Learning Chatbot,” in 2019 4th International Conference on Information Systems and Computer

Networks (ISCON), Nov. 2019, pp. 768–772. doi: 10.1109/ISCON47742.2019.9036167.

T. Saha, D. Gupta, S. Saha, and P. Bhattacharyya, “Reinforcement Learning Based Dialogue Management

Strategy,” 2018, pp. 359–372. doi: 10.1007/978-3-030-04182-3_32.

G. Xu, H. Lee, M.-W. Koo, and J. Seo, “Optimizing Policy via Deep Reinforcement Learning for Dialogue

Management,” in 2018 IEEE International Conference on Big Data and Smart Computing (BigComp), Jan.

, pp. 582–589. doi: 10.1109/BigComp.2018.00101.

A. Tsantekidis, N. Passalis, and A. Tefas, “Deep reinforcement learning,” in Deep Learning for Robot

Perception and Cognition, Elsevier, 2022, pp. 117–129. doi: 10.1016/B978-0-32-385787-1.00011-7.

P. A. Olujimi and A. Ade-Ibijola, “NLP techniques for automating responses to customer queries: a systematic

review,” Discov. Artif. Intell., vol. 3, no. 1, p. 20, May 2023, doi: 10.1007/s44163-023-00065-5