It is a long standing issue that many patients encounter difficulties in adhering to their prescribed medication regimens. Interventions that are designed to improve medication adherence often come in a variety of heterogeneous approaches, many of which tend to have limited or negligible efficacy [1]. In order to foster shared decision making among future clinicians, many universities have incorporated the training of empathetic responding in their curricula, with the aim of distilling “emotional meaning” to the patient-clinician relationship [2,3]. However, there is a limitation in terms of medical and healthcare students understanding the intricate narratives surrounding patients’ lived experiences of illnesses and medications. While many perceive themselves as clinical experts due to their educational qualifications, they may struggle to grasp the patients’ medication experiences from a perspective that is outside what they learn in school. Healthcare education needs to go beyond empathic responding and train our students – our future healthcare professionals – in narrative competence, which is defined as by Rita Charon as the proficiency to absorb, interpret and respond to stories [4].

Just like the character Mantis, from Guardians of the Galaxy, who is able to sense and influence emotions, some therapists are well-versed in getting in touch with patients’ feelings. However, pharmacists generally lack the knowledge and skills to understand or facilitate patient narrative sharing. The ideal scenario involves pharmacy students engaging in real-life therapy sessions guided by therapists, encountering patients’ narratives, and subsequently reflecting on these experiences. However, this approach can only be done as part of experiential learning, which depends on the limitations of time, manpower and clinical patient load. Furthermore, variations in patient encounters also imply that the learning experiences of each student may not be standardized, which renders it difficult to be adopted on a larger scale.

Clinical Art Therapy as a Form of Patient Narrative

Clinical Art Therapy (CATx) is defined by the British Association of Art Therapists as a “form of psychotherapy that uses visual and tactile media as a means of expression and communication” [5]. During therapy, many different art materials can be used as media, such as drawings, paintings, writings and visual art, among others. CATx serves to foster self-esteem and self-awareness, cultivate emotional resilience, enhance social skills, reduce conflicts and distress, and even improve cognitive and sensorimotor functions [6]. As such, CATx can be used for patients with anxiety, depression, dementia, cancer and cognitive impairment, since these patients have difficulty expressing themselves in words [7]. CATx provides an indirect and safe way to connect the patients with others, thus is a useful therapeutic method to help them open up and share their opinions, feelings and experiences to complement the information obtained through conventional diagnostic tests [7].

During COVID-19, it was reported that public exhibitions of art therapy expressions were able to promote a sense of connection and understanding among patients and clinicians, as well as foster a sense of community catharsis in support of the patient’s feelings of isolation and depression during the pandemic [8]. Although a balance was needed between converging the privacy of the patient and publicity of an exhibition gallery, these exhibitions were able to provide a therapeutic value by enabling patients to experience a soothing, central and organized focus for their lives [9]. In addition, the freshness of the original intent of the patient could be preserved through the exhibition displays, which could help in terms of isolating the narrative, yet provide an effective and unbiased way for the art therapist and clinician to gather data about the patient [10].

Leveraging on Generative AI and the Metaverse to Understand Patients’ Medication Stories

Generative artificial intelligence (Gen AI) applications, such as OpenAI’s ChatGPT (text generator) and DALL-E (art image generator), are potential solutions that can address the challenges of standardization of training in narrative competence. The advantage of Gen AI is its ability to create new and potentially innovative content based on what it has learnt previously from existing content, which spans text, images, and audio [11,12]. Hence, it can potentially capture patient stories of their “lived experiences” of their medical conditions and medication therapies in a more holistic manner through multimodal personifications of their medication journeys. Through art therapy, patients can humanize their lived experiences of their medications in the form of character portraits. These character portraits can offer an alternative learning approach to therapy sessions, allowing pharmacy and healthcare students to be exposed to patients’ narratives of their lived experiences through an engaging means. Watch examples of how a Hero and Villain character are brought to live below!

Enter MAGIC – Our Metaverse Art Gallery of Image Chronicles

MAGIC is a metaverse art gallery that depicts patients’ symbolic medication experiences through Hero-Villain portrayals. By combining clinical art therapy with Gen AI and the metaverse, MAGIC aims to improve the pharmacist-patient relationship through relatable medication experiences. Patients’ personifications of their medications of choice were crafted into character portraits of either Heroes or Villains using Gen AI image generators (e.g. Nightcafe) and AI text generators (e.g. ChatGPT), after which AI sound/voice generators (e.g. HeyGen) were used to bring these Hero/Villain characters to life! These portraits were then mounted in an in-house created Metaverse Art Gallery that was split into 3 realms, each with a unique theme. Have a look at the trailer below to see what MAGIC looks like!

Publications, Presentations & Invited Talks:

• Ho J, Toh P, Yap KY. A digital health humanities approach to the three-body problem in healthcare education: Balancing the science, art and tech with M.A.G.I.C. Stud Health Technol Inform 2024; 318: 72-77.
• Yap KY, Ho J, Toh PST. Development of a Metaverse Art Gallery of Image Chronicles (MAGIC) for healthcare education: A digital health humanities approach to patients’ medication experiences. Information 2024; 15(8): 431.
• Yap KY, Toh P, Ho J. (9-10 Sep 2023) Heroes versus Villains: Humanizing Medication Therapies through A Metaverse Art Gallery of Image Chronicles for Education (MAGIC-E). Paper presented as e-poster and physical poster presentation (abstr ER-000154) at the 32^nd Singapore Pharmacy Congress, Grand Copthorne Waterfront Hotel, Singapore.
• Yap KY, Toh P, Ho J. (23-24 Nov 2023) Development of a Generative Artificial Intelligence Metaverse Art Gallery of Image Chronicles (MAGIC): Personalizing Patients’ Medication Journeys as Heroes and Villains. Paper presented as e-poster presentation at the AI Health Summit 2023, W Singapore Hotel – Sentosa Cove, Singapore. Organized by the SingHealth Duke-National University of Singapore (DUKE-NUS).
• Yap KY, Toh P. (3 Apr 2024) “I am Mantis” – Transforming Medication Journeys through the Power of Art, AI and the Metaverse. Invited talk as part of the NTU Medical Humanities Seminar Series. Organized by the School of Humanities, College of Humanities, Arts, and Social Sciences, Nanyang Technological University, Singapore. [DOI: 10.13140/RG.2.2.14438.43847]

References:

1. Nieuwlaat R, Wilczynski N, Navarro T, et al. Interventions for enhancing medication adherence. Cochrane Database Syst Rev. 2014;(11). [DOI: 10.1002/14651858.cd000011.pub4]
2. Rogers CR. The necessary and sufficient conditions of therapeutic personality change. J Consult Psychol. 1957;21(2):95. [DOI: 10.1037/h0045357]
3. Rogers CR. The therapeutic relationship: Recent theory and research. Aust J Psychol. 1965;17(2):95-108. [DOI: 10.1080/00049536508255531]
4. Charon R. Narrative Medicine: A Model for Empathy, Reflection, Profession, and Trust. JAMA. 2001;286(15):1897-1902. [DOI:10.1001/jama.286.15.1897]
5. British Association of Art Therapists. Art therapy. Available at: https://baat.org/art-therapy/. 2023. Accessed 19 Apr 2023.
6. American Art Therapy Association. About art therapy. Available at: https://arttherapy.org/about-art-therapy/. 2022. Accessed 19 Apr 2023.
7. Hu J, Zhang J, Hu L, Yu H, Xu J. Art therapy: A complementary treatment for mental disorders. Front Psychol. 2021;12:686005. [DOI: 10.3389%2Ffpsyg.2021.686005]
8. Devine S. Therapeutic impact of public art exhibits during COVID-19. Art Therapy. 2022;40(1):50-54. [DOI: 10.1080/07421656.2022.2131359]
9. Vick RM. Ethics on exhibit. Art Therapy. 2011;28(4):152-158. [DOI: 10.1080/07421656.2011.622698]
10. Prokofiev F. Allowing the Artwork to Speak: The use of a visual display as research method in a retrospective study of four years’ artwork in art therapy with a four-year old boy. Art Therapy Online. 2013;4(1):318;1-5. [DOI: 10.25602/GOLD.atol.v4i1.318]
11. Shahriar S. GAN computers generate arts? a survey on visual arts, music, and literary text generation using generative adversarial network. Displays. 2022;73:102237. [DOI: 10.48550/arXiv.2108.03857]
12. Cao Y, Li S, Liu Y, et al. A comprehensive survey of AI-generated content (AIGC): A history of generative AI from GAN to ChatGPT. ArXiv. 2023: abs/2303.04226. [DOI: 10.48550/arXiv.2303.04226]

MAGIC Team Members:

• Clin A/Prof Kevin Yap (Division of Pharmacy, Singapore General Hospital)
• Ms Phylaine Toh (Art Therapy & Music Therapy Unit, Singapore General Hospital)
• Dr Yap Kai Zhen (Department of Pharmacy and Pharmaceutical Sciences, National University of Singapore)

In collaboration with:

• Ms Jayen Ho (Final Year Project student, 2022-2023), School of Applied Science, Temasek Polytechnic, Singapore
• Mr Hud Bin Mohammed Helmi (Sci-Phi student, 2023-2024), Department of Pharmacy and Pharmaceutical Sciences, National University of Singapore
• Ms Eunice Toh Si Ying (Sci-Phi student, 2023-2024), Department of Pharmacy and Pharmaceutical Sciences, National University of Singapore