| 摘要: | 本研究深入探討了在醫療保健環境中，特別是在皮膚科門診，通過解釋非語言線索（如面部表情和語音調）來理解同情心參與的情況。在這項研究中，我們識別並分析了非語言模仿的實例，包括面部、音頻和結合音頻-面部的行為。這使我們能夠對醫生所表現出的同情心參與度進行分類，特別強調區分自發和有意的模仿。通過使用面部情感識別（FER）和語音情感識別（SER）系統進行多模態分析，我們發現醫生經常使用中性面部表情，可能是為了保持情感平衡的互動。此外，我們注意到，在諮詢結束時，快樂的表情有所增加，這意味著這些會議的情感氣氛有所提升。我們的機器學習模型在檢測自發模仿方面表現出了強大的結果，尤其是面部模仿模型，在區分低和高同情心水平方面表現出強大的區別能力。然而，我們最值得注意的發現是有意的模仿：音頻-面部模仿模型，儘管其區分能力略低於面部模型，但在面部表情不可見的情況下非常有效，因此在我們日益面罩化的醫療環境中，這是一種無價的工具。我們還發現，患者年齡和性別等人口因素在所有模型中都起到了重要的作用，因此強調了它們在預測同情心中的重要性。我們的發現為醫療保健環境中非語言模仿、同情心和患者滿意度之間的複雜關係提供了寶貴的洞見，並指出了提高同情心溝通和患者護理的可能途徑。
This research delves into the understanding of empathetic engagement within healthcare settings, specifically in dermatology outpatient clinics, by interpreting non-verbal cues such as facial expressions and tone of voice using machine learning models. In this study, we identified and analyzed instances of non-verbal mimicry, which include facial, audio, and combined audio-facial behaviors. This allows us to classify the degrees of empathetic engagement displayed by physicians, with a special emphasis on distinguishing between spontaneous and intentional mimicry. Using Facial Emotion Recognition (FER) and Speech Emotion Recognition (SER) systems for a multimodal analysis, we found that physicians often employ neutral facial expressions, likely as a strategy to maintain an emotionally balanced interaction. Moreover, we noted an increase in expressions of happiness towards the end of consultations, implying an enhancement in the emotional atmosphere during these sessions. Our machine learning models demonstrated robust results in detecting spontaneous mimicry, particularly with the facial mimicry model, which showed strong discriminative power between low and high empathy levels. However, our most notable finding relates to intentional mimicry: the audio-facial mimicry model, although demonstrating a slightly lower discriminative ability than the facial model, proved extremely effective in situations where facial expressions were not visible, therefore making it an invaluable tool in our increasingly masked healthcare environment. We also found that demographic factors such as patient age and gender played a significant role across all models, thereby underlining their importance in predicting empathy. Our findings present invaluable insights into the intricate relationship between non-verbal mimicry, empathy, and patient satisfaction levels in healthcare settings and point to potential avenues for enhancing empathetic communication and patient care. |
