Automated Facial Landmark Analysis vs. Manual Coding: Accuracy in Dog Emotional Expression Classification

Abstract

Emotional expression in dogs is central to dog-human interactions. Reliable indicators are essential for interpreting animal emotions; however, their predictive value remains debated. Fireworks, which elicit fear in many dogs, provide a real-world context for examining affective states. We used machine learning to classify firework vs. non-firework situations from dogs behaviour and expressions using two approaches: (1) ethogram-based manual coding of behaviours and (2) automated analysis of facial landmarks. The Random Forest model based on manual coding achieved a high accuracy (0.83) and 1.0 predictive validity, identifying backwards-directed ears and blinking as key fear indicators. The best automated facial landmark model reached up to 0.80 accuracy and 0.77 predictive validity. Manual coding performed better, likely due to richer semantic content and full-body observations. Our findings demonstrate that machine learning can classify canine emotional states from both manually coded and automated methods, offering potential for future automated welfare monitoring. HighlightsO_LIMachine learning approaches classify firework-related fear in dogs reliably C_LIO_LIEar position and blinking are key indicators C_LIO_LIEthogram-based coding yielded stronger predictions than automatically detected facial landmarks C_LIO_LIAutomated landmark analysis shows potential for scalable welfare monitoring C_LI O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=160 SRC="FIGDIR/small/683931v1_ufig1.gif" ALT="Figure 1"> View larger version (48K): org.highwire.dtl.DTLVardef@ec9ab8org.highwire.dtl.DTLVardef@1a1bc0corg.highwire.dtl.DTLVardef@140c1e5org.highwire.dtl.DTLVardef@79d23e_HPS_FORMAT_FIGEXP M_FIG Graphical abstract C_FIG Subject areasEmotional expression {middle dot} Behavioural classification {middle dot} Machine learning {middle dot} Facial landmarks {middle dot} Facial expressions