Research Themes
AI for Performing Arts
In this theme we explore the aesthetic aspects of human communicative behavior. For example, what are the mechanisms of communication between a musical conductor and an orchestra, how do the musicians interpret the conductor motion? Why do humans get stimulation from watching a dancer? And, how can a completely different embodiment, e.g., a swarm of drones, express feelings and attitudes while performing on stage? We investigate these questions in collaboration with a range of performing arts professionals; musicians, conductors, and dancers.
Current Projects
- OrchestrAI: Deep generative models of the communication between conductor and orchestra (WASP, SeRC 2023-present)
Selected Papers
- Mert Mermerci, Emile Pascoe, Fredrik Edström, and Hedvig Kjellström. Real-time control of a virtual orchestra by recognition of conducting gestures. arXiv preprint arXiv:2604.27957, 2026.
- Sara Eriksson, Åsa Unander-Scharin, Vincent Trichon, Carl Unander-Scharin, Hedvig Kjellström, and Kristina Höök. Dancing with drones: Crafting novel artistic expressions through intercorporeality. In ACM CHI Conference on Human Factors in Computing Systems, 2019.
- Kelly Karipidou*, Josefin Ahnlund*, Anders Friberg, Simon Alexanderson, and Hedvig Kjellström. Computer analysis of sentiment interpretation in musical conducting. In IEEE Conference on Automatic Face and Gesture Recognition, 2017. (*Joint first authors)
AI for Healthcare
In this theme, the main focus is how human cognitive processes and health status can be inferred from observable behavior. Past and present such projects include computerized analysis of cognitive decline and motion analysis to detect motor disease in infants.
Current Projects
- The relation between motion and cognition in infants (SeRC 2023-present)
- UNCOCO: UNCOnscious COmmunication (WASP 2023-present)
Selected Papers
- Chen Ling, Henglin Shi, and Hedvig Kjellström. FIELDS: Face reconstruction with accurate inference of expression using learning with direct supervision. arXiv preprint arXiv:2511.21245v3, 2026.
- Patrik Jonell*, Birger Moëll*, Krister Håkansson*, Gustav Eje Henter, Taras Kucherenko, Olga Mikheeva, Göran Hagman, Jasper Holleman, Miia Kivipelto, Hedvig Kjellström, Joakim Gustafson and Jonas Beskow. Multimodal capture of patient behaviour for improved detection of early dementia: Clinical feasibility and preliminary results. Frontiers in Computer Science 3, 2021. (*Joint first authors)
- Xueru Zhang*, Ruibo Tu*, Yang Liu, Mingyan Liu, Hedvig Kjellström, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualification? In Neural Information Processing Systems, 2020. (*Joint first authors)
- Ruibo Tu, Kun Zhang, Bo Christer Bertilson, Hedvig Kjellström, and Cheng Zhang. Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation. In Neural Information Processing Systems, 2019.
- Cheng Zhang, Hedvig Kjellström, Carl Henrik Ek, and Bo Christer Bertilson. Diagnostic prediction using discomfort drawings with IBTM. In Machine Learning for Healthcare, 2016.
AI for Animal Science
In this area we develop models of animal behavior and non-verbal communication. This includes modeling dog communication, and automatic video detection of signs of pain in horses. An important current strand of research is our work on creating accurate 3D pose and shape models of horses and dogs.
Current Projects
- ANITA: ANImal TrAnslator (VR 2024-present)
- MARTHA: MARkerless 3D capTure for Horse motion Analysis (KTH, FORMAS 2020-present)
Selected Papers
- Ci Li, Yi Yang, Zehang Weng, Elin Hernlund, Silvia Zuffi, and Hedvig Kjellström. Dessie: Disentanglement for articulated 3D horse shape and pose estimation from images. In Asian Conference on Computer Vision, 2024.
- Ci Li, Ylva Mellbin, Johanna Krogager, Senya Polikovsky, Martin Holmberg, Nima Ghorbani, Michael J. Black, Hedvig Kjellström, Silvia Zuffi, and Elin Hernlund. The poses for equine research dataset (PFERD). Nature Scientific Data 11, 497, 2024.
- Silvia Zuffi, Ylva Mellbin, Ci Li, Markus Hoeschle, Hedvig Kjellström, Senya Polikovsky, Elin Hernlund, and Michael J. Black. VAREN: Very accurate and realistic equine network. In IEEE Conference on Computer Vision and Pattern Recognition, 2024.
- Sofia Broomé, Marcelo Feighelstein, Anna Zamansky, Gabriel Carreira Lencioni, Pia Haubro Andersen, Francisca Pessanha, Marwa Mahmoud, Hedvig Kjellström, and Albert Ali Salah. Going deeper than tracking: A survey of computer-vision based recognition of animal pain and affective state. International Journal of Computer Vision 131, 2023.
- Sofia Broomé, Karina Bech Gleerup, Pia Haubro Andersen, and Hedvig Kjellström. Dynamics are important for the recognition of equine pain in video. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
