In my research I study human communication and develop computational models that allow computers and robots to have face-to-face conversations with humans. This involves both verbal and non-verbal (gaze, prosody, etc) aspects of communication, and my research involves phenomena such as turn-taking, feedback, joint attention, and language acquisition. Since social robots are likely to play an important role in our future society, the technology has direct applications, but it can also be used to increase our understanding of the mechanisms behind human communication. This requires an interdisciplinary approach, which includes phonetics, linguistics, language technology, artificial intelligence and machine learning.
In 2014, I co-founded the company Furhat Robotics together with Samer Al Moubayed and Jonas Beskow at KTH. I am working part-time as a Chief Scientist in the company.
I am the President of SIGdial, the ACL (Association for Computational Linguistics) Special Interest Group on Discourse and Dialogue, and Associate Editor for the Human-Robot Interaction section of Frontiers in Robotics and AI.
I am an alumni member of the Young Academy of Sweden - an independent, cross-disciplinary forum for some of the most promising young researchers in Sweden in all disciplines.
I have a MSc in Cognitive Science from Linköping University and I did my PhD in Speech Communication at KTH in 2007. The title of my thesis is Error Handling in Spoken Dialogue Systems: Managing Uncertainty, Grounding and Miscommunication. I was a post-doc at the University of Potsdam in 2008, became Docent in Speech Technology at KTH in 2012, Assistant Professor in 2013, Associate Professor in 2016 and Professor in 2018.
Master thesis opportunities
- Interested in doing your Master thesis in any of the projects listed below? See here
Ongoing research projects
|Representation Learning for Conversational AI. How can we develop generic representation learning that is better suited for conversation than current language models? Funded by WASP (2021-2026).|
|PREDICON - Prediction and Coordination for Conversational AI. How can we use deep learning to predict human actions in a conversation? Funded by the Swedish Research Council (2021-2024).|
|AnalyTIC - Understanding Predictive models of Turn-taking in Spoken Interaction. How can we use deep learning models to better understand the signals used to coordinate turn-taking in human communication? Funded by Riksbankens Jubileumsfond (2021-2023).|
|Robot learning of symbol grounding in multiple contexts through dialog. How can a robot learn to understand the meaning of words in a conversation with a human? Funded by WASP (2018-2023).|
Social robots accelerating the transition to sustainable transport. How can social robots be used as hosts on driverless busses? Funded by Energimyndigheten (2020-2024).
|Early Language Development in the Digital Age (e-LADDA). How can social robots be used for language learning? A Marie Skłodowska-Curie Innovative Training Network, funded by EU MSCA ITN (2020-2023)|
|COIN: Co-adaptive Human-Robot Interactive systems. How can a robot adapt its presentation based on multimodal feedback from the human listeners? Funded by SSF (2016-2021)|
|COBRA: Conversational Brains. Modelling the linguistic, cognitive and brain mechanisms that allow conversation to unfold in both human-human and human-machine interactions. A Marie Skłodowska-Curie Innovative Training Network, funded by EU MSCA ITN (2020-2023)|
My research is funded by:
- Faculty advisor for the KTH Fantom team in the Amazon Alexa Prize challenge (2018)
- Lead Judge, IBM Watson A.I. XPRIZE (2017-2021)
- Local char, SIGDIAL 2019
- Area chair, Dialog systems, ACL 2019-2021
- Area chair, Dialog systems, NAACL 2018, 2021
- Area chair, Dialog systems, Interspeech 2018
- Area chair, Dialog systems, EACL 2017
- Honourable mention Award, CUI 2019, Dublin
- Nominated for Test-of-time Award, NAACL 2018, New Orleans
- Nominated for Best Paper Award at HRI 2017, Vienna
- Best Paper Award at ICSR 2016, Kansas City
- Nominated for Outstanding Paper Award at ICMI 2015, Seattle
- Two nominations for Best Paper Award at SIGdial 2013, Metz
- Robotdalen Innovation Award (2nd prize) 2013
- Outstanding Demo Award at ICMI 2012, Santa Monica
- Best Paper Award at SIGdial 2010, Tokyo
Ongoing PhD supervision
- Bram Willemsen (2019-), Main supervisor
- Erik Ekstedt (2018-), Main supervisor
- Nils Axelsson (2018-), Main supervisor
- Ronald Cumbal (2019-), Co-supervisor
- Dmytro Kalpakchi (2018-), Co-supervisor
- Robert Johansson (2016-), Co-supervisor
- Dimosthenis Kontogiorgos (2016-), Co-supervisor
- Peter Blomsma (2021-), Co-supervisor
Past PhD supervision
- Maike Paetzel (2015-2020), Co-supervisor
- Martin Johansson, (2012-2020), Main supervisor
- Raveesh Meena (2011-2016), Main supervisor
- Jana Götze (2011-2016), Co-supervisor