Guest seminar Hayato Yamamoto

Title: Auditory–Vibrotactile Asynchrony Detection In Deaf/Hard-Of-Hearing And Hearing Using A Single Device.

The seminar is given in Japanese accompanied with sign language, and automatically interpreted into English.

Abstract: This study examines how Deaf and Hard-of-Hearing (DHH) individuals detect asynchrony between auditory and vibrotactile stimuli to improve the “VIBES” singing-timing support system. Twenty DHH and seven hearing participants completed a cross-modal synchrony judgment task in which they judged whether a single device’s paired auditory and vibrotactile stimuli were simultaneous. We tested two conditions: a regular rhythm (RR) condition with contextual beats at 120 beats per minute (BPM) around the target stimulus and a no-rhythm (NR) condition with only the target stimulus. In the DHH group, the point of subjective simultaneity (PSS)—the time offset at which two stimuli are perceived as simultaneous—under the NR condition shifted to -15 [ms] (vibrotactile lead of 15 [ms]; p=0.018, r=0.54), indicating that an earlier vibrotactile cue enhances perceived simultaneity. DHH participants also exhibited a trend toward a wider temporal binding window (TBW—the interval during which stimuli from different senses are perceived as one event) when the auditory stimulus preceded the vibrotactile stimulus (p=0.066), although variability across individuals was high. No significant effect of condition on PSS or TBW was observed in the hearing group. These results suggest that DHH listeners may detect asynchrony by comparing a target event to surrounding stimuli. Therefore, practical deployment of VIBES will require optimizing vibrotactile timing for each user based on individual PSS and TBW measurements. Future work will expand the hearing sample and conduct finer-grained analyses to develop an optimized vibrotactile timing model that maximizes perceptual clarity for DHH users.