Learning second language (L2) sounds is influenced by the first language (L1) sound system. This current study seeks to examine how English language learners who have a smaller L1 vowel inventory perceive English sounds. To reveal L2 sound perception, we measured the mouse trajectories related to the hand movements performed by Javanese and Sundanese language learners as they selected their responses. This study aims to map the patterns of real-time processing through compatible hand movements, to reveal any uncertainties participants have when making selections. The participants completed a categorization task, classifying stimuli as long or short vowels. Results show clear regional differences in the OVE that corresponds to the regional differences in fricative devoicing. The devoicing process seems thus to trigger a reduction of the OVE, but not (yet) its full disappearance. We argue that these results are in line with the full merger hypothesis. The results showed that the Javanese listeners exhibited significantly lower velocity values than the English listeners for the similar vowels /I, ɛ, ʊ/ between 826-1200 ms post-stimulus interval. The Sundanese listeners showed lower velocity values than the English listeners for /I/ approximately between 676-1200 ms post-stimulus interval. For the perception of the new vowels /i:, æ, ɜ:, ʌ, ɑː, u:, ɔ:/, the Javanese listeners showed lower velocity in making a decision between 826-1200 ms post stimulus interval. The Sundanese listeners showed lower velocity only for the vowels /ɜ:, ɔ:, æ/ between 676-1200 ms post stimulus interval. Lower velocity values of the hand movement suggest that during the processing of L2 vowels, the L2 learners experienced greater uncertainty in decision making. The results are consistent with the Second Language Linguistic Perception Model (Escudero, 2005), which predicts how L2 learners struggle to perceive new L2 sounds, when the L1 perception grammar has less perceptual categories than the L2.