Skip to main content

Research Repository

Advanced Search

Intent Inference for Hand Pointing Gesture-Based Interactions in Vehicles

Ahmad, Bashar I.; Murphy, James K.; Langdon, Patrick M.; Godsill, Simon J.; Hardy, Robert; Skrypchuk, Lee


Bashar I. Ahmad

James K. Murphy

Simon J. Godsill

Robert Hardy

Lee Skrypchuk


Using interactive displays, such as a touchscreen, in vehicles typically requires dedicating a considerable amount of visual as well as cognitive capacity and undertaking a hand pointing gesture to select the intended item on the interface. This can act as a distractor from the primary task of driving and consequently can have serious safety implications. Due to road and driving conditions, the user input can also be highly perturbed resulting in erroneous selections compromising the system usability. In this paper, we propose intent-aware displays that utilize a pointing gesture tracker in conjunction with suitable Bayesian destination inference algorithms to determine the item the user intends to select, which can be achieved with high confidence remarkably early in the pointing gesture. This can drastically reduce the time and effort required to successfully complete an in-vehicle selection task. In the proposed probabilistic inference framework, the likelihood of all the nominal destinations is sequentially calculated by modeling the hand pointing gesture movements as a destination-reverting process. This leads to a Kalman filter-type implementation of the prediction routine that requires minimal parameter training and has low computational burden; it is also amenable to parallelization. The substantial gains obtained using an intent-aware display are demonstrated using data collected in an instrumented vehicle driven under various road conditions.

Journal Article Type Article
Acceptance Date Mar 23, 2015
Online Publication Date Apr 28, 2015
Publication Date 2016-04
Deposit Date Dec 12, 2019
Journal IEEE Transactions on Cybernetics
Print ISSN 2168-2267
Electronic ISSN 2168-2275
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 46
Issue 4
Pages 878-889
Keywords Bayesian inference, finger tracking, human computer interactions (HCIs), interactive displays, Kalman filtering
Public URL