Multimodal Analysis of Pilot’s Fatigue During a Multi-Phase Flight Mission

Abstract

One troubling threat to successful flight missions is attributed to fatigue induced and errors. Therefore, discovering effective methods to assess fatigue has been a major topic discussed by professional pilots and aviation experts. Fatigue is a major human factor related issue in aviation and currently subject to increased discussion by aviation administrations and professional pilots. Therefore, effective assessment of fatigue will provide opportunities to reduce the risk of fatigue-induced errors. Currently available subjective measures that assess fatigue can be somewhat affected by external and internal factors, that might cause biased judgment. Therefore, Psychomotor Vigilance Task (PVT), which provides objective measures, can be a viable approach to measure fatigue. In addition, eye movement analysis might augment the fatigue assessment, because eye movement analysis is an unobtrusive approach that does not require direct contact with the participant and can be measured for a long duration. However, it is unknown how eye movement characteristics are correlated with fatigue. In this research, a multi-modal fatigue measurement framework was developed by combining the PVT analysis with eye movement analysis. In detail, PVT measures (i.e., reaction time, lapses & false starts) and eye movement characteristics (i.e., eye fixation duration, pupil size, number of eye fixations, gaze entropy) were measured to determine pilots’ fatigue level under different flight conditions. The results show that significant correlations exist among the eye movement characteristics and the PVTs measures. The proposed multi-modal approach show promise on evaluating pilot fatigue in near real time, which in turn might enable timely recovery interventions.