Micro facial expression recognition using generative adversarial network generated super resolution images

Abstract

Recent years have seen a remarkable acceleration in the development of micro facial expression recognition techniques. The recordings made in daily lives engaging with the actual environment provide the prime sources for many studies, yet the quality of these data is sometimes low. As a result, low resolution micro expressions have become a new area of research. Due to the lack of clear understanding to differentiate inter class features and to accurately recognize themicro facial expression among several categories is challenging. The ability to distinguish between micro-facial features is further diminished by the low resolution of such images which doubles the recognition challenge. This work provides a novel method that makes use of theGenerative Adversarial Network to apply a super resolution methodology to overcome the low-resolution conflicts on facial micro expression. The overall performance of the model was assessed with the recognition accuracy achieved using a support vector machine. Additionally,the image quality was measured employing different methods to assess reconstruction performance. The proposed approach was tested using low resolution images simulated from the CASME-II, Spontaneous Micro-expression database (SMIC-HS) and SMIC-subHS dataset, demonstrating the utility of the method. The ESRGAN model achieves the best reconstruction performance for micro expressions, with image metrics of 30.887 dB PSNR, 0.000865 MSE, and 0.938 SSIM, considering the scale factor in five defined classes as happiness, surprise, disgust, depression, and others.