The Computer Expression Recognition Toolbox
Littlewort, Wu, Whitehill, Fasel, Movellan, Bartlett
Computer Expression Recognition Toolbox (CERT)is an end-to-end system for fully automated facial expression recognition that operates in real-time. CERT was developed at the University of California, San Diego, originating from a collaboration between Ekman and Sejnowski (Bartlett et al., 1996, 1999, 2006; Donato et al., 1999; Littlewort et al., 2006). The current system automatically detects frontal faces in the video stream and codes each frame with respect to 40 continuous dimensions, including basic expressions of anger, disgust, fear, joy, sadness, surprise, contempt, a continuous measure of head pose (yaw, pitch, and roll), as well as 30 facial action units (AUÕs) from the Facial Action Coding System. The technical approach to CERT is an appearance-based, discriminative approach. Current research focuses on machine learning and computer vision approaches to improve facial expression detection performance, analysis of dynamics, and temporal segmentation. This system is being employed in pioneering experiments on spontaneous behavior, including automated discrimination of posed from genuine expressions of pain, automated detection of driver drowsiness, adaptive tutoring systems, and an intervention for children with autism.
NSF CISE/CRI: Automated Facial Expression Measurement: Toolbox and Database 7/15/05-6/30/10. PI: Bartlett
NSF IIS ITR: Automatic Analysis of Spontaneous Facial Expressions 9/1/02-8/31/04 PI: Movellan
Sikka K, Dykstra, Sathyanarayana, Littlewort, Bartlett (2013). Multiple Kernel Learning for Emotion Recognition in the Wild. Proc. 15th ACM International conference on multimodal interaction. Pages 517-524. Best paper award. Download pdf
Sikka K, Wu T, Susskind J, and Bartlett M (2012). Exploring Bag of Words Architectures In the facial expression domain. In A Fitzgibbon, S Lazebnik, P Perona, Y Sato, and C Schmid (Eds), Lecture Notes in Computer Science, Computer Vision – ECCV 2012, Vol. 7584, Springer, p. 250-259. Download pdf
Wu T, Butko NJ, Ruvolo P, Whitehill J, Bartlett MS, Movellan JR (2012). Multi-Layer Architectures for Facial Action Unit Recognition. IEEE Transactions on Systems, Man and Cybernetics -- Part B (TSMC-B). Download pdf
Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, and Bartlett M (2011) The Computer Expression Recognition Toolbox (CERT). Proc. IEEE International Conference on Automatic Face and Gesture Recognition, p. 298-305. Download pdf
Wu, T., Bartlett, M.S., and Movellan, J. (2010). Facial expression recognition using gabor motion energy filters. IEEE CVPR workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis, p. 42-47. Download pdf
Bartlett, M., Littlewort, G., Whitehill, J., Vural, E., Wu, T., Lee, K., Ercil, A., Cetin, M. Movellan, J. (2010). Insights on spontaneous facial expressions from automatic expression measurement.Ó In Giese,M. Curio, C., Bulthoff, H. (Eds.) Dynamic Faces: Insights from Experiments and Computation. MIT Press.Download pdf
Bartlett, M., Whitehill, J. (2010). Automated facial expression measurement: Recent applications to basic research in human behavior, learning, and education.Ó In Handbook of Face Perception, Andrew Calder, Gillian Rhodes, James V. Haxby, and Mark H. Johnson (Eds). Oxford University Press. Download pdf
Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., & Movellan, J. (2009). Towards practical smile detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 99(2) p. 2106-2111. Download pdf
Bartlett, M.S., Littlewort-Ford, G.C., and Movellan, J.R. (2008). Computer Expression Recognition Toolbox. Demo: 8th IntÕl IEEE Conference on Automatic Face and Gesture Recognition. Amsterdam. Download pdf
Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lainscsek,C., Fasel, I., Movellan, J.R. (2006). Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia 1(6) p. 22-35. Download pdf
Littlewort, G., Bartlett, M., Fasel, I., Susskind, J., and Movellan, J. (2006). Dynamics of facial expression extracted automatically from video. Image and Vision Computing 24(6), p. 615-625. Download pdf
Bartlett, M.S., Littlewort, G., Frank, M.G., Lainscsek, C., Fasel, I., and Movellan, J. (2005). Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior. IEEE International Conference on Computer Vision and Pattern Recognition. p. 568-573. Download pdf
Bartlett, M.S., Littlewort, G., Braathen, B., Sejnowski, T.J., & Movellan, J.R. (2003). A prototype for automatic recognition of spontaneous facial actions. In S. Becker & S. Thrun & K. Obermayer, (Eds.) Advances in Neural Information Processing Systems, Vol 15, p. 1271-1278, MIT Press. Download pdf
Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.R. (2003). Real time face detection and expression recognition: Development and application to human-computer interaction. CVPR Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction. Download pdf
Bartlett, M.S., Donato, G.L., Movellan, J.R., Hager, J.C., Ekman, P., and Sejnowski, T.J. (2000). Image representations for facial expression coding. In S. Solla, T. Leen, & K. Mueller, Eds. Advances in Neural Information Processing Systems 12, Cambridge, MA: MIT Press, p. 886-892.
Donato, G.L., Bartlett, M.S., Hager, J.C., Ekman, P., and Sejnowski, T.J. (1999). Classifying Facial Actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10) p. 974-989. Abstract, Download pdf
Bartlett, M. Stewart, Viola, P.A., Sejnowski, T.J., Golomb, B.A., Larsen, J., Hager, J.C., and Ekman, P. (1996). Classifying facial action, Advances in Neural Information Processing Systems 8, MIT Press, Cambridge, MA. p. 823-829.