CERT is a system for automated FACS (Facial Action Coding System, see http://mplab.ucsd.edu/grants/project1/research/Fully-Auto-FACS-Coding.html for more details). Each frame of video, or individual image is first given to an automated face and eye detector. The face is cropped and rotated then passed into the action coding system. The result from that system is a set of values for eight different facial action units:

1	Inner brow raise
2	Outer brow raise
4	Brow corrugator
5	Upper lid raise
10	Nasolabial furrow, "disgust"
12	Lip corner pull, "smile"
14	Dimpler
20	Lip stretch, (pull down corners of the mouth) "Eeek!"

Each value gives a relative indication of how much a particular AU is present.

AUCoderDemo

AUCoderDemo is a demo implementation of CERT that runs on Mac OS X. It can process individual images, folders of images as well as many live Quicktime video sources. For each image or frame of video, it attempts to find a face, then calculate Facial Action Units for that face. Values are displayed alongside the image or video along with a short graph of previous AU values. A separate window can be used to record AU values along with eye location information which can be saved to a text file.

Basic Use. When you first start AUCoderDemo the main window should be visible. Along the top of the main window are three buttons allowing you to load one image, a folder of images or start processing video. A wide range of image types can be loaded (jpeg, png, etc.). To start processing video, make sure your camera is attached and operating, then click on the video button. Click the video button again to stop. You can adjust the width and height of the video image before starting by using the fields at the bottom of the window. Larger image sizes will require more processing power and/or time.

AU value graphs. Along the right hand side of the main window are eight (plus one) small graphs that display the most recent AU value along with a graph displaying up to one hundred previous values. The scale on each graph is adjusted independently based on the largest and smallest values seen since the application was launched. Because of the simplistic scaling algorithm, some changes in value over time may be difficult to see. Recording data and saving it for later analysis is a more accurate way to detect trends and changes in AU values. Brief descriptions of each AU will be displayed as you hover the cursor over each AU graph.

Three check boxes near the bottom of the window allow you to turn on or off some features. Unchecking the "Find Faces" button will stop all processing of video and is really only useful for checking the base frame rate of the video. The "Calc AU's" check box will toggle processing of the eight Facial Action Units. The "Smiles" check box turns on or off a separate module that measures smiles and is distinct from the AU 12 processing. Two text boxes at the bottom of the window give eye locations. And a third shows an estimate of the frame rate, if video is being processed.

The "Face/AU Data" window can be used to capture values for later use. Clicking the "Start Recording" button will start recording all data (whether from images or video) and the button title will change to "Stop Recording". Data won't show up in the window until the "Stop Recording" button is clicked. Recorded Data includes AU values (for AUs 1, 2, 4, 5, 10, 12, 14 and 20), eye locations, smile detector value (if smile detection is turned on), and file name if images are being processed (as opposed to live video).

Choosing the "Preferences..." menu item from the AUCoderDemo menu brings up a small settings window. None of these settings are particularly useful for demos. The Eyefinder settings have already been optimized. The "Filter AU values" check box toggles a system that attempts to convert the raw AU values into probabilities that a particular expression is present with values from 0.0 to 1.0 rather than the unbounded raw values. Unfortunately, that system is not working correctly for any value other than the Smile detector. The "Display Gray" check box displays the gray scale version of the image that is actually being processed by the system. This can be useful for diagnosing problems, as some image types don't get converted to gray scale properly. If the "Capture" check box is turned on, images with found faces in them will be saved to disk (in the Sites/Faces folder of the current user) at a maximum rate of one per second. At most twenty images will be saved, at which point the numbered file names will roll back to "CapturedFace1.jpg" overwriting earlier ones.

Source Code

The CERT2_0 directory contains all of the platform independent source code needed to build a command line tool that can be used to generate AU values from an image, or a library that can be used to include CERT in another application.

There are four third party libraries that are required to run CERT: FFTW, libXML, cBlas and Boost. See the ThirdPartyLibraries.rtf file for information and URL's about them.

The CERT2_VS2005 directory contains a Microsoft Visual Studio project that can be used to build a static library and a command line application that demonstrates how to build and use CERT. After building the demo, CERTDemo.exe will be created in the debug subdirectory directory. Note that the demo looks for dlls for the third party libraries and will fail to load if the dlls are not in the windows path or the same directory as the exe. Copy them to the debug subdirectory if necessary. There is a directory with three test images inside CERT2_VS2005. The approximate AU values for these three images are:
1005.bmp 
5.6333114, -0.71475668, 1.9317574, -7.7926074, -2.3050803, -1.1298686, -14.421441, -7.4386707

1010.bmp 
-3.8970356, -8.6795982, 0.64116601, -5.6882162, -7.4389054, -2.905899, -10.705396, -9.4013099

1014.bmp 
1.7217556, -5.3578102, 4.6767152, -7.5143453, -2.8906815, -4.2021838, -16.549772, -11.502446

Starting at the top, the CERTWrapper directory contains a main function (in main.cpp) which reads an image from a path given on the command line or entered when prompted and returns the eight AU values. This file also includes sample code showing how to convert an image into the internal representation needed by CERT (see the RImage<float> class), using either a third-party library called ImageMagick (http://www.imagemagick.org/Magick++/), or using native Window's code. The main function then uses a class called MP_CERT to generate the AU values. MP_CERT can be used as is, but is also meant to be a demo showing how the CERT algorithm works, and how all of the other pieces work together.

MP_CERT requires two pieces of information. A path to the directory that contains the SVM weight data files (which defaults to "SVMWeights" in the current working directory) and the image to process, in the form of an RImage. The MP_CERT constructor first sets the directory for the SVMWeights files, then pre-loads all of them. Then it adjusts some settings on the eyefinder and the face rotator.

The calcAUs method of MP_CERT performs three main steps.
1) Find face and eyes in the image, returning a std::vector of FaceObjects.
2) Use the best estimate for the eye locations (both_eyes.xLeft, both_eyes.yLeft, etc.) to crop and rotate a face.
3) Use one of the Gabor classes (multi-threaded or single-threaded) to calculate the AU values on the cropped and rotated face.

AU values are returned by CERT for AU's: 1, 2, 4, 5, 10, 12, 14 and 20. They are returned in that order in a vector.

In addition to the values for the eight AU's currently being calculated it can also be useful during testing to return the face and eye data. It is possible for the eye and face finder to return more than one face. If you are processing video it might make sense to skip those frames.

SVMWeights: The XML formatted data files in this directory need to be accessible when calculating AU's. This is done either by having all of the data files available in the application's current working directory, or by passing a directory path (as a string) to the mp_SVMWeights class. The constructor of the mp_CERT class shows an example of this. Directories can be given either as relative paths from the current working directory, or as absolute paths. The paths should be given as system native paths, e.g. "/Users/Shared/data/SVMWeights" on a Mac OS X system, or "C:\CERT\mp_auCoder\SVMWeights" on a Window's system. Loading all of the SVMWeights can take a significant amount of time for the first image, but can also be done in a preliminary step (see the loadAllWeights() method of the mp_SVMWeights class).

MP_Gabor: This class calculates Facial Action Units given an aligned and cropped face image. There are two versions, one is single threaded and the other is multi-threaded. Both have the same interface which consists of setting the image to be worked on then calling the main entry method CERT_Gabor, which returns a vector with the AU values. The multi-threaded version runs a portion of the code in three separate threads using the Boost library's thread classes. 

The Misc directory has code that can be used to get release version info either offline or at runtime. Any questions or bug reports should include both version numbers (mpt and cert). In the source files the version is recorded in the line: #define SVN_REV "<version #>". At runtime the getCertRevisionString() and getMPTRevisionString() functions can be used to get version strings.

Code-level documentation generated by Doxygen is available in the CERT2_0 directory at: Documentation/html/index.html