Duchenne experiment data accompanying the paper by Whitehill, et al., Whose Vote Should Count More: Optimal Integration of Labels from Labelers of Unknown Expertise", NIPS 2009:
The data above were used to obtain the results presented in the NIPS 2009 paper. The data can be used for academic purposes; please cite the paper mentioned above.
- Click here for the Duchenne labels obtained from the Mechanical Turk workers. Format: each row contains three columns. The first column contains the image ID; the second column contains the labeler ID; the third column contains the label (0 = non-Duchenne, 1 = Duchenne).
- Click here for the "ground truth" Duchenne labels obtained from expert facial expression coders. Format: each row contains two columns. The first column contains the image ID; the second column contains the ground truth Duchenne label (0 or 1).
- Click here for a Matlab script to compute the accuracy of estimating the Duchenne labels using GLAD versus Majority Vote.
- The faces themselves are unfortunately not available as they are proprietary.
In addition, we have since improved upon the GLAD algorithm presented in the NIPS 2009 paper: for better results on the Duchenne data, please see Ruvolo, et al., Exploiting Structure in Crowdsourcing Tasks via Latent Factor Models, MPLab Tech Report 2010.