Databases

RUFACS1 (Rochester/UCSD FacialActionCodingSystem Database 1)

Data collection set up. a. The frontal camera was mounted in a bookshelf above the interrogator’s head. b. Two side view cameras were wall-mounted. A fourth camera was mounted under the interrogator’s chair. c. Interrogators were retired members of the police and FBI, including this county sherrif

The RU-FACS-1 database (short for Rutgers and UCSD FACS database) is being collected by Mark Frank at Rutgers University as part of a collaboration with the Machine perception lab to automate FACS. This database consists of spontaneous facial expressions from multiple views, with ground truth FACS codes provided by two facial expression experts. The data collection equipment, environment, and paradigm were designed with advice from machine vision consultant Yaser Yacoob and facial behavior consultant Paul Ekman. The system records synchronized digital video from 4 Point Grey Dragonfly video cameras and writes directly to RAID.

Subjects participated in a false opinion paradigm in which they were randomly assigned to either lie or tell the truth about their opinion on an issue about which the person has indicated strong feelings (e.g., Frank & Ekman, 1997; 2004). The subject attempts to convince an interviewer he or she is telling the truth. Interviewers are current and former members of the police and FBI. The participants are informed ahead of time of the following pay-offs: (1) If they tell the truth and are believed, they will receive $10; (2) If they lie and are believed they will receive $50; (3) If they are not believed regardless whether they are lying or telling the truth, they are told that they will receive no money and will have to fill out a long boring questionnaire. This paradigm reliably generates many different expressions in a short period of time, which is key for collecting data for training computer vision systems. The paradigm involves natural interaction with another person and does not make the subject aware that the emphasis of the study is on his or her facial expression.

We have collected data from 100 subjects, 2.5 minutes each. This database constitutes a significant contribution towards the 400-800 minute database recommended in the feasibility study for fully automating FACS. To date we have human FACS coded the upper faces of 20% the subjects, which is now available for release. We are on target for finishing the coding by the end of June, and to complete the machine based scoring by the end of the project in 8/31/04.

ORATOR (Expressive Speech Database)

ORATOR was collected by Holger Quast

The following downloadable package contains the wave soundfiles, evaluations, and descriptions of the Orator speech database.
Download: ORATOR (Expressive Speech Database)

For this purpose, 145 spoken versions of the same 8-sentence monologue were recorded:
(1) In der Vergangenheit ist schon einiges an guter Vorarbeit geleistet worden.
(2) Die Ziele, die wir jetzt verfolgen, sind die gleichen und müssen auch auf die gleiche Weise behandelt werden.
(3) Unsere Aufgabe ist nun, noch einmal die Zeiteinteilung durchzusehen.
(4) Sie überprüfen dann das Weitere.
(5) Bitte notieren Sie die Punkte, die Sie heraussuchen, und tragen Sie uns diese vor!
(6) Wir erledigen alles Andere.
(7) Glauben Sie, dass Sie das schaffen?
(8) Gut!

13 professional actors were asked to interpret this text in different social situations producing a total of 117 samples. 14 non-actors spoke the monologue in their natural register yielding the remaining 28 recordings. 20 non-German evaluators listened to these samples and rated their affective content. For this evaluation, sentence 2 of the above monologue was used. 5 additional sentences were included to validate the scores.

The Orator database consists of the following directories and files:

Readme.pdf
Audio\complete monologues\1.wav ... 145.wav - The 145 recorded monologues. Format is the original DAT 48kHz mono pcm wav.
Audio\evaluated sentences\1_2.wav ... 150_2.wav - The 150 evaluated sentences (sentence 2 of the monologue + 5 extra-samples; for instance, file 23_2.wav contains the second sentence of recording 23.wav). Resampled to CD-rate 44.1kHz.
Speakers & Recordings.pdf . - The list of speakers of the 145 recorded monologues. Additionally, for the evaluation, 5 extra-samples were played: two repetitions and three sentences other than #2. These extra-samples are files 146_2.wav ... 150_2.wav.
Evaluation Results.xls - Holds all scores the 20 evaluators attributed to the audio samples in an Excel-sheet. These results should be normalized with respect to mean and variance, see the tech report. Also refer to the report to see which impressions can be recognized in the data.
Quast MPLab Tech Report 2002.2.pdf - This paper explains the project in detail. Read it to get a better understanding of what the data looks like, motivation, signal processing, pattern recognition, and results of the work.

TULIPS1 (Audiovisual Database)
The database was compiled by Javier R. Movellan
The following downloadable package contains the files and description necessary to use the Tulips1 Audiovisual database. Download: TULIPS1 (Audiovisual Database) Tulips 1.0 is a small Audiovisual database of 12 subjects saying the first 4 digits in English. Subjects are undergraduate students from the Cognitive Science Program at UCSD. The database was compiled at Javier R. Movellan's laboratory at the Department of Cognitive Science, UCSD. If you use these data for your research please refer to it as the "TULIPS1 database, (Movellan, 1995)". Movellan J. R. (1995) Visual Speech Recognition with Stochastic Networks. in G. Tesauro, D. Toruetzky, & T. Leen (eds.) Advances in Neural Information Processing Systems, Vol 7, MIT Pess, Cambridge.

Back to: Automatic Analysis of Spontaneous Facial Expressions