Identifying Expressions of Emotion in Text

Table 7. Features Used in emotion classification

GI Features	WN-Affect Features	Other Features
Emotion words	Happiness words	Emoticons
Positive words	Sadness words	Exclamation (“!”) and
Negative words	Anger words	question (“?”) marks
Interjection words	Disgust words
Pleasure words	Surprise words
Pain words	Fear words

2. Experiments and Results

For our binary classification experiments, we used Naïve Bayes, and Support Vector Machines (SVM), which have been popularly used in sentiment classification tasks [6, 9]. All experiments were performed using stratified ten-fold cross validation. The naïve baseline for our experiments was 65.6%, which represents the accuracy achieved by assigning the label of the most frequent class (which in our case is NE) to all the instances in the dataset. Each sentence was represented by a 14-value vector, representing the number of occurrences of each feature type in the sentence. Table 9 shows the classification accuracy obtained with the Naïve Bayes and SVM text classifiers. The highest accuracy achieved was 73.89% using SVM, which is higher than the baseline. The improvement is statistically significant (we used the paired t-test, p=0.05).
To explore the contribution of different feature groups to the classification performance, we conducted experiments using (1) features from GI only, (2) features from WordNet-Affect only, (3) combined features from GI and WordNet-Affect, and
(4) all features (including the non-lexical features). We achieved the best results when


Table 8. Emotion classification accuracy