Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

Transcription

1 Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis based on the article with the same name by Theresa Wilson, Janyce Wiebe and Paul Hoffmann Department of Computational Linguistics Saarland University Proseminar Sentimentanalyse May 20th, 2015

2 Overview 1 Introduction 2 Methods 3 Experiments 4 Conclusion Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 2 / 36

3 Overview 1 Introduction 2 Methods 3 Experiments 4 Conclusion Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 3 / 36

4 What is Sentiment Analysis? Sentiment Analysis Negative or Positive? opinions emotions evaluations, etc Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 4 / 36

5 What is Sentiment Analysis? Let s start with separate words. Negative, positive or neutral? Lexicon Word Tag trust well reason reasonable horrid polluter think Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 5 / 36

6 Motivation: Why Phrase-Level Sentiment Analysis? Typical Approach: Lexicon Word Tag trust pos well pos reason pos reasonable pos horrid neg polluter neg think neut No Context = Prior Polarity Works well on document level E.g. distinguishing between positive and negative reviews Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 6 / 36

7 Motivation: Why Phrase-Level Sentiment Analysis? How about this example? Philip Clapp, president of the National Environment Trust, sums up well the general thrust of the reaction of environmental movements: There is no reason at all to believe that the polluters are suddenly going to become reasonable. Lexicon Word Tag trust pos well pos reason pos reasonable pos polluter neg Polarity in Context Word Tag trust well reason reasonable polluter Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 7 / 36

8 Motivation: Why Phrase-Level Sentiment Analysis? How about this example? Philip Clapp, president of the National Environment Trust, sums up well the general thrust of the reaction of environmental movements: There is no reason at all to believe that the polluters are suddenly going to become reasonable. Lexicon Word Tag trust pos well pos reason pos reasonable pos polluter neg Polarity in Context Word Tag trust neut well pos reason neg reasonable neg polluter neg Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 8 / 36

9 Overview 1 Introduction 2 Methods 3 Experiments 4 Conclusion Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 9 / 36

10 Phrase-Level Sentiment Analysis Negation local? E.g. not good Things to be considered: longer-distance dependencies? E.g. does not look very good negation of the subject? E.g. no one thinks that s good used to intensify rather than change polarity? E.g. not only good but amazing Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 10 / 36

11 Phrase-Level Sentiment Analysis Things to be considered: Word Sense E.g. Environmental Trust vs. win people s trust Syntactic role of a word E.g. polluters vs. they are polluters Diminishers E.g. little truth, little threat Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 11 / 36

12 What to begin with? Two steps: 1. Is the phrase neutral or polar? 2. If polar: positive? negative? both? neutral? Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 12 / 36

13 Manual Annotation Scheme Corpus for Experiments: Multi-Percpective Question Answering (MPQA) Opinion Corpus The corpus contains: Annotations of subjective expressions (words/phrases to express opinion, emotion,speculation, etc ) The task is: Manually annotate these subjective expressions with their contextual polarity Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 13 / 36

14 Manual Annotation Scheme This means: Interpret the whole sentence! E.g. They have not succeeded, and will never succeed in breaking the will of this valiant people. Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 14 / 36

15 Agreement Study Is this annotation reliable? Agreement Study 2 annotators 10 documents with 447 subjective expressions Results Overall agreement: 82 % Kappa (k): 0.72 Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 15 / 36

16 Corpus 425 documents (8,984 sentences) 28 % no subjective expressions 25 % 1 subjective expression 47 % 2 or more subjective expressions Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 16 / 36

17 Corpus Annotated documents Development Data 66 documents, 2,808 subjective expressions used for data exploration and feature development Training and Test Data 359 documents, 13,183 subjective expressions used in 10-fold cross-validation experiments Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 17 / 36

18 Prior-Polarity Subjectivity Lexicon How to determine prior polarity of a word/phrase? An additional lexicon of subjectivity clues may help Lexicon of over 8,000 subjectivity clues (only single words clues) Words in lexicon are grouped: strongsubj (subjective in most contexts) weaksubj (have only certain subjective usages) Lexicon was expanded with new entries All the clues were tagged (positive, negative, both or neutral) Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 18 / 36

19 Overview 1 Introduction 2 Methods 3 Experiments 4 Conclusion Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 19 / 36

20 Why 2-step approach? Let t use prior polarity alone for identifying contextual polarity! PPC Gold Neut Pos Neg Both Total Neut Pos Neg Both Total Accuracy: 48 % Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 20 / 36

21 Step 1. Neutral-Polar Classification The neutral-polar classifier uses 28 features Document Feature document topic Word Features word token word part-of-speech word context prior polarity reliability class: strongsubj or weaksubj Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 21 / 36

22 Step 1. Neutral-Polar Classification The neutral-polar classifier uses 28 features Modification Features (binary) preceeded by adjective preceeded by adverb preceeded by intensifier is intensifier modifies strongsubj modifies weaksubj modified by strongsubj modified by weaksubj Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 22 / 36

23 Step 1. Neutral-Polar Classification Dependency Tree for the sentence The human rights report poses a substantial challenge to the US interpretation of good and evil. Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 23 / 36

24 Dependency Tree poses subj obj report challenge (neg) det adj mod det adj p The human rights (pos) a substantial (pos) to pobj interpretation det mod p the US of and pobj conj good (pos) conj evil (neg) Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 24 / 36

25 Step 1. Neutral-Polar Classification The neutral-polar classifier uses 28 features Sentence Features strongsubj clues in current sentence: count strongsubj clues in previous sentence: count strongsubj clues in next sentence: count weaksubj clues in current sentence: count weaksubj clues in previous sentence: count weaksubj clues in next sentence: count Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 25 / 36

26 Step 1. Neutral-Polar Classification The neutral-polar classifier uses 28 features Sentence Features adjectives in sentence: count adverbs in sentence: count cardinal number in sentence: binary pronoun in sentence: binary modal in sentence: binary Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 26 / 36

27 Step 1. Neutral-Polar Classification The neutral-polar classifier uses 28 features Structure Features (binary) in subject in copular in passive Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 27 / 36

28 Step 1. Neutral-Polar Classification Results Acc Polar Rec Polar Prec Polar F word token word+priorpol features Neut Rec Neut Prec Neut F word token word+priorpol features Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 28 / 36

29 Step 2. Polarity Classification The polarity classifier uses 10 features Word Features word token word prior polarity: pos, neg, both, neut Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 29 / 36

30 Step 2. Polarity Classification The polarity classifier uses 10 features Polarity Features negated: binary negated subject: binary modifies polarity: pos, neg, neut, both, notmod modified by polarity: pos, neg, neut, both, notmod conj polarity: pos, neg, neut, both, notmod general polarity shifter: binary negative polarity shifter: binary positive polarity shifter: binary Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 30 / 36

31 Dependency Tree poses subj obj report challenge (neg) det adj mod det adj p The human rights (pos) a substantial (pos) to pobj interpretation det mod p the US of and pobj conj good (pos) conj evil (neg) Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 31 / 36

32 Step 2. Polarity Classification Results Positive Polarity Acc Rec Prec F word token word+priorpol features Negative Polarity Rec Prec F word token word+priorpol features Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 32 / 36

33 Step 2. Polarity Classification Results Both Rec Prec F word token word+priorpol features Neutral Rec Prec F word token word+priorpol features Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 33 / 36

34 Overview 1 Introduction 2 Methods 3 Experiments 4 Conclusion Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 34 / 36

35 Conclusion What differs this approach from others? From lexicon of words with prior polarities to instances of these words in corpus and their contextual polarity wide range of features 4 tags: positive, negative, both or neutral What does it bring? The results are significantly better than baseline = Context is important! Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 35 / 36

36 References This presentation was solely based on Th. Wilson, J. Wiebe & P. Hoffmann: Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis 36 / 36