Polarity Annotation Instructions: ================================================== Section 1: Overview We are annotating the polarity for both the ons and the expressive subjective elements (ESEs) in context. See Wiebe 2002 for information on ons and ESEs. Briefly, a private state is an internal mental or emotional state, including beliefs, emotions, evaluations, goals, and judgments. A speech event is a speaking event. This includes both speaking and writing. The source of a speech event is the writer or speaker. The source of a private state is the experiencer of the private state. Ons are explicitly mentioned private states or speech events: I hate Bill. John thought he won the race. Mary planned for her presentation. ESES are private states that are implicitly expressed by the wording, and is commonly used in sarcasm, irony, negative evaluation, and frustration: What a jerk! He has a few screws loose. He never even bothered to call me. Polarity refers to the type of emotion, judgment, or evaluation that is expressed by the source. Is anything positive or negative being expressed? I'm happy that she won. (positive emotion of the nested source "writer,I") He is a bad baseball player. (negative evaluation by nested source "writer,He") The question to ask when annotating for polarity is: According to the source, what is the polarity of the on or ESE in the given context? Tagging the polarity of ons is further discussed later. Briefly, we look at whether the on directly refers to a private state and judge the polarity according to the private state. What is meant by "directly refers to a private state" is also discussed later. For instances where a word is marked both as an ESE and an on, keep in mind they may have different sources and may be expressing different polarities. Take the following contrived example: Ex: John's speech was highly technical. He blathered on and on about CPUs, processors, and the MIPS assembly language. The ESE "blathered" (with the source, the writer) is negative. The on "blathered" does not directly refer to any private state of John. It is therefore neutral. Note that "blathered" doesn't imply that according to the writer the person blathering has a private state that is negative, positive, or both. Blathered is a speech event. ================================================== Section 2: Tags Polarity can have the following values for both ons and ESEs: 1) Neutral - the on or ESE does not express anything positive or negative. 2) Negative - the on or ESE only expresses something negative. 3) Positive - the on or ESE only expresses something positive. 4) Both - the on or ESE express a mixture of things, some that are positive others that are negative. In addition to each of these tags, there are uncertain-neutral, uncertain-negative, uncertain-positive, and uncertain-both. Use these only if you are unsure of the tag. In these cases, make your best guess as to the polarity of the on or ESE. In GATE, the feature to select when annotating is "polarity." ================================================== Section 3: The Meaning Of The Tags The following is to give you a better idea of what is meant by the "positive" and "negative" polarity. A private state directly referred to by an on or ESE can express multiple attitudes. The following example is taken from "Instructions for Annotating Opinion Types and Targets" (Wilson, 2004): I think people are happy because Chavez has fallen. ON: happy There are two attitudes here expressed for the on "happy" by the source, "people": - a positive attitude that Chavez fell. - a negative attitude towards Chavez. Each attitude has a particular attitude type. In the previous example we had two attitude types. To determine the polarity of the on "happy", we combine the polarities of the attitude types expressed by the on or ESE in order to determine the polarity of the on. The following lists the polarities of the various attitudes: Attitude Type Measure of Intensity Polarity Tag of Attitude --------------------------------------------------------------------- Positive attitude - degree of positiveness positive Negative attitude - degree of negativeness negative Intention - degree of determination neutral Not-intention - degree of determination neutral Arguing - degree of certainty or neutral strength of belief Uncertainty - degree of uncertainty neutral or speculation Other - degree of ____ neutral The measure of intensity refers to the scale with which we measure the strength or intensity of an attitude type. For example, "I promise to send you the paper tomorrow" has a a higher degree of determination than "I will will try to send you the paper tomorrow." Not-intention refers to a person's intent not to do something: "I don't plan to finish my homework tomorrow." After determining the polarity of attitudes expressed by the ons or ESEs, we combine the polarities of the different attitudes and assign the polarity to the on or ESE in the following manner: 1) Neutral - there are no positive or negative attitudes. 2) Positive - there are positive attitudes, possibly neutral attitudes also. 3) Negative - there are negative attitude, possibly neutral attitudes also. 4) Both - there are positive and negative attitudes, possibly neutral attitudes also. See "Instructions for Annotating Opinion Types and Targets" (Wilson, 2004) for more details on the attitude types. In summary, the task of determining the polarity of an on or ESE can be viewed as determining the attitudes expressed by the on or ESE and then using the polarities of the attitudes to determine what polarity should be assigned to the on or ESE. ================================================== Section 4: The Importance Of Marking Ons and ESEs In Context For ons and ESEs, we are looking at the private state that is "most directly referred to" by the on or ESE. This is the private state that is most focused by the on. Judge the on and ESE, in context and according to the source of the private state. The following are examples illustrating the importance of marking the polarity in context: A) Felipe loves ice cream. B) Mary thought Bill was a jerk. In A) the on "love", expresses a private state about loving ice cream. It has a positive polarity. In B) the on "thought" is also a private state (cognition). In this thinking private state, Mary has a negative attitude towards Bill. It should be marked as negative. This on is different from that in A) in that the context is critical in determining the polarity of private state. We need to know what Mary is thinking about to determine if it is positive or negative. But even words that are usually associated with positive or negative polarity are dependent on context: Ex: He hoped Sarah would lose the race. There is a positive attitude towards Sarah losing the race and a negative attitude towards Sarah. ================================================== Section 5: Specific Instructions For Marking Ons For ons, don't mark: 1) ons with the is-implicit attribute 2) ons with onlyfactive=yes 3) ons with onlyfactive=no and on-strength=neutral Ex: Bill, said "What a jerk!" We do not mark ons with on-strength=neutral because they are used to report a privates state, but are not a part of the private state itself. The on "said" in the last example is only-factive=no and on-strength=neutral. It should not be marked. While it reports Bill's private state, it is not part of the private state. If an on that is a speech event expresses a private state, mark the polarity for that private state: Ex: Mary criticized Bill for his actions. The on "criticized" is part of the private state expressing Mary's negative attitude towards Bill. It is marked as negative. ================================================== Section 6: General Suggestions For Tagging Following this section are several example sentences that have the polarity of the ons and ESEs marked. It is recommended that you go through these to get a feel for how we are tagging polarity. Examples 1-6, 8-11, 18-22 provide a context for the sentence being tagged (typically a sentence before and after the sentence to be tagged) as well as the annotations for each of the ons and ESEs in the sentence. Examples 7, 12-17, 23 and onward do not have any context for the sentence being tagged and/or do not have not all of the ons or ESEs that are tagged for sentences. The gold standard tag decided on are labeled as "TAG." Usually provided with the tag is an explanation on how the polarity of the word was judged. Below are some guidelines to keep in mind when tagging: 1) In addition to positive and negative attitudes, an on or ESE may also have attitudes expressing intentions, not-intentions (ex: "don't intend"), arguing, and uncertainty. Use the guidelines mentioned in Section 3. Examples of ons/ESEs that have a combination of attitude types for which the polarity is neutral (the number preceding the on or ESE is the example which the on or ESE is found in Section 7): 1 ("But though") 3 ("But") 4 ("a big deal", "guaranteed") 7 ("may") 8 ("But for Washington", "the whole point of") 19 ("thousands of investors") 20 ("defended", "affirming") 27 ("to understand") 33 ("You could understand") 38 ("are contemplating") Examples of emphasis with polarity tag neutral: 24 ("including an 11-year-old girl") 39 ("first and only") Examples of ons/ESEs that have a combination of attitude types for which the polarity is other than neutral: 5 ("urged", "needs to cooperate") 9 (may well") 10 ("That may be so") 22 ("constant aggressive attitude") Examples of cognition, argument with polarity 26 ("would realize") 27 ("but to understand") 28 ("think") 31 ("likened") 2) Mark the polarity that is explicitly mentioned for the given ESE or on in context. Don't mark polarity that is inferred. This is analogous to the distinction made for private states in section 2.4 of "Instructions for Annotating Opinions in Newspaper Articles" (Wiebe, 2002). Examples where polarity might be inferred but is not explicitly stated: 2 ("guaranteed") 6 ("offer private recommendations") Examples where polarity is stated explicitly: 2 ("are now clamoring") 10 ("no business") 3) Look at some of the examples that give an idea for what should be tagged as "both". Take into consideration all of the positive and negative attitudes that are explicitly mentioned for the given ESE or on in the given context. Don't consider positive or negative attitudes that are inferred. Examples which should not be tagged as both: 2 ("guaranteed") 6 ("offer private recommendations") 13 ("want") 14 ("wants") 21 ("seeks") Examples which should be tagged as both: 2 ("are now clamoring") 12 ("want") 15 ("want") 16 ("wants") 17 ("good and evil") 18 ("agreed") 26 ("would realize") 4) Various interesting polarity examples: 28 ("frigid") 28 ("tropical") 32 ("leeway") 35 ("significant") 36 ("hardly qualify") 37 ("may have erred") 40 ("more precise") 40 ("more clearly") 29 ("had "no complaints") 30 ("don't care") 23 ("expressed sympathy") 23 ("political stances that are totally biased") ================================================== Section 7: Examples 1) DOCID: non_fbis/06.29.16-13721 The 110 al-Qaeda and Taliban prisoners admitted to "Gitmo" by the end of last week are, said Rumsfeld, "the hardest of the hard core," men who had killed "dozens and dozens of people." But though it may lack tennis courts and a putting green, the amenities are better than you'd find in a cave at Tora Bora. True, prisoners are now confined to 6-ft. by 8-ft. chain-link enclosures with concrete floors and tin roofs (Rumsfeld thinks it's "pejorative" to call them cages). ESE: But though SOURCE: w TAG: neutral. While "But though" may give a clue that certain words have a certain polarity, it is used for arguing for a world view. ====== 2) DOCID: non_fbis/06.29.16-13721 Three detainees are said to have claimed to be British citizens. Politicians and commentators in London are now clamoring that all held in Gitmo must be guaranteed treatment in accordance with international law. The Daily Telegraph, a paper usually so conservative that it makes Pat Buchanan look vegetarian, warned Washington of the need to draw a "distinction between civilized society and the apocalyptic savagery of those who would destroy it." ON: are now clamoring SOURCE: w,thoseinlondon TAG: both. "Clamoring" suggests a negative attitude of the politicians and commentators. The politicians and commentators have a positive attitude towards guaranteeing treatment according to international law. ESE: must SOURCE: w,thoseinlondon TAG: neutral. The word "must" may give a clue that there is an argument, but it does not have any polarity. ESE: guaranteed SOURCE: w,thoseinlondon TAG: positive. You might want infer that, in addition to the positive polarity, there is a negative attitude towards the current state of how Gitmo prisoners are being treated. Don't. ====== 3) DOCID: non_fbis/06.29.16-13721 That may well apply to members of al-Qaeda, a free-floating band of terrorists. But not all of those at Gitmo are al-Qaeda men. Some--the Pentagon won't say how many--were members of the Taliban and presumably thought they were part of the Afghan army. ESE: But SOURCE: w TAG: neutral. "But" is used as part of an argument for a world view. ====== 4) DOCID: non_fbis/06.29.16-13721 The last seems a distinct possibility; the Pentagon plans to build 2,000 cells at Camp X-Ray. "This will be a big deal down there for at least two years, guaranteed," says Army Lieut. General B.B. Bell, who commands Fort Hood, Texas, the base from which military police have been deployed to Cuba. But it's hard to find a justification for such detention in the Geneva Convention or anywhere else. ESE: a big deal SOURCE: w,bell TAG: neutral. The army lieutenant is arguing for his world view and not stating whether building the cells is positive or negative. ESE: guaranteed SOURCE: w,bell TAG: neutral. "Guaranteed" is used to expressed his certainty that the Pentagon's plans will be a big deal. ====== 5) DOCID: temp_fbis/20.45.09-11809 At a meeting of several officials held at the outset of the summit, the President first took up economic issues and then referred to his "axis of evil" remark. He urged Japan to follow the US by saying: "The international community needs to cooperate to change Iran, Iraq, and North Korea's action pattern (of spreading weapons of mass destruction). All options (including military action) are on the table." ON: urged SOURCE: w,bush TW: uncertain-positive JW: positive PH: neutral TAG: positive. While "Urged" argues for a world view, there is also a positive emotion also present, that it would be good for Japan to follow the US. ESE: needs to cooperate SOURCE: w,bush TAG: positive. Bush is arguing for cooperation and has a positive attitude towards cooperation. ====== 6) DOCID: non_fbis/06.29.16-13721 So far, humanitarian groups have been muted in their criticism of the conditions at Guantanamo. Last week a delegation from the International Committee of the Red Cross arrived to inspect the camp and offer private recommendations on its operation. But in the European press, the prisoners' lot has become a public issue of contention among those who demand that U.S. conduct be above suspicion. ON: offer private recommendations SOURCE: w,redcrossdel TAG: positive. Don't label it as both. You might be tempted to infer that the positive recommendations might infer a negative attitude towards "its operation," but this is not directly mentioned in the text. Don't mark it. ====== 7) DOCID: non_fbis/06.29.16-13721 The Pentagon has resisted calling the detainees prisoners of war, preferring the terms unlawful combatants or battlefield detainees. It's easy to see why. Under the Geneva Convention, those holding true POWs are bound to release them at the end of hostilities; but that is the last thing the U.S. wants to do with men who may be al-Qaeda operatives. Moreover, by convention (though the law seems to be murky here) POWs don't need to tell their captors anything other than their name, rank, serial number and birthday. ESE: but SOURCE: w TAG: neutral. ESE: last thing SOURCE: w TAG: neutral. The writer is characterizing the negative attitude that the US may have. It is not their positive or negative attitude. ON: wants SOURCE: w,us TAG: negative. The US has a negative attitude towards releasing men who might be al-Qaeda operatives. ESE: may SOURCE: w TAG: neutral. "May" is speculative in context. ====== 8) DOCID: non_fbis/06.29.16-13721 Moreover, by convention (though the law seems to be murky here) POWs don't need to tell their captors anything other than their name, rank, serial number and birthday. But for Washington, the whole point of the detention is to conduct interrogations and thus head off new acts of terrorism. The Geneva Convention does contemplate that some irregular forces captured in battle need not be considered POWs. ESE: But for Washington SOURCE: w TAG: neutral. The writer is arguing for a word view of the purpose for Washington's detention. ON: the whole point of SOURCE: w,us TW: uncertain-both JW: neutral PH: neutral TAG: neutral. "The whole point" is signaling an argument for a world view. ====== 9) DOCID: non_fbis/06.29.16-13721 The Geneva Convention does contemplate that some irregular forces captured in battle need not be considered POWs. That may well apply to members of al-Qaeda, a free-floating band of terrorists. But not all of those at Gitmo are al-Qaeda men. ESE: may well SOURCE: w TAG: positive. "May well" is conceding that it may be the case that the Geneva Convention applies to some al-Qaeda members. ====== 10) DOCID: non_fbis/06.29.16-13721 Washington says no, because the Taliban had no clear chain of command and was not a legitimate government. That may be so; unfortunately, as Amnesty International has pointed out, under the Geneva Convention the Pentagon has no business making such a determination. Those who fall into the enemy's hands are entitled to POW status until a "competent tribunal" has determined their status. ESE: That may be so SOURCE: w TAG: positive. This is a difficult example. In addition to speculation as to whether or not the Taliban has a clear chain of command and a legitimate government, there is also a positive attitude conceding that the views of Washington could be right. ESE: unfortunately SOURCE: w TAG: negative. "Unfortunately" expressed the negative attitude of the writer. ESE: no business SOURCE: w,ai TAG: negative. ESE: such a SOURCE: w TAG: negative ====== 11) DOCID: temp_fbis/20.45.09-11809 However, Koizumi said, "I take the remark as the indication of the President's strong resolve to fight terrorism." The US has repeatedly suggested the possibility of its taking military action against the government of Iraqi President Saddam Husayn. It is possible that if the US military attack Iraq, Japan, which has shown understanding of the axis of evil remark, will be asked to provide active support to the US as it did during US attacks on Afghanistan. ON: repeatedly suggested SOURCE: w,US TAG: positive. Like "recommendation", "suggested" has a positive polarity. According to the US, the suggestion of taking military action is positive. ESE: the possibility SOURCE: w,US TAG: positive. The US has a positive attitude towards the possibility of taking military action. ====== The use of the "both" tag 12) DOCID: 20020219/20.52.48-16582 This leaves Iraq, which the Washington circles want to antagonize and to go to war with. ON: want SOURCE: w, washington TAG: both. According to the writer, Washington has a positive attitude towards antagonizing and going to war with Iraq and a negative attitude toward Iraq. ====== 13) DOCID: temp_fbis/20.45.09-11809 In reply, the President said, "We want to resolve all issues peacefully, and we intend to continue diplomatic efforts." However, it is unclear whether Japan can actually prevent the US from taking military action. ON: want SOURCE: w,bush,US TAG: positive. The President has a positive attitude towards resolving issues peacefully. ====== 14) DOCID: non_fbis/06.29.16-13721 Under the Geneva Convention, those holding true POWs are bound to release them at the end of hostilities; but that is the last thing the U.S. wants to do with men who may be al-Qaeda operatives. ON: wants SOURCE: w,us TAG: negative. The U.S. does not want to release men who may be al-Qaeda operatives ====== 15) DOCID: temp_fbis/21.37.46-9337 Those that want to secede from the nation and engage in terrorist activities are terrorists, not any particular ethnic group," said Ma Dazheng, director of the Xinjiang Uygur Autonomous Region's Development Research Institute at the Chinese Academy of Social Sciences. ON: want SOURCE: w,ma,those TAG: both. A positive attitude toward seceding and a negative attitude towards the nation. The nation is mentioned in the text. ====== 16) DOCID: temp_fbis/21.24.00-10191 He says he wants to destroy the "infrastructure" behind the suicide attacks against the Israelis. ON: wants SOURCE: w,sharon,sharon TAG: both. A positive attitude toward the destruction and a negative attitude towards the infrastructure behind the suicide attacks. ====== 17) DOCID: 20020219/20.52.48-16582 Besides, politicians refer to good and evil only for purposes of intimidation exaggeration. ESE: good and evil SOURCE: nested-source=w, politicians TAG: both. "Good" is positive, "evil" is negative. ====== 18) DOCID: 20020207/20.22.42-14586 He argued: "When the United States has faced a situation involving extremist groups holding hostages, they have sent intermediaries to negotiate and no one has said that this shows they have relations with those groups." [Caracas El Nacional (Internet version) in Spanish on 7 February carries a related 760-word item that says that some opposition deputies agreed that the government's tactless actions are to blame for the concerns voiced by US Foreign Secretary Colin Powell. The report cites Convergence Deputy Alejandro Arzola saying that Powell's position clearly reflects the concern that exists in the international community and Latin America over the Venezuelan situation: the atmosphere of conflict and the threats against freedom of expression, democracy, and legal security. ON: agreed SOURCE: w, 760, somedep TAG: both. A positive attitude towards the belief that the government's actions are to blame, a negative attitude toward the government's tactless actions. ESE: tactless actions SOURCE: w, 760, somedep TAG: negative. ESE: are to blame SOURCE: w, 760, somedep TAG: negative. ON: the concerns voiced SOURCE: w, 760, somedep, powell TAG: negative. ====== 19) DOCID: 20020207/20.22.42-14586 The report cites Convergence Deputy Alejandro Arzola saying that Powell's position clearly reflects the concern that exists in the international community and Latin America over the Venezuelan situation: the atmosphere of conflict and the threats against freedom of expression, democracy, and legal security. Arzola stated: "The government's positions are not only harming the country at the domestic level, but also at the international level because thousands of investors are watching on their television sets what is happening in Venezuela, where the rule of law is under fire." Arzola added: Any recommendation by a country friendly to Venezuela, especially a country that is Venezuela's No.1 trade partner, must be heeded, as well as recommendations by the Church and the nation, which is demanding a rectification. ON: positions SOURCE: w, arzola, venezuela TAG: neutral. ESE: not only harming SOURCE: w, arzola TAG: negative. ESE: because SOURCE: w, arzola TAG: neutral. ESE: thousands of investors SOURCE: w, arzola TAG: neutral. What is expressed by "thousands of investors" is emphasis, not polarity, using in arguing for a world view. ESE: the rule of law is under fire SOURCE: w, arzola TAG: negative. ===== 20) DOCID: 20020207/20.22.42-14586 All this has sounded the alarm for the US Executive and suits the opposition's objectives to bring about an intervention or a blockade, which would hurt the people it claims to defend." Oddly, he defended the content of the video by affirming - as the President did - that this is evidence of a humanitarian action. He argued: "When the United States has faced a situation involving extremist groups holding hostages, they have sent intermediaries to negotiate and no one has said that this shows they have relations with those groups." ESE: Oddly SOURCE: w TAG: neutral. "Oddly" is a characterization by the writer of Carreno's defending, but is not clearly positive or negative. ON: defended SOURCE: w, carreno TAG: neutral. Carreno is arguing for a world view that the video was evidence of a humanitarian action. It is not explicitly telling us whether Carreno has a positive or negative attitude towards the content in the video. ON: affirming SOURCE: w, carreno TAG: neutral. His affirming the evidence does not give whether Carreno has a positive or negative towards the evidence, but rather arguing that it is the case that it was evidence of a humanitarian action. ====== 21) DOCID: 20020207/20.22.42-14586 We demand equal treatment for our foreign policy." The opposition seeks intervention or blockade MVR Deputy Pedro Carreno went even further and established a direct relation between the concerns voiced by the Secretary of State and the disclosure of a video that presumably confirms relations between the Venezuelan Government and the Colombian rebel groups. ON: seeks SOURCE: w, opposition TAG: positive. A positive attitude towards "intervention" or "blockade". Don't mark it as negative because of "intervention" or "blockade" is negative. Only mark polarity that is tied to a target explicitly mentioned in the text. (Directly named, as a pronoun, as part of one of its arguments, etc.) ====== 22) DOCID: 20020207/20.22.42-14586 At present, because of the misguided policy of cutting production, we are only selling them 1.1 million barrels. If to this, we add the President's constant aggressive attitude and readiness to establish ties between Venezuela and parasitic economies that give us nothing, but fulfill his political commitments, we will very likely continue to lose our edge in the US market." According to Fermin, the chief of state still does not understand that if there is no confidence, there will be no investments. ESE: If SOURCE: w, fermin TAG: neutral. ESE: constant aggressive SOURCE: w, fermin TAG: negative. ON: constant aggressive attitude SOURCE: w, fermin, chavez TAG: negative. According to the writer, Chavez has an aggressive attitude, a private state which is negative. ON: readiness to establish ties SOURCE: w, fermin, chavez TAG: positive. ESE: parasitic economies SOURCE: w, fermin TAG: negative. ESE: give us nothing SOURCE: w, fermin. TAG: negative. ESE: very likely continue to lose our edge SOURCE: w, fermin TAG: negative. ====== 23) DOCID: 20010914/22.04.48-17941 Nobody who has a live conscience and human feelings, whether he is Palestinian or otherwise, could not have been moved by these events and expressed sympathy for the families of the US victims, regardless of the US political stances that are totally biased to Israel and Israel's use of the most advanced US weapons to curb the Palestinian intifadah. ON: expressed sympathy SOURCE: w, nobody TAG: negative. This is a difficult example. In this case the source "nobody" is sympathizing with the families of US victims. That is, "nobody" is sharing in (or feeling) the grief, anger, shock, or other negative emotions that the families are experiencing. So in sympathizing, "nobody" is also experiencing a negative emotion. Reference: http://www.m-w.com ON: political stances that are totally biased SOURCE: w, us TAG: positive. Another challenging example! In this case, the US has a positive attitude (a bias) towards Israel and Israel's use of weapons in curbing the Palestinian intifadah. You might be tempted to also say that the US also has a negative attitude towards the Palestinian intifadah. Don't! This is inferring too much. ====== 24) DOCID: 20010914/22.04.48-17941 While the world is busy following the news of the attacks on the US targets and feeling sorry for these acts, Israel carried out an incursion into Janin and the neighboring villages, bombarded more Palestinian internal security sites, demolished more homes in East Jerusalem, and killed an additional number of Palestinian activists and non-activists, including an 11-year-old girl. ESE: including an 11-year-old girl SOURCE: w TAG: neutral. Emphasis. You might be tempted to mark this as negative, but it is used is to add support to an argument that is negative through its role of adding additional information. ====== 25) DOCID: 20010914/22.04.48-17941 It is very important for the United States to realize the difference between the US and Israeli cases. ESE: It is very important SOURCE: w TAG: Neutral. Arguing for world view. ON: to realize SOURCE: w, us GOLD: Neutral. In realizing the difference between the US and Israeli cases, this is not positive or negative. One might want to infer that it is good for the writer or for the people he is speaking on the behalf of (the Palestineans), but we are only talking about the US here. ====== 26) DOCID: 20010914/22.04.48-17941 This is because the Palestinians were, and still are, hoping that the Americans would realize, if they have not already realized, the justice of the Palestinian cause and the enormous injustice suffered by the Palestinians, who were expelled from their homeland, whose land was occupied, and who were deprived of the minimum political and human rights. ON: would realize SOURCE: w, pals, americans TAG: Both. In realizing that the Palestinian's cause is just and the injustice that the Palestinians have suffered, the US would have a positive attitude towards the Palestinian's cause and a negative attitude towards the injustice. Note: Refer to the previous example for a case where realize is neutral. ====== 27) DOCID: 20010914/22.04.48-17941 The Palestinians want nothing from Washington but to understand their cause and stand beside right and justice. ON: want nothing SOURCE: w, pals TAG: Positive. The Palestinians have a positive attitude towards US understanding their cause and standing beside right and justice. ESE: but to understand SOURCE: w TAG: Positive. ON: to understand SOURCE: w, pals, wash TAG: Neutral. Positive too subtle. ====== 28) DOCID: non_fbis/12.44.14-20223 AFTER SPENDING months in frigid Afghanistan caves subsisting on Allah knows what, the 158 detainees the United States is holding in tropical Guantanamo Bay must think they've died and gone to heaven. ESE: frigid SOURCE: w TAG: Negative. According to the writer, "frigid" is negative (people don't want to spend time in a frigid cave). ESE: Allah knows what SOURCE: w TAG: Negative. Used to express negative attitude. Sarcasm? ESE: tropical SOURCE: w TAG: Positive. According to the writer, "tropical" is positive (it is much better than the "frigid" caves) ESE: must SOURCE: w TAG: Neutral. ON: think SOURCE: w, detainees TAG: Positive. ====== 29) DOCID: non_fbis/12.44.14-20223 Also, a British Foreign Ministry team, which spent three days at Club Gitmo, reported that three British citizens being held there had "no complaints about their treatment." ON: had "no complaints SOURCE: w,bfmteam,brits TAG: Neutral. There is an absence of complaints. There isn't anything that is stating that they had a positive attitude towards their treatment. ====== 30) DOCID: non_fbis/12.44.14-20223 And they apparently don't care if the criticism makes them look clueless as to what really is inhumane. ESE: apparently SOURCE: w TAG: Neutral. Evaluation. ON: don't care SOURCE: w,somepeople TAG: Neutral. Here, the absence of caring if the criticism does not mean they have a negative attitude towards something. By doing this, one would infer too much. ====== 31) DOCID: non_fbis/12.44.14-20223 First, there's the reporter who likened living in the tropics without air conditioning to "torture." ESE: First SOURCE: w TAG: Neutral. Part of arguing. ON: likened SOURCE: w,rep1 TAG: . GOLD: Negative. The reporter is using the comparison of the "tropics without air" to "torture" to express a negative attitude. ====== 32) DOCID: non_fbis/12.44.14-20223 The detainee status however, gives the U.S. military the leeway to interrogate these former operatives about any future terrorist attacks. ESE: however SOURCE: w TAG: Neutral ESE: leeway SOURCE: w TAG: Positive. According to writer would be good. ====== 33) DOCID: non_fbis/12.44.14-20223 You could understand the outrage if the U.S. military were torturing or otherwise mistreating the detainees. ESE: You could understand SOURCE: w TAG: Neutral. Arguing. ON: could understand SOURCE: w,you TAG: Neutral. Understanding the outrage doesn't suggest that "you" has a positive or negative attitude. ====== 34) DOCID: 20020304/20.50.30-97 I am convinced that improving the environment through technological progress can actually enhance our competitiveness and economic growth. ON: am convinced SOURCE: w, wall, wall TAG: Neutral. By being convinced, one understands or strongly believes something to be the case. It is being used as support for an argument. ====== 35) Referring to President Bush's climate change plan for the US that he announced on 14 February the Commissioner noted: "These proposals will not lead to a reduction of greenhouse gas emissions in the United States but allow a significant increase. ON: These proposals SOURCE: w, wall, bush TAG: Neutral. ESE: significant SOURCE: w, wall TAG: Negative. According to Wall, it would be bad if there were a significant increase in greenhouse gas emissions. ====== 36) DOCID: non_fbis/06.12.31-26764 There are many countries in the world that hardly qualify as pluralistic democracies. ESE: hardly qualify SOURCE: w TAG: Negative. The writer is expressing a negative attitude towards "many countries in the world." ====== 37) DOCID: non_fbis/06.12.31-26764 On this count, the national security adviser may have erred. ESE: On this count SOURCE: w TAG: Neutral. ESE: may have erred SOURCE: w TAG: Negative. It is bad to make an error. ====== 38) DOCID: non_fbis/06.12.31-26764 One can name more than a dozen countries which are contemplating the acquisition of mass destruction weapons at one speed or another. ESE: One can name SOURCE: w TAG: Neutral. ON: are contemplating SOURCE: w,countries TAG: Neutral. Your inferring too much if you say it is negative because contemplating the acquisition of mass destruction weapons implies there is a negative attitude towards something. ESE: at one speed or another SOURCE: w TAG: un GOLD: Negative. Sarcastic qualitic of document. Difficult. ====== 39) DOCID: non_fbis/06.12.31-26764 Moreover, the US is the first and only nation ever to deploy such weapons. ESE: Moreover SOURCE: w TAG: Neutral. ESE: first and only SOURCE: w GOLD: Neutral. Emphasis. The evaluation is supporting the writer's argument. ====== 40) DOCID: non_fbis/06.12.31-26764 If this is the case, Rice could have elaborated more precise criteria for membership in the "axis of evil," by establishing more clearly the connection between the nature of the regime and the acquisition of weapons of mass destruction. ESE: If this is the case SOURCE: w TAG: Neutral. ESE: more precise SOURCE: w TAG: Positive. ESE: more clearly SOURCE: w TAG: Positive. ====== 41) DOCID: 20020217/20.53.10-28693 Wu said he believed overall Taiwan interests were protected from knocks by the Bush administration because of the island's "strong support" from decision-makers in the White House, State Department and Department of Defense. ON: believed SOURCE: w, Wu, Wu TAG: Neutral. Speculation. ESE: protected from knocks SOURCE: w, Wu, Wu TAG: Positive. The decision makers in the White House (the Bush administration) are not going to "knock" Taiwan's interests. ON: strong support SOURCE: w, Wu, Wu, leaders TAG: Positive. ================================================== Section 8: References Wiebe, J. (2002). Instructions for Annotating Opinions in Newspaper Articles. Department of Computer Science Technical Report TR-02-101 , University of Pittsburgh, Pittsburgh, PA. Wilson, T. (2004). Instructions for Annotating Opinion Types and Targets.