NOTE: This page contains papers through 2005. All of our newer annotation studies are about subjectivity and sentiment. Those papers and the data are on my "Subjectivity Analysis, Sentiment Analysis, Opinion Extraction" publications page.

A 10,000 sentence corpus of articles from the world press has been manually annotated for private states (opinions, emotions, etc), including attributes such as their sources and intensities.
This data is available at http://www.cs.pitt.edu/mpqa

Wiebe, Janyce, Wilson, Theresa, and Cardie, Claire (2005). Annotating expressions of opinions and emotions in language. Language Resources and Evaluation (formerly Computers and the Humanities) 1(2)

Wiebe, J. (2002). Instructions for Annotating Opinions in Newspaper Articles. Department of Computer Science Technical Report TR-02-101 , University of Pittsburgh, Pittsburgh, PA.

Wilson, T. (2002). Instructions for Performing Opinion Annotation in the Gate Annotation System

Theresa Wilson and Janyce Wiebe (2003). Annotating Opinions in the World Press. 4th SIGdial Workshop on Discourse and Dialogue (SIGdial-03). ACL SIGdial.
Here is a corrected version of the paper that appears in the proceedings.
Here is the original.

Bruce, Rebecca F. & Wiebe, Janyce M. (1999). Recognizing subjectivity: a case study in manual tagging. Natural Language Engineering 5 (2).

Wiebe, Janyce M., Bruce, Rebecca F., & O'Hara, Thomas P. (1999). Development and use of a gold standard data set for subjectivity classifications. In Proc. 37th Annual Meeting of the Assoc. for Computational Linguistics (ACL-99). Association for Computational Linguistics, University of Maryland, June, pp. 246-253.

Wiebe, Janyce, Bruce, Rebecca, & Duan, Lei. (1997) Probabilistic event categorization. In Recent Advances in Natural Language Processing (RANLP-97). Tsigov Chark, Bulgaria, Sept. 1997, pp. 163-170.

Bruce, Rebecca & Wiebe, Janyce. (1998). Word sense distinguishability and inter-coder agreement. In Proc. 3rd Conference on Empirical Methods in Natural Language Processing (EMNLP-98). Association for Computational Linguistics SIGDAT, Granada, Spain, June, pp. 53-60.

O'Hara, T., Wiebe, J., & Payne, K. Instructions for Temporal Annotation of Scheduling Dialogs. Used for our JAIR-98 paper. Sample annotations are available in This Directory.

Wiebe, Janyce, Maples, Julie, Duan, Lei, & Bruce, Rebecca (1997). Experience in WordNet sense tagging in the Wall Street Journal. In Proc. ANLP-97 Workshop, Tagging Text with Lexical Semantics: Why, What, and How? Association for Computational Linguistics SIGLEX, Washington, D.C., April 1997, pp. 8-11.

Wiebe, Janyce (1997). Writing annotation instructions. Position paper for leading a working session on writing annotation instructions. In Proc. ANLP-97 Workshop, Tagging Text with Lexical Semantics: Why, What, and How?, Association for Computational Linguistics SIGLEX, Washington, D.C., April 1997, pp. 87.

Here is the word-sense data used in Bruce, Rebecca & Wiebe, Janyce (1994). Word-sense disambiguation using decomposable models. In Proc. 32nd Annual Meeting of the Assoc. for Computational Linguistics (ACL-94), pp. 139-146.