CS 2731 / ISSP 2230: Introduction to Natural Language Processing (Fall 2020)
Professor
 Dr. Diane Litman
TA
 Nhat Tran (nlt26 at pitt.edu)
When & Where TuTh 1:15-2:30, via Zoom (https://canvas.pitt.edu/courses/47042) or 630 William Pitt Union
Office Hours Litman: Th 2:30-3:30 via class Zoom, by advance appointment Tu 9-10
Tran: We 3:00-4:30pm and Fr 10:00-11:00am via https://pitt.zoom.us/j/8491488727
Description This course provides an introduction to the field of natural language processing - the creation of programs that can understand, generate, and learn languages used by humans. It will expose students to applications by means of computational techniques including dynamic programming, hidden markov models, probalistic grammars, and machine learning algorithms.
Prerequisites: CS 1501 (algorithms) OR consent of the instructor
Textbook: Speech and Language Processing (3rd edition online draft - free!)
Required Work (tentative!!!) Homeworks (36%): 3 total - written and programming
Exams (30%): final
Project (30%): code, presentation and written report
Participation (4%): discussion boards, other activities
CourseMirror (2% Extra Credit): submitting reflections
Late Penalty: For assignments that may be accepted late, the penalty is 2.5% per day up to 5 days including Saturday, Sunday, and holidays. Assignments are due by 11:59pm.
Date/Topic/Activities (Synchronous)
Readings/Videos (Asynchronous, before class)

Other Links; Assignments
August 20, 25
Introduction		 Ch 1
Video: ACL 2020 McKeown Keynote
Reading: NLP is chasing the wrong goal		 CourseMIRROR: download app, submit reflections
Canvas discussion board
I. Words
August 25, 27
Text Normalization		 Ch 2 (2.1-2.4) Unix for Poets, pages 1-19
regular-expressions.info
September 1, 3
Language Modeling with N-Grams		 Ch 3 (3.1-3.4)
Before 9/1:
  • Videos: #12-14, links in Canvas module
  • Reading: The Social Impact of NLP
    • Before 9/3:
  • Videos: #15-17
    		• HW1: assigned 9/3, due 9/22
    September 8, 10, 15, 17
    Part-of-Speech Tagging     		Ch 8 (8.1-8.4.6)
    Before 9/8:
  • J&M 8.1-8.3
  • Video: #56
    Before 9/10:
  • J&M 8.4
  • Video: HMM/Viterbi
    Before 9/13:
  • Reading: Case Study of AAE
    		• Reading Discussion: due 9/14
    II. Syntax
    September 17, 22, 24
    Constituency Grammars and Parsing     		Ch 12 (through 12.5), 13
    Before 9/15:
  • Videos 58-60
    Before 9/17:
  • Videos 61-65
    		•
    September 24, 29
    Statistical Constituency Parsing     		Ch 14 (14.1-14.5, 14.8)
    Before 9/22:
  • Reading: Case Study of Gender Bias
    		• Reading Discussion: due 9/23
    HW2 (see Canvas): assigned 9/24, due 10/8
    III. Machine Learning
    September 29, October 1
    Naive Bayes Classification (and Sentiment)     		Ch 4 (through 4.8)
    Before 9/29:
  • Videos 24-28
    Before 10/1:
  • Videos 29-32
    By 10/12:
  • Reading: Is Your Classifier Actually Biased?
    		•  Reading Discussion: due 10/12
    October 6, 8, 13
    Logistic Regression     		Ch 5 (5.1-5.2; concepts in 5.3-5.7)
    Before 10/6:
  • Videos 38-40
  • Ch. 21.3 (Crowdsourcing)
    Before 10/8 (optional but recommended):
  • Videos 41-43
    By 10/19:
  • Reading: Language from police body camera footage shows racial disparities in officer respect
    		• Reading Discussion: due 10/19
    October 13, 15, 20
    Representation Learning (and Vector Semantics)     		Ch 6
    Before 10/13
  • J&M 6.1-6.3, Videos 87, 90
    Before 10/15
  • J&M 6.4-6.7, Video 91
    Before 10/20
  • J&M 6.8-6.13
    		• HW3 (see Canvas): assigned 10/15 due 10/29
    Word2Vec Tutorial
    Reading Discussion: due 10/26
    October 22, 27
    Neural Nets (and Language Models)

    Contextual Embeddings

    		Ch 7 (skip 7.4) (before 10/22)

    (before 10/27) Contextual Word Representations: Putting Words into Computers
    (before 10/27) BERT (Bidirectional Encoder Representations from Transformers)

    IV. Semantics
    October 29, November 3, 5, 10
    Word Senses and WordNet     		Ch 19
    Videos 88-89     		Introduction to Story Cloze Project (Corpus Paper, Shared Task Paper)

    Reading Discussion: due 11/4

  • Is GPT-3 Intelligent?
  • A robot wrote this entire article. Are you scared yet, human?
  • Building Gmail style smart compose with a char ngram language model

    Overview article on word and sense similarity (November 2019)

    • November 10, 12
    Information Extraction     		Ch 18 (18.1-2)
    Videos 44-51     		25 Years of IE (November 2019)
    V. Discourse and Applications
    November 17, 19, 24, December 1
    Dialogue Systems and Chatbots     		Ch 26 Reading Discussion: due 11/18
    December 1, 3 Project Presentations
    Assigned November 30, Due December 4 Take-Home Final Exam