- Final exam will be held on Monday, April 18, 2011 at 1:00pm during the class. The exam will focus primarily on the second part of the course, but questions related to the previous material may be included.
- Project presentations will be held on Monday, April 25, at 1:00-4:00pm. Your presentations should be 7 minutes long. Two additional minutes will be reserved for questions. Please check term project guidelines for further details.
- Project reports are due on Wednesday, April 27, 2011 at midnight. The report should be structured like a conference paper or a journal article. The report should include the introduction, methods, results, discussion and conclusions sections. The key learning methods used in the project should be described to sufficient depth so that they are selfexplanatory. The clarity of the report will be one of the criterion to grade the project. Here are some examples of projects students submitted in past:
- Graded homework assignments 8 and 9 can be picked up at TA's office.
- Homework 1 ( program solutions )
- Homework 2
- Homework 3 ( program solutions )
- Homework 4 ( program solutions )
- Homework 5 ( program solutions )
- Homework 6 ( program solutions )
- Homework 7 (program solutions for Problem 2 and Problem 3 )
- Homework 8
- Homework 9
Links
Course description
Lectures
Homeworks
Term projects
Matlab
Abstract
The goal of the field of machine learning is to build computer systems that learn from experience and that are capable to adapt to their environments. Learning techniques and methods developed by researchers in this field have been successfully applied to a variety of learning tasks in a broad range of areas, including, for example, text classification, gene discovery, financial forecasting, credit card fraud detection, collaborative filtering, design of adaptive web agents and others.
This introductory machine learning course will give an overview of many models and algorithms used in modern machine learning, including linear models, multi-layer neural networks, support vector machines, density estimation methods, Bayesian belief networks, mixture models, clustering, ensamble methods, and reinforcement learning. The course will give the student the basic ideas and intuition behind these methods, as well as, a more formal understanding of how and why they work. Students will have an opportunity to experiment with machine learning techniques and apply them a selected problem in the context of a term project.
Prerequisites
Knowledge of matrices and linear algebra (CS 0280), probability (CS 1151), statistics (CS 1000), programming (CS 1501) or equivalent, or the permission of the instructor.
Textbook:
- Chris Bishop. Pattern recognition and Machine Learning Springer, 2006.
- R.O. Duda, P.E. Hart, D.G. Stork. Pattern Classification. Second edition. John Wiley and Sons, 2000.
- C. Bishop. Neural networks for Pattern Recognition. Oxford University Press, 1995.
- J. Han, M. Kamber. Data mining. Concepts and Techniques. 2nd edition Morgan Kauffman.
- T. Mitchell. Machine Learning. Mc Graw Hill, 1997.
Lectures
| Lectures | Topic(s) | Assignments | |
|---|---|---|---|
| January 5 |
Introduction to Machine Learning.
Readings: Bishop: Chapter 1 |
. | |
| January 10 |
Machine Learning: designing a learning system
Readings: Bishop: Chapter 1 |
. | |
| January 12 |
Matlab tutorial
Readings: Bishop: Chapter 1 |
. | |
| January 19 |
Matlab tutorial
Readings: Bishop: Chapter 1 |
Homework assignment 1 ( Data and programs for HW1 ) | |
| January 24 |
Density estimation
Readings: Bishop: Chapter 2.1. |
. | |
| January 26 |
Density estimation
Readings: Bishop: Chapter 2.2-3. |
Homework assignment 2 ( Data for HW2 ) | |
| January 30 |
Density estimation III
Readings: Bishop: Chapter 2 |
||
| February 2 |
Linear regression
Readings: Bishop: Chapter 3.1-2. (optional 3.3-3.5) |
Homework assignment 3 ( Data for HW3 ) | |
| February 7 |
Classification model learning.
Readings: Bishop: Chapter 4.1-2. |
||
| February 9 |
Classification model learning II.
Readings: Bishop: Chapter 4.1-2. |
Homework assignment 4 ( Data for HW4 ) | |
| February 14 |
Support vector machines
Readings: Bishop: Chapter 4.1.2-4, Chapter 7. |
||
| February 16 |
SVMs for regression. Multilayer neural networks.
Readings: Bishop: Chapters 7,5. |
Homework assignment 5 ( Data for HW5 ) | |
| February 21 |
Multi-way classification. Decision trees.
Readings: Bishop: 4.2, 4.3.4., 7.1.3. |
||
| February 23 |
Bayesian belief networks
Readings: Bishop: Chpater 8.1-8.2. |
||
| February 28 |
Bayesian belief networks. Inference and parameter learning.
Readings: Bishop: 8.2, 8.4. |
Homework assignment 6 ( Data for HW6 ) | |
| March 2 | Midterm exam
Readings: Bishop: |
||
| March 14 |
Learning with hidden variables and missing values. Expectation maximization.
Readings: Bishop: Chapter 9 |
||
| March 16 |
Clustering.
Readings: Bishop: Chapter 9 |
Homework assignment 7 ( Data for HW7 ) | |
| March 21 |
Learning structure of BBN. Dimensionality reduction: PCA
Readings: Bishop: 12.1. |
||
| March 23 |
Dimensionality reduction: Feature selection
Readings: Bishop: |
Homework assignment 8 ( Data for HW8 ) | |
| March 28 |
Ensemble methods: mixture of experts, bagging
Readings: Bishop: Chapter 14.
|
||
| March 30 |
Ensemble methods: boosting
Readings: Bishop: Chapter 14. |
Homework assignment 9 ( Data for HW9 ) | |
| April 4 |
Reinforcement learning
Readings: Kaelbling, Littman, Moore. Reinforcement Learning: a survey |
||
| April 6 |
Reinforcement learning: RL with delayed rewards
Readings: Kaelbling, Littman, Moore. Reinforcement Learning: a survey |
||
| April 11 |
Concept learning. PAC learning.
Readings: |
||
| April 18 | Final exam | ||
| April 25 | Project presentations at 1:00-4:00pm | ||
Homeworks
The homework assignments will have mostly a character of projects and will require you to implement some of the learning algorithms covered during lectures. Programming assignmets will be implemented in Matlab. See rules for the submission of programs.
The assignments (both written and programming parts) are due at the beginning of the class on the day specified on the assignment. In general, no extensions will be granted.
Collaborations:
No collaboration on homework assignments, programs, and exams is permitted unless you are specifically
instructed to work in groups.
Term projects
The term project is due at the end of the semester and accounts for a significant portion of your grade.
Matlab
Matlab is a mathematical tool for numerical computation and manipulation,
with excellent graphing capabilities. It provides a great deal of support
and capabilities for things you will need to run Machine Learning
experiments. The CSSD at UPitt offers $5 student licenses for Matlab. To obtain
the licence please check the following link to the
Matlab CSSD page .
In addition, Upitt has a number of Matlab licences running on both
unix and windows platforms. See the following web
page for the details.
Other Matlab resources on the web:
Online
MATLAB documentation
Cheating policy:
Cheating and any other anti-intellectual behavior, including giving your work to someone else, will be dealt
with severely and will result in the Fail (F) grade. If you feel you may have violated the rules
speak to us as soon as possible. Please make sure you read, understand and abide by
the Academic Integrity Code for the Faculty and College of Arts and Sciences.
Students With Disabilities: Course webpages from Spring 2010, Spring 2007, Spring
2004 and Spring 2003
Online
Mathworks documentation including MATLAB toolboxes
If you have a disability for which you are or may be requesting an
accommodation, you are encouraged to contact both your instructor and
Disability Resources and Services, 216 William Pitt Union, (412)
648-7890/(412) 383-7355 (TTY), as early as possible in the term. DRS
will verify your disability and determine reasonable accomodations for
this course.
Last updated by Milos
on 12/31/2009