198:535 Pattern Recognition
Spring 2006
Instructor:
Casimir Kulikowski
Cuorse meeting Time: 6:30-9pm
Venue: Hill 254
Office : CoRE 333
Office hours: By Appointment
TA: Akshay Vashist
Office hours: Wednesdays 11am-1pm, CoRE 346
Course Description
The course will cover the fundamental in the statistical theory of
classifier design for analysis of complex experimental data. Starting with classical
models and methods of statistical decision-making and prediction, clustering, and
pattern recognition, it will relate them to current approaches in machine learning,
data mining, multi-criteria decision making, optimization, and their applications.
There will be weekly homeworks for the first 2/3 of the class, and a closed-book
midterm exam. Students will also carry out projects developing software for the
analysis of synthetic and real-world data. The course will draw on material from several
texts as well as papers from the recent literature.
References
Primary
[DHS] R.O. Duda, P.E. Hart and D.G. Stork:
Pattern Classification ,
2nd edition, John Wiley & Sons, 2001.
Others
[HTF] T. Hastie, R. Tibshirani and J. Friedman:
The elements of Statistical Learning: Data Mining, Inference and Prediction,
Springer-Verlag, 2001.
[WK] S.M. Weiss and C.A. Kulikowski:
Computer Systems that learn,
Morgan Kaufmann, San Mateo, California, 1991.
[KF] K. Fukunaga:
Introduction to Statistical Pattern Recognition,
2nd edition Academic Press, New York 1990.
[SW] S. Watanabe:
Pattern Recognition: Human and Mechanical ,
John Wiley & Sons, New York 1985.
Preliminary Schedule
| Date/Week |
Topics
References/Reading
|
HW |
| Week 1 (Jan. 17) |
Organization of the class
Intro to PR, clustering & related fields.
Intro to Bayesian decision theory.
[DHS 2.1 & 2.2]
|
HW1
Prob. 2.1 - 2.4 DHS
Due: Jan. 24 |
| Week 2 (Jan. 24) |
Bayesian Decision Theory - relation to minimax
and hypothesis testing (Neyman-Pearson).
Multivariate Normal densities and their discriminants.
Error probabilities and bouds [DHS 2.3-2.8]
Estimation of performance [WK]
|
|
| Week 3 (Jan. 31) |
Binary and other discriminant features.
Bayesian Belief Networks.
Introduction to Maximum Likelihood
[DHS 2.9-2.11 & 31.-3.2]
|
|
| Week 4 (Feb. 7) |
Bayesian parameter estimation,
sufficient statistics and complexity
[DHS 3.3-3.8]
|
|
| Week 5 (Feb. 14) |
PCA, MDA, IDA, EM, HMMs
[DHS 3.9-3.10 & selections from Ch. 10]
|
|
| Week 6 (Feb. 21) |
Performance analysis and algorithm independent ML
[DHS Ch. 9]
|
|
| Week 7 (Feb. 28) |
Non-parametric techniques for density and
posterior probabilities, series expansion approximations
[DHS Ch. 4]
|
|
| Week 8 (Mar. 7) |
Linear discriminant functions and perceptrons.
[DHS 5.1-5.5]
|
|
| Mar. 14 |
SPRING BREAK
|
|
| Week 9 (Mar. 21) |
Other descnet procedures.
[DHS 5.6-5.10]
|
|
| Week 10 (March 28) |
Support Vector Machines
[DHS 5.11-5.12]
|
|
| Week 11 (Apr. 4) |
Multilayer neural networks.
[DHS Ch. 6]
|
|
| Week 12 (Apr. 11) |
Stochastic and Evolutionary methods
[DHS Ch. 7]
|
|
| Week 13 (April 18) |
Unsupervised learning and clustering.
[DHS 10.1-10.9]
|
|
| Week 14 (Apr. 25) |
Validity, dimensionaltiy reduction.
[DHS 10.10-10.14]
|
|
| Week 15 (May 2) |
Project Presentations
|
|