ECE 271B - Statistical Learning II



This is a second course in statistical learning. It complements 271A, covering the topic of discriminant methods for statistical learning. Since discriminant methods are quite different from the generative methods covered in 271A, the latter is not a pre-requisite for 271B.

Topics covered include: linear discriminants; the Perceptron; the margin and large margin classifiers; learning theory; empirical vs structural risk minimization; the VC dimension; kernel functions; reproducing kernel Hilbert spaces; regularization theory; Lagrangian optimization; duality theory; the support vector machine; boosting; Gaussian processes, applications.

There are no exams, course evaluation is based on homework and a project to be jointly determined by the student and instructor.

Lectures: Tu-Th, 12:30-1:50 PM, CSB 005
Instructor: Nuno Vasconcelos
n u n o @ e c e . u c s d . e d u, EBU1-5603
office hours: Fridays 9:30-11:00 AM
Teaching Assistant: Hamed Masnadi-Shirzai (hmasnadi @ u c s d . e d u)
Office hours: Mondays 1-2:30 PM room 5512 EBU1
Syllabus: [ps, pdf]
Homework: Problem set 1 [ps, pdf]
Problem set 2 [ps, pdf]
Problem set 3 [ps, pdf]

Problem set 4 [ps, pdf]

Project: Proposal guide [ pdf]
Readings: Lecture 1: Introduction [slides]
Lecture 2: Optimization [slides]
Lecture 3: Dimensionality reduction [slides]
Lecture 4: PCA and LDA [slides]
Lecture 5: The Rayleigh Quotient [slides]
Lecture 6: Linear discriminants [slides]
Lecture 7: The perceptron and margin [slides]
Lecture 8: Neural networks [slides]
Lecture 9: Kernels [slides]
Lecture 10: Dot product kernels [slides]
Lecture 11: Reproducing kernel Hilbert spaces [slides]
Lecture 12: Regularization and the representer theorem [slides]
Lecture 13: The KKT conditions and duality theory [slides]
Lecture 14: Duality theory [slides]
Lecture 15: Support vector machines [slides]
Lecture 16: Soft-margin support vector machines [slides]
Lecture 17: Boosting [slides]
Lecture 18: VC dimension
Lecture 19: Structural risk minimization
Lecture 20: Project presentations
Lecture 20: Project presentations