Practical Probabilistic Modeling with Graphical Models

EE 639 Advanced Topics in Signal Processing and Communication

Fall 2009

[Home] [Lectures] [Homework]

News

8/31	We are going to move effectively today. Please refer to the new location and time of the lecture below.
8/26	Visit course website at http://www.vis.uky.edu/~cheung/courses/ee639/index.html and our discussion group at http://groups.google.com/group/ee639

Professor

Dr. Sen-ching Cheung (cheung at engr.uky.edu)

Office: FPAT 687B (x7-9113)

Office hours: MWF 9-11am

Office: Room 831 VisCenter at Kentucky Utility Building (7-1257 ext. 80299)
Office hours: By appointment only

Schedule

Regular class: MW 3:00pm-4:15pm (Relocated to the small conference room 869 at VisCenter)

Final Examination: Take-home final (will be issued on 12/14, due 12/16)

Course Description

A central tenant of any empirical sciences is to construct probabilistic models for prediction and estimation based on available data. The enormous advances in computing, sensing and networking technologies provide us with an unprecedented capability in collecting and storing an inordinate amount of data. Much of these data are noisy, inter-related and high-dimensional. For decades, mathematicians and engineers in different disciplines have developed specialized probabilistic models to characterize and utilize various types of data. One particular framework has gradually emerged as the most appropriate tool to unify disparate techniques and to build complex models in a modular and algorithmic fashion. This framework is the Probabilistic Graphical Model, the focus of the EE639 course this semester.

Probabilistic graphical models are probabilistic models that encode local information of conditional independency among a large number of random variables. By choosing an appropriate (sparse) graph to describe the data, powerful and rigorous techniques exist to perform prediction, estimation, data-fusion as well as handling uncertainty and missing data. Many classical multivariate systems in pattern recognition, information theory, statistics and statistical mechanics are special cases of this general framework – examples include hidden Markov models, regression, mixture models, Kalman filters and Ising models. In this course, we will study how graph theory and probability can be elegantly combined under this framework to represent many commonly-used probabilistic tools, and how they can be applied in solving many practical problems. We will put equal emphasis on both theoretical understanding of the subject and practical know-how on using it in real applications. The grading is based on three components: a number of homework assignments throughout the semester, a take-home final and a final project.

Tentative Topics

Basic Probability and Statistics
Introduction to Graphical Models
Simple Graphical Models: Linear Classification and Regression
Parameter Estimation for Completely Observed Graphical Models
Expectation Maximization: Parameter Estimation for Incomplete Graphical Models
Exact Inference on Graphical Models
Factor Analysis, Hidden Markov Model and Kalman Filtering
Approximate Inference on Graphical Models
~~Kernel Methods and Sparse Kernel Machines~~
Applications: Error Control Coding, Image Segmentation, Speech Processing and others.

Grading

Your grade will be based on:
Homework	40%
Take-home Final (due 12/16)	20%
Final Project (poster session and report due 12/4)	40%

The letter grade assignment is based on the following scale: from 100 to 90 pts => A, from 89 to 80 pts. => B, from 79 to 70 pts => C, from 60 to 69 pts. => D, from 59 to 0 pts. => E.
Homework will be assigned throughout the semester. A random question in each homework will be graded. Late homework will not be accepted.
Each student must complete a final project applying graphical models to research problems or surveying latest research areas in graphical models. All findings will be presented in a poster session and summarized in a final report.
Each student must also complete a take-home final within a specific time-limit without consulting other people or sources beyond those readings included in this course.
I have a zero-tolerance policy for all forms of plagiarism, from duplicating one sentence from an un-referenced source in your report, copying a homework answer from your friend or solutions from previous terms to cheating in the exams. Not only you will lose all the points for that assignment, the incident will also be reported to the Department Chair who will determine the appropriate disciplinary action.

Required Text

Preprints of chapters from “An Introduction to Probabilistic Graphical Models" by M. I. Jordan and relevant research papers will be available in the password-protected portion of this website.

Prerequisites:

Required: Undergraduate level linear algebra (matrix operations), multivariate calculus, probability (random variables, discrete and continuous, especially Gaussian, distributions) and statistics (confidence level and hypothesis testing).
Desired: Basic understanding of graph theory, stochastic processes, information theory and optimization.
Students will need to be familiar with Matlab, S+, R or a related matrix-oriented programming language.

Sen-ching Samson Cheung

Last modified: August 16, 2009.