ELEC 531: Statistical Signal Processing

Rice University, Fall 2004

Instructor: Clay Scott
Classroom: Sewell 462
Office: Duncan Hall 2117
Campus extension: 3776
Email: cscott at rice dot edu
Office hour: Fri. 11-12

Feedback form

Course assistants:

Name	office hour	email	office number
Wai Lam (William) Chan	Tues. 2-3	wailam	DH 2046
Jyoti Uppuluri	Mon. 1-2	juppu	DH 2121

Required text: none

Recommended texts: (on reserve at Fondren)
1) Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory, by Steven Kay, 1993
2) Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory, by Steven Kay, 1998

Another helpful text: (on reserve at Fondren)
1) Statistical Signal Processing, Louis Scharf, 1991

Prerequisites

Basic probability: familiarity with densities, probability mass functions, expected value, mean and variance, independence, conditional distributions, characteristic function. ELEC 533 or equivalent taken concurrently.
Basic linear algebra: vector spaces, linear transformations, inner products, orthonormality. Familiarity with projections, the SVD, and least-squares problems will be useful, but not essential. We'll develop these concepts as necessary.
Digital signal processing: Nothing essential, but familiarity with filtering of linear time-invariant systems and the DFT may aid understanding.
Programming: Basic MATLAB skills (can learn these along the way). Here is a quick tutorial. There will be times when I provide you with MATLAB code to make your life easier.

If you feel underprepared, talk with me.

Final Grade

Homework: 50%
Class participation 5%
Midterm exam: 10%
Final project: 15%
Final exam: 20%

Homework Policy

There will be in the range of 10-12 homeworks, about one per week. If you need to turn an assignment in late, please contact me in advance. Unexcused late homeworks will not be accepted.

I consider homeworks to be the most important part of the class. When writing up your problem sets, you are expected to

write in complete and coherent sentences. Uncommented strings of equations will not cut it. If a grader has trouble comprehending your work, it may be checked wrong.
make rigorous, mathematically precise arguments whenever possible.
write legibly
add comments to any code that you are asked to hand in, and compose the code using the stylistic elements of structured programming (descriptive variable/function names, indenting nested operations)

Topics to be covered

The Linear Model

Linear least squares
Polynomial signals
Sinusoidal signals
Linear signal subspaces

Estimation Theory

Maximum likelihood and the EM algorithm
Minimum variance unbiased estimators (Rao-Blackwell theorem, Cramer Rao lower bound, BLUE)
Bayesian estimators (MAP, MMSE)
Linear Bayesian estimators (Wiener filter, Kalman filter)

Detection Theory

Detection criteria (Bayes risk, Prob. of error, Neyman-Pearson)
The likelihood ratio test (LRT)
Key examples: signal constellations and the matched filter, binary symmetric channel
Detection with unknown signal parameters (UMP tests, Karlin-Rubin theorem, GLRT, Bayes factor)

Adaptive Filtering (time permitting)

LMS algorithm
Particle filters
The El Farol problem

Collaboration and the Honor Code

You may work together on homeworks, but you are required to write up/code your solutions by yourself. You may not refer to material from previous offerings of this course, including problem sets and solution sets. If you find a problem worked in a book or on the web, resist the temptation to copy it.

Students with Disabilities

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

Problem sets and related files

Description	Problems	Solutions
HW 1	hw1.pdf	sol1.pdf
HW 2	hw2.pdf	sol2.pdf
HW 3	hw3.pdf	sol3.pdf
HW 4	hw4.pdf	sol4.pdf
HW 5	hw5.pdf	sol5.pdf
HW 6	hw6.pdf	sol6.pdf
HW 7	hw7.pdf	sol7.pdf
HW 8	hw8.pdf	sol8.pdf
HW 9	hw9.pdf	sol9.pdf
HW 10	hw10.pdf	sol10.pdf
HW 10	hw11.pdf	sol11.pdf

q.m (Q-function)

qinv.m (Inverse Q-function)

Final Project

Project description

Files: fmri.mat, ref.mat, fmri.m.

A paper that may stimulate your thinking (note: this paper is more sophisticated that what I would expect you to do): Generalized likelihood ratio detection for fMRI using complex data , Nan, F.Y.; Nowak, R.D., IEEE Transactions on Medical Imaging, Volume: 18 Issue: 4 , April 1999 Page(s): 320 -329

2003 project description

Additional resources

Last summer Prashant made a nice applet for viewing and understanding different probability density and mass functions. This is a good resource for those of you who might be less familiar with certain distributions we'll be using. It allows you to play with the different parameter settings of a density/mass function and see how the shape of the function changes. The instructions for viewing the applet are listed below. You need a special program called LabVIEW, which you can download, in order to view the applet.

To run the applet, you first need to install the LabVIEW Run-Time Engine and the Connexions VI Runner (just one installer for the pair). Download and install setup_plus_rte.exe from http://cnx-246-91.ece.rice.edu/~prash/installers and you should be ready to go. If you're on a computer that's already got LabVIEW, you should install setup_minus_rte.exe. Once that's installed, go to http://cnx-246-91.ece.rice.edu/~prash/apps.html and click on the Common Probability Distributions link. If all goes well, the VI Runner should download and run the VI for you. The installation information is also posted at http://cnx.rice.edu/content/m11550/latest/