ECE 531: Detection and Estimation
University of Illinois at Chicago, ECE
Spring 2011

Instructor: Natasha Devroye, devroye@ece.uic.edu
Course coordinates: Tuesday, Thursday from 2-3:15pm in Lincoln Hall 103.

Office hours: Tuesdays from 3:30-4:30pm and Thursdays 5-6pm in SEO 1039, or by appointment

Welcome to ECE 531! This course is a graduate-level introduction to detection and estimation theory, whose goal is to extract information from signals in noise. A solid background in probability and some knowledge of signal processing is needed.

Course Textbook: Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory, by Steven M. Kay, Prentice Hall, 1993 and (possibly) Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory, by Steven M. Kay, Prentice Hall 1998.
Other useful references:
Harry L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, II, III, IV
H. Vincent Poor, Introduction to Signal Detection and Estimation
Louis L. Scharf and Cedric Demeure, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis
Carl Helstrom, Elements of Signal Detection and Estimation. It's out of print, so here's my pdf copy.
A nice survey on the EM algorithm

Notes: I will follow the course textbooks fairly closely, using a mixture of slides (highlighting the main points and with nice illustrations) and more in-depth blackboard derivations/proofs in class. I will post a pdf version of the slides as they become ready here, but the derivations will be given in class only.

Topics: Estimation Theory:
General Minimum Variance Unbiased Estimation, Ch.2+Ch.3, and Chapter 5 notes
Cramer-Rao Lower Bound, Ch.3
Linear Models+Unbiased Estimators, Ch.4 and Ch. 6 notes
Maximum Likelihood Estimation, Ch.7 notes
Least squares estimation, Ch.8 notes
Bayesian Estimation, select Ch.10-12 notes
Kalman filtering, select Ch.12-13 notes
Detection Theory:
Statistical Detection Theory, Ch.3 notes
Deterministic Signals, Ch.4< notes
Random Signals, Ch.5 notes
Statistical Detection Theory 2, Ch.6 notes

Grading: Weekly homeworks (15%), Exam 1 = max(Exam1, Exam 2, Final) (20%), Exam 2 = max(Exam 2, Final) (20%), Project (20%), Final exam (25%).

Homework: Will be handed out each Thursday, due the next Thursday (1 week). All assignments from HW2 onwards MUST be submitted electronically as a latex file, and a printed pdf copy handed in during class. I will create the solutions from the best solutions I receive (with credit to the authors!). Submit the latex file via the appropriate assignment # on the Blackboard site. Please make sure the "Title" you enter is of the form "HW#_Student_Name" (e.g. HW4_Natasha_Devroye). You can find latex resources for Windows and for MAC by googling around. Finally, here is a template for you to use for homework submissions.
HW1: out 1/13 due 1/20 -- Book 1, problems 2.1, 2.3, 2.8, 2.9, Solutions
Hw2: out 1/20 due 1/27 -- Book 1, problems 3.3, 3.11, 3.15 Solutions
HW3: out 1/27 due 2/3 -- Book 1, problem 4.6, 4.13, 5.3, 5.9 Solutions
HW4: out 2/3, due 2/10 -- Book 1, problem 6.7, 6.9, 7.3, 7.14 Solutions
HW5: out 2/17, due 2/24 -- Book 1, problem 7.18, 7.20, 8.5, 8.10 Solutions
HW6: out 2/24, due 3/3 -- Book 1, problem 8.20, 8.27, 10.3 Solutions
HW7: out 3/3, due 3/10 -- Book 1, problem 11.3, 12.1, 12.11 Solutions
HW8: out 3/10, due 3/17 -- Book 1, problem 13.4, 13.12, 13.15 Solutions
HW9: out 3/31, due 4/7 -- Book 2, problem 3.4, 3.6, 3.12, 3.18 Solutions
HW10: out 4/7, due 4/14 -- Book 2, problem 4.6, 4.10, 4.19, 4.24 Solutions
HW11: out 4/14, due 4/21 -- Book 2, problem 5.14, 5.17, 6.2 Solutions

Exams: For midterm 1, you may have one 8.5x11 double-sided sheet which you can fill with anything you like. No other books, notes or calculators. For midterm 2, you may have 2 of these crib sheets and for the final exam you may have 3 such crib sheets.
Practice midterm1 2009, Midterm 1 2009, Pratice midterm2 2009, Midterm 2 2009, Practice final 2009

Project: The project, to be done individually, will conists of an in-depth study of an implementation of detection and estimation principles. The goal is to explore contemporary research topics in the area of detection, estimation and generally statistical signal processing that are not covered in class. Pick (or suggest) a topic of interest to you and provide a comprehensive treatment of it: introduce the problem/topic, survey what has been done by whom on the topic (we expect many citations to relevant journal and conference papers), implement (in Matlab, C, whatever works for you) the detection/estimation technique for various scenarios and make a demo, to be performed and explained live in class during your presentation, that illustrates its performance. You may compare various methods (e.g. different tracking algorithms), or may look at your implementation under a variety of conditions. The goal is to show a deep understanding of the subtleties of that detection/estimation scheme, where is it useful, its limitations and strengths, and some of the nitty-gritty details which you did not expect to encounter. As the applications of detection and estimation theory span several fields, there is no single journal in which to look for articles. One useful resource is IEEE Xplore where you can search many of the relevant IEEE journals such as IEEE Transactions on Signal Processing, IEEE Transactions on Wireless Communications, IEEE Transactions on Antennas and Propagation, IEEE Transactions on Information Theory, and all sorts of other ones. More recent developments are often found in conference proceedings, which are sometimes also found on IEEE Xplore, or sometimes by navigating conference websites. Again, as detection and estimation spans a variety of fields, there is no unique relevant conference but some relevant ones include the IEEE Workshop on Statistical Signal Processing, IEEEE ICASSP, Asilomar Conference on Signals, Sysems and Computers, ISIT, Globecom, IEEE Radar Conference, and many more! Plagiarism -- or copying someone else's work or code -- is not permitted and will be dealt with according to UIC policy, see UIC's policy. Some help with bibtex and more. Here's a template which you can use if you like. It's in the standard IEEE format for journal articles. You'll need the IEEEtran document class (IEEEtrans.cls needs to be in the directory in which your file is), which you can download and read more about here.
The project will consist of 3 parts: 1) a 5-8 pages, single spaced latex 11-12pt report, 2) a live demo, of your implementation, or comparison between different implementations/techniques, and 3) a presentation in front of the class which will introduce the class to your selected topic through slides and the live demo, which should be 15 minutes long. More details will be given during the term.
Project grading: You will be graded on the quality of your written report, your live demo, and your presentation. Possible project topics: the EM algorithm and its applications, the Kalman filter and (one or more of) its applications, spectral estimation, white-spaces detection, distributed detection and estimation, sensor fusion, sequential detection and estimation, applications in your domain of interest (biology, image processing, optics, etc.), Markov Chain Monte Carlo, particle filters, all aspects of radar signal processing (detection, tracking, SAR imaging). Your project must have an implementable component illustrating the key ideas and performance.
Project timeline: Choose topic (2/23), make a list of relevant papers, skeleton of the paper, outline of the demo (3/29), hand-in final paper to entire class -- will be posted so other students can read before presentation (4/19), in-class presentations and demos (4/21, 4/26).

Important dates (subject to change):
1/11: First class, introduction. Introduction slides
1/13: Minimum variance unbiased estimation and the Cramer-Rao Lower Bound.
1/18: Cramer-Rao Lower Bound.
1/20: Cramer-Rao Lower Bound, Linear Model.
1/25: Linear Model.
1/27: General MVUE.
1/28: MAKEUP lecture for 2/8, to be held in LH 103 from 4-5pm.
2/1: General MVUE.
2/3: Best Linear Unbiased Estimator (BLUE).
2/8: Devroye out of town, class cancelled.
2/10: Midterm 1.
2/15: Maximum likelihood estimation (MLE).
2/17: MLE.
2/22: MLE
2/24: MLE, Least Squares Estimation. Deadline to select project topic
3/1:
3/3: Selected topics Ch.10-12, Bayesian estimation.
3/8: Bayesian estimation.
3/10: Kalman filtering.
3/15: Midterm 2.
3/17: Detection theory - intro to statistical detection theory.
3/22: Spring Break, no class
3/24: Spring Break, no class
3/29: Detection theory, chapter 3 continued. Project outline due.
3/31: Detection theory - Deterministic Signals Ch 4.
4/5: Ch.4 on Detection of Deterministic Signals in Gaussian noise continued.
4/7: Ch.5 on Detecion of Random Signals in Gaussian noise.
4/12: Ch 5 + Ch.6 on Statistical Decision Theory II.
4/14: Ch.6 on Statistical Decision Theory II.
4/19: Buffer. Written project report due.
4/21: Last class Review notes.
4/26: Project presentations.
4/28: Project presentations.
Final exam week 5/2 - 5/6