Logistics | Course Info | Syllabus | Other Resources
Logistics
- Lectures: Tue, Thu, 10:30am-11:50am, Gates B1
- Office Hours and Sections: Google Calendar
For SCPD students, please email scpdsupport@stanford.edu or call 650-741-1542.
Instructor
Course Assistants
Charlie Marx
ctmarx [at] stanford.edu
Sofian Zalouk
szalouk [at] stanford.edu
Garrett Thomas
gwthomas [at] stanford.edu
Chaitanya Patel
chpatel [at] stanford.edu
Devansh Sharma
devansh [at] stanford.edu
Course Information
Course Description
Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.
The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover: (1) Bayesian networks, undirected graphical models and their temporal extensions; (2) exact and approximate inference methods; (3) estimation of the parameters and the structure of graphical models.
Prerequisites
Students are expected to have background in basic probability theory, statistics, programming, algorithm design and analysis. If you are able to comfortably complete homework 1 then you likely have all the relevant background knowledge.
Recommended Readings
Corresponding Textbook: (“PGM”) Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman. MIT Press.
Course Notes: Available here. Student contributions welcome!
Lecture Videos: here
Further Readings:
- (“GEV”) Graphical models, exponential families, and variational inference by Martin J. Wainwright and Michael I. Jordan. Available online.
- Modeling and Reasoning with Bayesian Networks by Adnan Darwiche. Available online (through Stanford).
- Pattern Recognition and Machine Learning by Chris Bishop. Available online.
- Machine Learning: A Probabilistic Perspective by Kevin P. Murphy. Available online (through Stanford).
- Information Theory, Inference, and Learning Algorithms by David J. C. Mackay. Available online.
- Bayesian Reasoning and Machine Learning by David Barber. Available online.
Grading Policy
Homeworks (70%): There will be five homeworks with both written and programming parts. Each homework is centered around an application and will also deepen your understanding of the theoretical concepts. Homeworks will be posted on Ed.
Final Exam (30%): There will be a final exam covering the material taught in the course. The final exam will be held March 21st, 3:30-6:30pm.
Extra Credit (+3%): You will be awarded with up to 3% extra credit if you answer other students’ questions on Ed in a substantial and helpful way, or contribute to the course notes on GitHub with pull requests.
Assignments
Written Assignments: Homeworks should be written up clearly and succinctly; you may lose points if your answers are unclear or unnecessarily complicated. You are encouraged to use LaTeX to writeup your homeworks (here is a template), but this is not a requirement.
Homework Submission: All students (non-SCPD and SCPD) should submit their assignments electronically via Gradescope.
Late Homework: You have 6 late days to use at any time during the term without penalty. For a particular homework, you can use only two late days without penalty. Once you run out of late days, you will incur a 25% penalty for each extra late day you use. Each late homework should be clearly marked as “Late” on the first page. For example, if you have four late days available and use them on a single homework, your grade will be reduced by 50% on that homework. If you use two late days each on the first three homeworks, using one late day (your seventh cumulatively) on the next homework will incur a 25% penalty.
Regrade Policy: If you believe that the course staff made an error in grading, you may submit a regrade request through Gradescope within one week of receiving your grade. Please be as specific as possible with your regrade request.
Collaboration Policy and Honor Code: You are free to form study groups and discuss homeworks and projects. However, you must write up homeworks and code from scratch independently without referring to any notes from the joint session. You should not copy, refer to, or look at the solutions from previous years’ homeworks in preparing your answers. It is an honor code violation to intentionally refer to a previous year’s solutions, either official or written up by another student. Anybody violating the honor code will be referred to the Office of Community Standards.
Syllabus
| Week | Dates | Topics | Corresponding Readings | Assignments |
|---|---|---|---|---|
| 1 | Jan. 9 & 11 | Introduction, Probability Theory, Bayesian Networks | PGM Ch. 1-3 | HW 1 (Jan 9 - Jan 23) |
| 2 | Jan. 16 & 18 | Undirected models | PGM Ch. 4 | |
| 3 | Jan. 23 & 25 | Learning Bayes Nets | PGM Ch. 16-17 | HW 2 (Jan 24 - Feb 2) |
| 4 | Jan. 30 & Feb. 1 | Exact Inference; Message Passing | PGM Ch. 9-10 | HW 3 (Feb 1 - Feb 13) |
| 5 | Feb. 6 & 8 | Sampling | PGM Ch. 12 | |
| 6 | Feb. 13 & 15 | MAP Inference; Structured prediction | PGM Ch. 13 | HW 4 (Feb 13 - Feb 27) |
| 7 | Feb. 20 & 22 | Parameter Learning | PGM Ch. 19-20 | |
| 8 | Feb. 27 & 29 | Bayesian Learning; Structure Learning | PGM Ch. 17-18 | HW 5 (Feb 27 - Mar 12) |
| 9 | Mar. 5 & 7 | Exponential families; variational inference | PGM Ch. 8 & 11; GEV Section 3 | |
| 10 | Mar. 12 & 14 | Advanced topics and conclusions |
Many thanks to David Sontag, Adnan Darwiche, Vibhav Gogate, and Tamir Hazan for sharing material used in slides and homeworks.
Additional TA Sessions
Attendence is optional but encouraged. The sections will be held 1:30-2:20pm on Fridays in NVIDIA Auditorium.
- Week 2: d-separation (Jan. 19, 1:30-2:20pm)
- Week 4: Variable elimination (Feb. 2, 1:30-2:20pm)
- Week 5: Junction Tree (Feb. 9, 1:30-2:20pm)
- Week 6: Metropolis Hastings and Gibbs sampling (Feb. 16, 1:30-2:20pm)
- Week 7: EM (Feb. 23, 1:30-2:20pm)
Other Resources
There are many software packages available that can greatly simplify the use of graphical models. Here are a few examples:
The Stanford University Honor Code
The Honor Code (https://communitystandards.stanford.edu/policies-and-guidance/honor-code) is a part of this course. The Honor Code is the university’s statement on academic integrity written by students in 1921. It articulates university expectations of students and faculty in establishing and maintaining the highest standards in academic work. The Honor Code is an undertaking of the students, individually and collectively: that they will not give or receive aid in examinations; that they will not give or receive unpermitted aid in class work, in the preparation of reports, or in any other work that is to be used by the instructor as the basis of grading; that they will do their share and take an active part in seeing to it that others as well as themselves uphold the spirit and letter of the Honor Code. The faculty on its part manifests its confidence in the honor of its students by refraining from proctoring examinations and from taking unusual and unreasonable precautions to prevent the forms of dishonesty mentioned above. The faculty will also avoid, as far as practicable, academic procedures that create temptations to violate the Honor Code. While the faculty alone has the right and obligation to set academic requirements, the students and faculty will work together to establish optimal conditions for honorable academic work.
Students with Documented Disabilities
Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is being made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: http://oae.stanford.edu).
Names and Pronouns
Use the names and pronouns (e.g., they/them, she/her, he/him, just a name, or something else) indicated by your classmates for themselves. If you don’t want to share a set of pronouns for yourself, that is perfectly acceptable, too. If your name or pronouns change during the course, we invite you to share this with us and/or other students, so we may talk with you and refer to your ideas in discussion as you would wish.