The goal of this course is to cover advanced research topics about conversational AI systems and review founding papers as well as recent work in task-oriented dialog systems, open-domain and social conversational systems, and conversations with embodied systems. We will review previous work on component-wise approaches, as well as end-to-end systems based on large language models (such as, OpenAI’s GPT4), and discuss where they converge and diverge.
The target audience is graduate students who plan to or are already working on these topics. If our time permits, I also plan to invite leading researchers in this field to present guest lectures.
I plan to present background work on conversational systems during the first two weeks, then we will review a set of papers on related topics, and these papers will be presented by students. Given the size of the class, I expect everyone will present one paper in class, and we will have 3 paper presentations for these weeks. Each week, students will also submit a review for one of the papers to be presented that week, and the choice of which one to review will be done by the student. In addition, students are expected to propose and work on a research project in one of these areas; we will discuss the proposals and progress throughout the course. These projects will be group projects of 3 students. In addition to working on their final projects, students will prepare a project report paper and will do a peer review of others’ proposals and project reports.
Course Information:
Lecture times: Wed & Fri, 2pm – 3:15pm
Location: Siebel 1302
Instructor Information:
Dilek Hakkani-Tür
email: dilek@illinois.edu
Office: SC 3304
Office Hours: Mon and Thu, 1-3pm (Office hours will be on zoom. Please get an appointment using the Calendly link I share in class and on canvas, and you will get a zoom link.)
Teaching Assistant:
Suyu Ge
email: suyuge2@illinois.edu
Office hours: Fridays 4-5pm (Zoom link for the office hour is shared on the class canvas page)
Grading:
- Paper Presentation: 10%
- ~10 Paper Reviews: 30% in total
- Final Project: 50%
- Proposal write-up 10%
- Final project write-up 25%
- Code & Demo 5%
- Presentations 10% (2 presentations in total: proposal and final)
- Project report peer review: 10%
Absence Policy: In person participation is mandatory. However, circumstances (e.g., serious illness and family emergency) occasionally occur where you may need to miss a class. Please email me and Suyu before the class, if you cannot be present during the class.
Course page on canvas: https://canvas.illinois.edu/courses/49769
Piazza link for discussions: https://piazza.com/illinois/spring2024/cs598dht (The access code for signing up to piazza is on the canvas page for the course.)
SCHEDULE:
Note that the schedule is subject to change.
Check Canvas for specific due dates and times of all assignments. You can also find the papers I will review and candidate papers I suggest for student presentations on the course Canvas page.
Week 1: Introduction
August 28: Course Introduction
- Class Goals and Logistics
- Brief history of conversational systems
- Types of conversational systems
- Overview of the topics we will cover in this class
Aug 30: Introduction to task-oriented dialogue systems (TODS)
- Spoken Language Understanding
- Dialogue State Tracking (DST)
Week 2: Task Oriented Dialogue Systems
Sept 4: Continuing review of SLU and DST in TODS
Sept 6: Continuing review of TODS
- Dialogue Management
- Learning dialogue policies
- Response Generation from Semantic Representations
Week 3: Task Oriented Dialogue Systems
- End-to-end Dialogue models
Week 4: Open Domain Dialogue Systems
Week 5: Augmented LLMs
Week 6 (Oct 2 & 4): Final Project Proposal Presentations
Week 7: Instruction Tuning
Week 8: Reasoning
Week 9: Multi-agent and reasoning
Week 10: Alignment, Preference Learning and Learning from Feedback
Week 11: Embodied Conversational AI
Week 12: Web Agents and multi-modal dialogues
Week 13: Dialogue Safety
Weeks 14 and 15 (December 4, 6, & 11): Final project presentations