PKU Class 2025 Fall: Optimizaiton for Deep Learning

Instructor: Kun Yuan (kunyuan@pku.edu.cn)

Teaching assistants:

• Yilong Song (2301213059@pku.edu.cn) [Leading Teaching Assistant]
  
• Mian Xiao (xiaom@stu.pku.edu.cn)
  
• Ruoxi Yu (2301111452@stu.pku.edu.cn)
  

Classroom: 3pm - 6pm Tuesday, 理教206

Office hour: 4pm - 5pm Thursday, 静园六院220

References

Martin Jaggi and Nicolas Flammarion, Optimization for Machine Learning, EPFL Class CS-439
Chris De Sa, Advanced Machine Learning Systems, Cornell CS6787
Zaiwen Wen, Optimization Methods, PKU 2024 Fall
Kun Yuan, Introduction to LLM, PKU 2025 Spring

Announcement

1. The lecture scheduled for September 9 has been canceled. It will be rescheduled for September 18, from 6:40 p.m. to 9:30 p.m., in Room 407 of the Third Teaching Building (三教407).

Materials

Lecture 1: Introduction

• Warm up: Preliminary [Notes]
  
• Part I: Introduction to Deep Learning [Slides]
  
• Part II: Introduction to LLM [Slides]

Lecture 2: Basics in Machine Learning and Langugae Models

• Part I: Basics in Machine Learning [Slides]
  
• Part II: Basics in Language models [Slides]

Lecture 3: Transformer

• Part I: Seq2Seq; Attention; Transformer [Slides]
• Part II: Parameters, Computations, and Memory Costs in Transformer [Slides01] [Slides02]