CS 189 at UC Berkeley
Introduction to Machine Learning
Lectures: 2-3:30 pm Mon-Wed online (Berkeley Academic Guide page)
Week 0 Overview
Welcome and Introduction
- Discussion 0 (solution)
- Homework 0 (zip, datahub)
- Slides 8/26 (download)
Week 1 Overview
Foundations: Regression, Classification, and Learning & Features and Regularization
- Note 2 : Linear Regression
- Note 3 : Feature Engineering
- Discussion 1 (solution)
- Homework 0 (zip, datahub)
- Homework 1 (zip, datahub)
- Slides 8/31 (download)
- Slides 9/2 (download)
Week 2 Overview
Foundations: Validation and Generalization
- Note 3 : Feature Engineering
- Discussion 2 (solution)
- Homework 1 (zip, datahub)
- Homework 2 (zip, datahub)
- Slides 9/9 (download)
Week 3 Overview
Foundations: The probability perspective & Core tradeoffs
- Note 4 : MLE and MAP for Regression
- Note 6 : Multivariate Gaussians
- Discussion 3 (solution)
- Homework 2 (zip, datahub)
- Homework 3 (zip, datahub)
- Slides 9/14 (download)
- Slides 9/16 (download)
- Slides 9/18 (download)
Week 4 Overview
Foundations: Kernel perspective & The nearest-neighbor perspective
- Note 5 : Bias-Variance Tradeoff
- Note 7 : MLE and MAP Pt. 2
- Note 8 : Kernels and Ridge Regression
- Discussion 4 (solution) (solution PDF)
- Homework 3 (zip, datahub)
- Homework 4 (zip, datahub)
- Slides 9/21 (download)
- Slides 9/23 (UPDATED) (download)
Week 5 Overview
Foundations: Core tradeoffs revisited --- a unified view of under- and over-parameterized learning & Eigenspace perspectives
- Note 8 : Kernels and Ridge Regression
- Note 9 : Total Least Squares
- Note 10 : PCA
- Discussion 5 (solution)
- Homework 4 (zip, datahub)
- Homework 5 (zip, datahub)
- Slides 9/23 (UPDATED) (download)
- Slides 9/28 (Slides) (download)
- Slides 9/28 (Notes) (download)
- Slides 9/30 (Slides) (download)
- Slides 9/30 (Notes) (download)
Week 6 Overview
Foundations: Dimensionality reduction and latent spaces & Feature selection and the role of sparsity
- Note 9 : Total Least Squares
- Note 10 : PCA
- Note 11 : CCA
- Discussion 6 (solution)
- Homework 5 (zip, datahub)
- Homework 6 (zip, datahub)
- Slides 10/5 (Notes) (download)
Week 7 Overview
Gradient descent and its role & Beyond linear models: neural networks
- Note 12 : Optimization
- Note 13 : Gradient Descent
- Note 14 : Neural Networks
- Note 15 : Training Neural Networks
- Discussion 7 (solution)
- Homework 6 (zip, datahub)
- Homework 7 (zip, datahub)
- Slides Backprop (10/12) (download)
- Slides 10/12 (download)
- Slides SGD Notes (10/14) (download)
- Slides 10/14 (download)
Week 8 Overview
Stochastic gradient descent & Natural losses for classification
- Note 12 : Optimization
- Note 13 : Gradient Descent
- Note 14 : Neural Networks
- Note 15 : Training Neural Networks
- Discussion 8 (solution)
- Homework 7 (zip, datahub)
- Homework 8 (zip, datahub)
- Slides 10/19 (download)
- Slides 10/21 (download)
Week 9 Overview
Generative approaches to classification & Clustering: basics
- Note 16 : Generative vs. Discriminative Classification
- Note 18 : Gaussian Discriminant Analysis
- Note 19 : Clustering
- Homework 8 (zip, datahub)
- Slides 10/26 (Slides) (download)
- Slides 10/26 (Lecture Video) (download)
Notes
See Syllabus for more information.
- Note 2: Linear Regression
- Note 3: Feature Engineering
- Note 4: MLE and MAP for Regression
- Note 6: Multivariate Gaussians
- Note 5: Bias-Variance Tradeoff
- Note 7: MLE and MAP Pt. 2
- Note 8: Kernels and Ridge Regression
- Note 9: Total Least Squares
- Note 10: PCA
- Note 11: CCA
- Note 12: Optimization
- Note 13: Gradient Descent
- Note 14: Neural Networks
- Note 15: Training Neural Networks
- Note 16: Generative vs. Discriminative Classification
- Note 18: Gaussian Discriminant Analysis
- Note 19: Clustering
Expand
Discussions
The discussion sections may cover new material and will give you additional practice solving problems. You should attend the discussion that you will be assigned to with your study group, and details about this will be made available on the course Piazza. See Syllabus for more information.
- Discussion 0: Discussion logistics in the zoom era (solution)
- Discussion 1: Learning 1d function (solution)
- Discussion 2: Test sets and validation (solution)
- Discussion 3: Probabilistic View of Linear Regression (solution)
- Discussion 4: Kernels and kernelization (solution) (solution PDF)
- Discussion 5: Overparametrized regression (solution)
- Discussion 6: PCA, LASSO and ridge regression (solution)
- Discussion 7: Introduction to Neural Networks (solution)
- Discussion 8: Neural networks and gradient descent (solution)
Expand
Homeworks
Your two lowest homework scores will be dropped and homework scores are rescaled such that 60% is full credit, but these drop should be reserved for emergencies. See Syllabus for more information.
- Homework 0: Getting Started (zip, datahub)
- Homework 1: Classification and Regression (zip, datahub)
- Homework 2: Validation and setting hyperparameters (zip, datahub)
- Homework 3: Probability-based Models (zip, datahub)
- Homework 4: Kernel regression (zip, datahub)
- Homework 5: Under- and Over-Parameterized Learning (zip, datahub)
- Homework 6: Midterm Practice and Regularizing Features (zip, datahub)
- Homework 7: Lasso, Sparsity and Neural Nets (zip, datahub)
- Homework 8: Neural Networks (zip, datahub)
Expand
Lecture Slides
See Syllabus for more information.
- Slides 8/26 (pdf)
- Slides 8/31 (pdf)
- Slides 9/2 (pdf)
- Slides 9/9 (pdf)
- Slides 9/14 (pdf)
- Slides 9/16 (pdf)
- Slides 9/18 (pdf)
- Slides 9/21 (pdf)
- Slides 9/23 (UPDATED) (pdf)
- Slides 9/28 (Slides) (pdf)
- Slides 9/28 (Notes) (pdf)
- Slides 9/30 (Slides) (pdf)
- Slides 9/30 (Notes) (pdf)
- Slides 10/5 (Notes) (pdf)
- Slides Backprop (10/12) (pdf)
- Slides 10/12 (pdf)
- Slides SGD Notes (10/14) (pdf)
- Slides 10/14 (pdf)
- Slides 10/19 (pdf)
- Slides 10/21 (pdf)
- Slides 10/26 (Slides) (pdf)
- Slides 10/26 (Lecture Video) (pdf)
Expand