CS 189 at UC Berkeley
Introduction to Machine Learning
Lectures: 2-3:30 pm Mon-Wed online (Berkeley Academic Guide page)
Week 0 Overview
Welcome and Introduction
- Discussion 0 (solution)
- Homework 0 (zip, datahub)
- Slides 8/26 (Slides) (link)
- Slides 8/26 (Video) (link)
Week 1 Overview
Foundations: Regression, Classification, and Learning & Features and Regularization
- Note 2 : Linear Regression
- Note 3 : Feature Engineering
- Discussion 1 (solution)
- Homework 0 (zip, datahub)
- Homework 1 (zip, datahub)
- Slides 8/31 (Slides) (link)
- Slides 8/31 (Video) (link)
- Slides 9/2 (Slides) (link)
- Slides 9/2 (Video) (link)
Week 2 Overview
Foundations: Validation and Generalization
- Note 3 : Feature Engineering
- Discussion 2 (solution)
- Homework 1 (zip, datahub)
- Homework 2 (zip, datahub)
- Slides 9/9 (Slides) (link)
- Slides 9/9 (Video) (link)
Week 3 Overview
Foundations: The probability perspective & Core tradeoffs
- Note 4 : MLE and MAP for Regression
- Note 6 : Multivariate Gaussians
- Discussion 3 (solution)
- Homework 2 (zip, datahub)
- Homework 3 (zip, datahub)
- Slides 9/14 (Slides) (link)
- Slides 9/14 (Video) (link)
- Slides 9/16 (Slides) (link)
- Slides 9/16 (Video) (link)
- Slides 9/18 (Slides) (link)
- Slides 9/18 (Video) (link)
Week 4 Overview
Foundations: Kernel perspective & The nearest-neighbor perspective
- Note 5 : Bias-Variance Tradeoff
- Note 7 : MLE and MAP Pt. 2
- Note 8 : Kernels and Ridge Regression
- Discussion 4 (solution) (solution PDF)
- Homework 3 (zip, datahub)
- Homework 4 (zip, datahub)
- Slides 9/21 (Slides) (link)
- Slides 9/21 (Video) (link)
- Slides 9/23 (Slides) (UPDATED) (link)
- Slides 9/23 (Video) (link)
Week 5 Overview
Foundations: Core tradeoffs revisited --- a unified view of under- and over-parameterized learning & Eigenspace perspectives
- Note 8 : Kernels and Ridge Regression
- Note 9 : Total Least Squares
- Note 10 : PCA
- Discussion 5 (solution)
- Homework 4 (zip, datahub)
- Homework 5 (zip, datahub)
- Slides 9/23 (Slides) (UPDATED) (link)
- Slides 9/23 (Video) (link)
- Slides 9/28 (Slides) (link)
- Slides 9/28 (Video) (link)
- Slides 9/28 (Notes) (link)
- Slides 9/30 (Slides) (link)
- Slides 9/30 (Video) (link)
- Slides 9/30 (Notes) (link)
Week 6 Overview
Foundations: Dimensionality reduction and latent spaces & Feature selection and the role of sparsity
- Note 9 : Total Least Squares
- Note 10 : PCA
- Note 11 : CCA
- Discussion 6 (solution)
- Homework 5 (zip, datahub)
- Homework 6 (zip, datahub)
- Slides 10/5 (Slides) (link)
- Slides 10/5 (Video) (link)
- Slides 10/7 (Video) (link)
Week 7 Overview
Gradient descent and its role & Beyond linear models: neural networks
- Note 12 : Optimization
- Note 13 : Gradient Descent
- Note 14 : Neural Networks
- Note 15 : Training Neural Networks
- Discussion 7 (solution)
- Homework 6 (zip, datahub)
- Homework 7 (zip, datahub)
- Slides Backprop (10/12) (link)
- Slides 10/12 (Slides) (link)
- Slides 10/12 (Video) (link)
- Slides SGD Notes (10/14) (link)
- Slides 10/14 (Slides) (link)
- Slides 10/14 (Video) (link)
Week 8 Overview
Stochastic gradient descent & Natural losses for classification
- Note 12 : Optimization
- Note 13 : Gradient Descent
- Note 14 : Neural Networks
- Note 15 : Training Neural Networks
- Discussion 8 (solution)
- Homework 7 (zip, datahub)
- Homework 8 (zip, datahub)
- Slides 10/19 (Slides) (link)
- Slides 10/19 (Video) (link)
- Slides 10/21 (Slides) (link)
- Slides 10/21 (Video) (link)
Week 9 Overview
Generative approaches to classification & Clustering: basics
- Note 16 : Generative vs. Discriminative Classification
- Note 18 : Gaussian Discriminant Analysis
- Note 19 : Clustering
- Discussion 9 (solution)
- Homework 8 (zip, datahub)
- Homework 9 (zip, datahub)
- Slides 10/26 (Slides) (link)
- Slides 10/26 (Video) (link)
- Slides 10/28 (Slides) (link)
- Slides 10/28 (Video) (link)
Week 10 Overview
Clustering: Mixture models & Margin in classification
- Note 19 : Clustering
- Note 20 : Support Vector Machines
- Discussion 10 (solution)
- Homework 9 (zip, datahub)
- Homework 10 (zip, datahub)
- Slides 11/2 (Slides) (link)
- Slides 11/2 (Video) (link)
- Slides 11/4 (Slides) (link)
- Slides 11/4 (Video) (link)
Week 11 Overview
Support vector machines
- Note 20 : Support Vector Machines
- Note 21 : Generalization and Stability
- Note 22 : Duality
- Discussion 11 (solution)
- Homework 10 (zip, datahub)
- Homework 11 (zip, datahub)
- Slides 11/9 (Slides) (link)
- Slides 11/9 (Video) (link)
Week 12 Overview
Decision Trees & Ensembles and Boosting
- Note 25 : Decision Trees
- Note 26 : Boosting
- Discussion 12 (solution)
- Homework 11 (zip, datahub)
- Homework 12 (zip, datahub)
- Slides 11/16 (Slides) (link)
- Slides 11/16 (Video) (link)
- Slides 11/18 (Slides) (link)
- Slides 11/18 (Video) (link)
Week 13 Overview
Deep neural networks
- Note 27 : Convolutional Neural Networks
- Discussion 13 (solution)
- Homework 12 (zip, datahub)
- Homework 13 (zip, datahub)
- Slides 11/23 (Slides) (link)
- Slides 11/23 (Video) (link)
Week 14 Overview
Deep neural networks (continued)
- Note 27 : Convolutional Neural Networks
- Homework 13 (zip, datahub)
- Slides 11/30 (Slides) (link)
- Slides 11/30 (Video) (link)
- Slides 12/2 (Slides) (link)
- Slides 12/2 (Video) (link)
Notes
See Syllabus for more information.
- Note 2: Linear Regression
- Note 3: Feature Engineering
- Note 4: MLE and MAP for Regression
- Note 6: Multivariate Gaussians
- Note 5: Bias-Variance Tradeoff
- Note 7: MLE and MAP Pt. 2
- Note 8: Kernels and Ridge Regression
- Note 9: Total Least Squares
- Note 10: PCA
- Note 11: CCA
- Note 12: Optimization
- Note 13: Gradient Descent
- Note 14: Neural Networks
- Note 15: Training Neural Networks
- Note 16: Generative vs. Discriminative Classification
- Note 18: Gaussian Discriminant Analysis
- Note 19: Clustering
- Note 20: Support Vector Machines
- Note 21: Generalization and Stability
- Note 22: Duality
- Note 25: Decision Trees
- Note 26: Boosting
- Note 27: Convolutional Neural Networks
Expand
Discussions
The discussion sections may cover new material and will give you additional practice solving problems. You should attend the discussion that you will be assigned to with your study group, and details about this will be made available on the course Piazza. See Syllabus for more information.
- Discussion 0: Discussion logistics in the zoom era (solution)
- Discussion 1: Learning 1d function (solution)
- Discussion 2: Test sets and validation (solution)
- Discussion 3: Probabilistic View of Linear Regression (solution)
- Discussion 4: Kernels and kernelization (solution) (solution PDF)
- Discussion 5: Overparametrized regression (solution)
- Discussion 6: PCA, LASSO and ridge regression (solution)
- Discussion 7: Introduction to Neural Networks (solution)
- Discussion 8: Neural networks and gradient descent (solution)
- Discussion 9: Clustering (solution)
- Discussion 10: EM Algorithm (solution)
- Discussion 11: Kernel Logistic Regression (solution)
- Discussion 12: Decision Trees (solution)
- Discussion 13: Convolutional Neural Networks (solution)
Expand
Homeworks
Your two lowest homework scores will be dropped and homework scores are rescaled such that 60% is full credit, but these drop should be reserved for emergencies. See Syllabus for more information.
- Homework 0: Getting Started (zip, datahub)
- Homework 1: Classification and Regression (zip, datahub)
- Homework 2: Validation and setting hyperparameters (zip, datahub)
- Homework 3: Probability-based Models (zip, datahub)
- Homework 4: Kernel regression (zip, datahub)
- Homework 5: Under- and Over-Parameterized Learning (zip, datahub)
- Homework 6: Midterm Practice and Regularizing Features (zip, datahub)
- Homework 7: Lasso, Sparsity and Neural Nets (zip, datahub)
- Homework 8: Neural Networks (zip, datahub)
- Homework 9: Classification and Clustering (zip, datahub)
- Homework 10: Mixture models & Margin in classification (zip, datahub)
- Homework 11: SVMs, Logistic Regression, and Adversarial Examples (zip, datahub)
- Homework 12: Decision trees, random forests, and boosting. (zip, datahub)
- Homework 13: Advanced Neural Networks and Exam Practice (zip, datahub)
Expand
Lecture Slides and Videos
See Syllabus for more information.
- Slides 8/26 (Slides) (link)
- Slides 8/26 (Video) (link)
- Slides 8/31 (Slides) (link)
- Slides 8/31 (Video) (link)
- Slides 9/2 (Slides) (link)
- Slides 9/2 (Video) (link)
- Slides 9/9 (Slides) (link)
- Slides 9/9 (Video) (link)
- Slides 9/14 (Slides) (link)
- Slides 9/14 (Video) (link)
- Slides 9/16 (Slides) (link)
- Slides 9/16 (Video) (link)
- Slides 9/18 (Slides) (link)
- Slides 9/18 (Video) (link)
- Slides 9/21 (Slides) (link)
- Slides 9/21 (Video) (link)
- Slides 9/23 (Slides) (UPDATED) (link)
- Slides 9/23 (Video) (link)
- Slides 9/28 (Slides) (link)
- Slides 9/28 (Video) (link)
- Slides 9/28 (Notes) (link)
- Slides 9/30 (Slides) (link)
- Slides 9/30 (Video) (link)
- Slides 9/30 (Notes) (link)
- Slides 10/5 (Slides) (link)
- Slides 10/5 (Video) (link)
- Slides 10/7 (Video) (link)
- Slides Backprop (10/12) (link)
- Slides 10/12 (Slides) (link)
- Slides 10/12 (Video) (link)
- Slides SGD Notes (10/14) (link)
- Slides 10/14 (Slides) (link)
- Slides 10/14 (Video) (link)
- Slides 10/19 (Slides) (link)
- Slides 10/19 (Video) (link)
- Slides 10/21 (Slides) (link)
- Slides 10/21 (Video) (link)
- Slides 10/26 (Slides) (link)
- Slides 10/26 (Video) (link)
- Slides 10/28 (Slides) (link)
- Slides 10/28 (Video) (link)
- Slides 11/2 (Slides) (link)
- Slides 11/2 (Video) (link)
- Slides 11/4 (Slides) (link)
- Slides 11/4 (Video) (link)
- Slides 11/9 (Slides) (link)
- Slides 11/9 (Video) (link)
- Slides 11/16 (Slides) (link)
- Slides 11/16 (Video) (link)
- Slides 11/18 (Slides) (link)
- Slides 11/18 (Video) (link)
- Slides 11/23 (Slides) (link)
- Slides 11/23 (Video) (link)
- Slides 11/30 (Slides) (link)
- Slides 11/30 (Video) (link)
- Slides 12/2 (Slides) (link)
- Slides 12/2 (Video) (link)
Expand