CS 189 at UC Berkeley
Introduction to Machine Learning
Lectures: 5-6:30 pm Tu-Th in Pimentel 1 (Berkeley Academic Guide page)
Week 0 Overview
Linear Regression, Features, Hyperparameters and Cross-Validation
Week 1 Overview
Features, Linear Regression, Regularization, Hyperparameters and Cross-Validation
- Note 2 : Linear Regression
- Note 3 : Features, Hyperparameters, Validation
- Discussion 1 (solution)
- Homework 0 (solution)
- Homework 1 (data) (solution)
- Homework 2 (data) (solution)
- Slides 9/3 (pdf)
- Slides 9/5 (pdf)
Week 2 Overview
MLE, MAP, Bias-Variance Tradeoff
- Note 4 : MLE and MAP for Regression (Part I)
- Note 5 : Bias-Variance Tradeoff
- Note 6 : Multivariate Gaussians
- Discussion 2 (solution)
- Homework 1 (data) (solution)
- Homework 2 (data) (solution)
- Slides 9/10 (pdf)
- Slides 9/12 (pdf)
Week 3 Overview
Weighted LS, MVN, Kernels
- Note 7 : MLE and MAP for Regression (Part II)
- Note 8 : Kernels, Kernel Ridge Regression
- Discussion 3 (solution)
- Homework 3 (data) (solution)
- Slides 9/17 (pdf)
- Slides 9/19 (pdf)
Week 4 Overview
Weighted LS, Kernels, PCA
- Note 8 : Kernels, Kernel Ridge Regression
- Note 9 : Total Least Squares
- Discussion 4 (solution)
- Homework 3 (data) (solution)
- Homework 4 (data) (solution)
- Slides 9/24 (pdf)
- Slides 9/26 (pdf)
Week 5 Overview
PCA, Total LS, CCA
Week 6 Overview
PCA, Total LS, CCA
- Note 9 : Total Least Squares
- Note 10 : Principal Component Analysis (PCA)
- Note 11 : Canonical Correlation Analysis (CCA)
- Discussion 6 (solution)
- Homework 4 (data) (solution)
- Slides 10/8 (pdf)
- Slides 10/10 (Power Outage)
Week 7 Overview
Midterm Review
- Note 10 : Principal Component Analysis (PCA)
- Note 11 : Canonical Correlation Analysis (CCA)
- Slides 10/15 (pdf)
- Slides 10/17 (Midterm)
Week 8 Overview
Non-linear LS, Gradient Descent, Neural Nets
Week 9 Overview
Classification, Logistic Regression, SVM, NNs, LDA
Week 10 Overview
K-Means, GMMs, EM, ROC Curves
- Note 19 : Expectation-Maximization (EM) Algorithm, k-means Clustering
- Discussion 9 (solution)
- Discussion 10 (solution)
- Homework 6 (data) (solution)
- Homework 7 (data) (solution)
- Slides 11/5 (pdf)
- Slides 11/7 (pdf)
Week 11 Overview
Decision Trees
- Note 25 : Decision Trees and Random Forests
- Note 26 : Boosting
- Discussion 11 (solution)
- Discussion 12 (solution)
- Homework 7 (data) (solution)
- Homework 8 (data) (solution)
- Homework 9 (data) (solution)
- Slides 11/12 (pdf)
- Slides 11/14 (pdf)
- Slides 11/19 (pdf)
- Slides 11/21
- Slides 11/26 (pdf)
- Slides 11/28 (Thanksgiving)
Week 12 Overview
LASSO, Sparsity, Feature Selection, Auto-ML
- Note 24 : Sparsity
- Note 27 : Convolutional Neural Networks (CNN)
- Homework 8 (data) (solution)
- Homework 9 (data) (solution)
- Slides 12/3 (pdf)
- Slides 12/5
Notes
See Syllabus for more information. You can find a list of week-by-week topics. Notes are not a substitute for going to lecture, as additional material may be covered in lecture. Notes are from a previous iteration of the course and may not be comprehensive. Refer to lectures.
- Note 1: Introduction
- Note 2: Linear Regression
- Note 3: Features, Hyperparameters, Validation
- Note 4: MLE and MAP for Regression (Part I)
- Note 5: Bias-Variance Tradeoff
- Note 6: Multivariate Gaussians
- Note 7: MLE and MAP for Regression (Part II)
- Note 8: Kernels, Kernel Ridge Regression
- Note 9: Total Least Squares
- Note 10: Principal Component Analysis (PCA)
- Note 11: Canonical Correlation Analysis (CCA)
- Note 12: Nonlinear Least Squares, Optimization
- Note 13: Gradient Descent Extensions
- Note 14: Neural Networks
- Note 15: Training Neural Networks
- Note 16: Discriminative vs. Generative Classification, LS-SVM
- Note 17: Logistic Regression
- Note 18: Gaussian Discriminant Analysis
- Note 19: Expectation-Maximization (EM) Algorithm, k-means Clustering
- Note 20: Support Vector Machines (SVM)
- Note 21: Generalization and Stability
- Note 22: Duality
- Note 23: Nearest Neighbor Classification
- Note 24: Sparsity
- Note 25: Decision Trees and Random Forests
- Note 26: Boosting
- Note 27: Convolutional Neural Networks (CNN)
Expand
Discussions
The discussion sections may cover new material and will give you additional practice solving problems. You can attend any discussion section you like. See Syllabus for more information.
- Discussion 0: Vector Calculus, Linear Algebra (solution)
- Discussion 1: Optimization, Least Squares, and Convexity (solution)
- Discussion 2: Ridge Regression and Multivariate Gaussians (solution)
- Discussion 3: Multivariate Gaussians and Kernels (solution)
- Discussion 4: Principal Component Analysis (solution)
- Discussion 5: Least Squares and Kernels (solution)
- Discussion 6: Optimization and Reviewing Linear Methods (solution)
- Discussion 7: Backpropagation and Computation Graphs (solution)
- Discussion 8: QDA and Logistic Regression (solution)
- Discussion 9: EM (solution)
- Discussion 10: SVMs and KNN (solution)
- Discussion 11: Decision Trees (solution)
- Discussion 12: LASSO, Sparsity, Feature Selection, Auto-ML (solution)
Expand
Homeworks
All homeworks are fully graded. Your lowest homework score will be dropped and homework scores are rescaled such that 80% is full credit, but this drop should be reserved for emergencies. See Syllabus for more information.
- Homework 0: Getting Started (solution)
- Homework 1: Prerequisites (data) (solution)
- Homework 2: Regression (data) (solution)
- Homework 3: Probabilistic Regression (data) (solution)
- Homework 4: Kernels, Correlations, and Total Least Squares (data) (solution)
- Homework 5: Optimization, Correlation, CCA (data) (solution)
- Homework 6: Backprop and Neural Networks (data) (solution)
- Homework 7: K-Means and EM (data) (solution)
- Homework 8: SVMs and Decision Trees (data) (solution)
- Homework 9: Ensemble Methods and Sparse Regression Revisited (data) (solution)
Expand
Lecture Slides
See Syllabus for more information.
- Slides 8/29 (pdf) (pdf)
- Slides 9/3 (pdf)
- Slides 9/5 (pdf)
- Slides 9/10 (pdf)
- Slides 9/12 (pdf)
- Slides 9/17 (pdf)
- Slides 9/19 (pdf)
- Slides 9/24 (pdf)
- Slides 9/26 (pdf)
- Slides 10/1 (pdf)
- Slides 10/3 (pdf)
- Slides 10/8 (pdf)
- Slides 10/10 (Power Outage)
- Slides 10/15 (pdf)
- Slides 10/17 (Midterm)
- Slides 10/22 (pdf)
- Slides 10/24 (pdf)
- Slides 10/29 (pdf)
- Slides 10/31 (pdf)
- Slides 11/5 (pdf)
- Slides 11/7 (pdf)
- Slides 11/12 (pdf)
- Slides 11/14 (pdf)
- Slides 11/19 (pdf)
- Slides 11/21
- Slides 11/26 (pdf)
- Slides 11/28 (Thanksgiving)
- Slides 12/3 (pdf)
- Slides 12/5
Expand