CS 189 at UC Berkeley

# Introduction to Machine Learning

Lectures: T/Th 3:30-5 p.m., 155 Dwinelle

### Week 0 Overview

## Least Squares Framework

### Week 1 Overview

## Features, Regularization, Hyperparameters and Cross-Validation

- Note 1 : Introduction (Draft)
- Discussion 01 (solution)
- Homework 0 (TeX) (solution)
- Homework 01 (TeX) (data) (solution)

### Week 2 Overview

## MLE, MAP, OLS, Bias-Variance Tradeoffs

### Week 3 Overview

## Weighted LS, Total LS, Eigenmethods

### Week 4 Overview

## TLS, PCA

- Note 6 : Multivariate Gaussians
- Note 7 : MLE and MAP for Regression (Part II)
- Note 8 : Kernels, Kernel Ridge Regression
- Note 9 : Total Least Squares
- Note 10 : Principal Component Analysis (PCA) (Draft)
- Discussion 04 (solution)
- Homework 03 (TeX) (data) (solution) (solution code)
- Homework 04 (TeX) (data) (solution)
- Homework 05 (TeX) (data) (solution)

### Week 5 Overview

## CCA, Nonlinear LS, Gradient Descent

### Week 6 Overview

## Neural Nets, Stochastic Gradient Descent

### Week 7 Overview

## Regression for Classification: Generative v. Discriminative

- Note 13 : Optimization (Draft)
- Note 14 : Neural Networks (Draft)
- Note 15 : Training Neural Networks (Draft)
- Discussion 07 (solution)
- Homework 07 (TeX) (data) (solution) (solution code)

### Week 8 Overview

## Loss Functions, Hinge-Loss, SVM

- Note 16 : Discriminative vs. Generative Classification, LS-SVM
- Note 17 : Logistic Regression
- Note 18 : Gaussian Discriminant Analysis
- Note 19 : Expectation-Maximization (EM) Algorithm, k-means Clustering
- Discussion 07 (solution)
- Discussion 08 (solution)
- Homework 08 (TeX) (data) (solution) (solution code)

### Week 9 Overview

## k-Means, EM

### Week 10 Overview

## Spring Break

### Week 11 Overview

## Decision Trees, Boosting, Ensemble Methods

### Week 12 Overview

## Convolutional Neural Nets, Regularization Revisited

### Week 13 Overview

## Unsupervised Learning: Nearest Neighbors

- Note 26 : Boosting (Draft)
- Discussion 11 (solution)
- Discussion 13 (solution)
- Homework 12 (TeX) (data) (solution) (solution code)

### Week 14 Overview

## Sparsity and Decision Trees

- Note 27 : Convolutional Neural Networks (CNN)
- Discussion 13 (solution)
- Discussion 14
- Homework 12 (TeX) (data) (solution) (solution code)
- Homework 13 (TeX) (data) (solution) (solution code)
- Homework 14 (TeX) (data) (solution)

## Notes

See Syllabus for more information. You can find a list of week-by-week topics. You can find a comprehensive compilation of the notes here.

- Note 1: Introduction (Draft)
- Note 2: Linear Regression
- Note 3: Features, Hyperparameters, Validation
- Note 4: MLE and MAP for Regression (Part I)
- Note 5: Bias-Variance Tradeoff
- Note 6: Multivariate Gaussians
- Note 7: MLE and MAP for Regression (Part II)
- Note 8: Kernels, Kernel Ridge Regression
- Note 9: Total Least Squares
- Note 10: Principal Component Analysis (PCA) (Draft)
- Note 11: Canonical Correlation Analysis (CCA)
- Note 12: Nonlinear Least Squares
- Note 13: Optimization (Draft)
- Note 14: Neural Networks (Draft)
- Note 15: Training Neural Networks (Draft)
- Note 16: Discriminative vs. Generative Classification, LS-SVM
- Note 17: Logistic Regression
- Note 18: Gaussian Discriminant Analysis
- Note 19: Expectation-Maximization (EM) Algorithm, k-means Clustering
- Note 20: Support Vector Machines (SVM)
- Note 21: Generalization and Stability (Draft)
- Note 22: Duality
- Note 23: Nearest Neighbor Classification (Draft)
- Note 24: Sparsity
- Note 25: Decision Trees and Random Forests (Draft)
- Note 26: Boosting (Draft)
- Note 27: Convolutional Neural Networks (CNN)

Expand

## Discussions

The discussion sections may cover new material and will give you additional practice solving problems. You can attend any discussion section you like. See Syllabus for more information.

- Discussion 0: Vector Calculus, Linear Algebra (solution)
- Discussion 01: Derivatives Review, Least Squares (solution)
- Discussion 02: Ridge Regression (solution)
- Discussion 03: Bias-Variance Tradeoff (solution)
- Discussion 04: Kernel and Multivariate Gaussians (solution)
- Discussion 05: Dimensionality reduction (solution)
- Discussion 06: Midterm Review (solution)
- Discussion 07: Backpropagation (solution)
- Discussion 08: GD/SGD (solution)
- Discussion 09: QDA and Logistic Regression (solution)
- Discussion 10: Expectation Maximization (solution)
- Discussion 11: SVMs/Nearest Neighbors (solution)
- Discussion 12: Orthogonal Matching Pursuit (solution)
- Discussion 13: Convolutional Neural Networks (solution)
- Discussion 14: Clustering

Expand

## Homeworks

All homeworks are partially graded and it is highly-recommended that you do them. Your lowest homework score will be dropped, but this drop should be reserved for emergencies. Here is the semester's self-grade form (See form for instructions). See Syllabus for more information.

- Homework 0: Review and Linear Regression (TeX) (solution)
- Homework 01: Least Squares (TeX) (data) (solution)
- Homework 02: Ridge Regression (TeX) (data) (solution)
- Homework 03: Probabilistic Models (TeX) (data) (solution) (solution code)
- Homework 04: Kernel methods (TeX) (data) (solution)
- Homework 05: Dimensionality reduction (TeX) (data) (solution)
- Homework 06: CCA and Midterm Redo (TeX) (data) (solution)
- Homework 07: Backpropagation (TeX) (data) (solution) (solution code)
- Homework 08: SGD and Classification (TeX) (data) (solution) (solution code)
- Homework 09: LDA, CCA (TeX) (data) (solution) (solution code)
- Homework 10: K Means and EM (TeX) (data) (solution) (solution code)
- Homework 11: SVMs and Neighbors (TeX) (data) (solution)
- Homework 12: Sparsity and Decision Trees (TeX) (data) (solution) (solution code)
- Homework 13: Boosting, Convolutional Neural Networks (TeX) (data) (solution) (solution code)
- Homework 14: K-SVD and Dropout (TeX) (data) (solution)

Expand