MeritBadges/BasicClassifiers

Introduction

Classifiers are a very popular branch of machine learning, with myriad practical applications

Subject Matter Expert

Josh

Requirements

1. Discuss classifiers, including their inputs, outputs
2. Describe the strengths and weaknesses of classifiers
3. Demonstrate an understanding of Naive Bayes classifiers
   1. Describe the idea of conditional probability
   2. Demonstrate the derivation of Bayes's Theorem
   3. Explain how Bayes's Theorem is applied to create a Bayesian classifier
   4. Demonstrate the creation of data structures appropriate for Naive Bayesian classification given a small sample dataset
4. Demonstrate an understanding of Support Vector Machines
   1. Discuss the idea of higher-dimensional feature spaces
   2. Discuss separability and the challenges it poses
   3. Discuss separating planes, both verbally and graphically
   4. Explain the idea and motivation of a maximum margin hyperplane
   5. Discuss the kernel trick
5. Demonstrate practical knowledge. The student will provide a training set and a test set. Then, using one of the above techniques, and training only on the training set, they must achieve 80+% accuracy on the test set.

Resources

Bayes Resources

• Andrew Moore's Probability for Data Miners slides These are great. They will get you all the probability you need for Naive Bayes, and are very clear/self-contained
• Andrew Moore's Naive Bayes slides A fantastic set of slides, but still missing some details you'd get in the classroom

SVM Resources

• Kernel trick at wikipedia
• SVM - Support Vector Machines a good collection of graphs showing the separating plane concept
• Andrew Moore's SVM Slides Again, Andrew Moore has excellent slides which provide an overview, but they're sometimes hard to follow.