Support vector machine(SVM)
(1) The cost function of SVM is
(2)The hypothesis is
(3)The decision boundary has a large margin which means the boundary is far from the data points.
Concretely，our hypothesis
and f will be in the following 2 ways:
Gaussian kernel
f(1) will always be 1, then we can use the algorithm to learn the theta.
Pay attention! f has the same dimension as the number of training examples.
Linear kernel(without the kernel)
Conclusion, well now we have so many models, which model is better?
We can make the decision based on the number of features n and the number of training examples m.
If n is small, m is intermediate(n = 1-1000, m = 1-10000) .SVM with the gaussian if n is small, m is large(n = 1- 1000, m =1-10000) we should add more features, we can use logistic regression or SVM without the kernel.If n is large (relative to m), then we'd better use logistic regression or SVM.
One VS all in SVM
The one VS all algorithm is the same as the one vs all algorithm mentioned in logistic regression. Suppose that we have k classes and we have to train k sets of theta. we have k outputs while we have a input , we choose the largest output as our output to 1 the other outputs become 0.
