# Ridge classification using python.sklearn

sklearn is a powerful module on python for machine learning. in this post, we introduce ridge regression. the solution of ridge regression is given by,

$\displaystyle \hat{\alpha} = (\Phi^T\Phi + \lambda I)^{-1}\Phi y .$

the code is like this:

import numpy as np
import pylab as pl
from scipy import stats
from sklearn import linear_model

# make artificial data
n1 = 50
n2 = 50
x = np.c_[np.concatenate([stats.norm.rvs(-1, 1, n1), stats.norm.rvs(1, 1, n2)]),
stats.norm.rvs(0, 1, 100)]
y = np.concatenate([np.repeat(-1., n1), np.repeat(1., n2)])
classifier = linear_model.RidgeCV().fit(x, y)

# for contour plot
offset = 0.1
xx, yy = np.meshgrid(np.linspace(np.min(x[:,0]-offset), np.max(x[:,0]+offset)),
np.linspace(np.min(x[:,1]-offset), np.max(x[:,1]+offset)))

Z = classifier.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)
pl.scatter(x[:n1,0], x[:n1,1], c="b", marker="x")
pl.scatter(x[n1:,0], x[n1:,1], c="r", marker="o")
pl.contour(xx, yy, Z, levels=[0])
pl.show()
#import matplotlib.pyplot as plt
#plt.plot(x[:n1,0], x[:n1,1], "bx")
#plt.plot(x[n1:,0], x[n1:,1], "ro")
#plt.contour(xx, yy, Z, levels=[0])
#plt.show()


The results are shown below. linear classifier like this case is powerful, but it doesn’t work well if the data is not linearly separable (see graph2). The simple way to solve this problem is using kernel models. We’ll show the models and it work well even if the data is not linearly separable.