Exercise 3¶
Pandas library¶
- Python library for data manipulation and analysis
- Based on numpy, matplotlib
- Makes some common data analysis tasks very easy
- Documentation: pandas.pydata.org
In [2]:
import pandas as pd
pd._version.get_versions()
Out[2]:
In [3]:
import urllib.request
print(urllib.request.urlopen(
"http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data").read().decode('ascii')[:100], '...')
In [49]:
print('...', urllib.request.urlopen(
"http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.names").read().decode('ascii')
[2410:2620], '...', sep='\n')
In [3]:
df = pd.read_csv("http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data",
header=None,
names=['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width', 'Species'])
In [4]:
type(df)
Out[4]:
Pandas DataFrames¶
- Row/Column datatype
- Essentially a matrix with labels and indexes
- Many useful operations
In [5]:
df.head(5)
Out[5]:
In [6]:
df.head(5).values ## extract numpy array
Out[6]:
In [7]:
df['Petal Length'] ## extract entire columns by name
Out[7]:
In [9]:
df.head(5).Species
Out[9]:
In [10]:
df.head(5)[['Species', 'Petal Length']] ## extract more than one column
Out[10]:
In [11]:
df.T ## Transpose with labels
Out[11]:
In [12]:
df.index ## DataFrame rows are indexed
Out[12]:
In [13]:
df.iloc[4] ## Extract individual rows by their index
Out[13]:
In [14]:
df.iloc[[4, 17, 23]]
Out[14]:
In [9]:
df.head(8)
Out[9]:
In [19]:
#(df.head(8)['Sepal Length'] < 5) #|
(df.head(8)['Sepal Length'] > 4.6)
Out[19]:
In [20]:
df[~(df['Sepal Length'] < 5)].head(3) ## Boolean indexing
Out[20]:
In [24]:
df2 = df.copy() ## Cloning DataFrames
all_species = list(df.Species.unique())
print(all_species)
## Adding colums ## Transforming existing columns
df2['Class'] = df.Species.map(lambda s: all_species.index(s))
df2.tail(5)
Out[24]:
In [25]:
df.describe().round(2) ## Quick summary statistics
Out[25]:
Note: (25th percentile == x) $\leftrightarrow$ (25% of the data are $\leq$ x) etc.
In [29]:
df.groupby("Species").mean().T
Out[29]:
In [41]:
df2['dot_color'] = df2.Species.map(lambda s: {'Iris-setosa': 'red',
'Iris-virginica': 'green',
'Iris-versicolor': 'blue'}[s])
## Quick plotting
df2.plot('Petal Width', 'Sepal Length', kind='scatter', c=df2.dot_color, legend=True);
More plots: "Box-and-whisker" plot¶
- Quick way to examine feature distributions
- Reference: upload.wikimedia.org/.../Boxplot_vs_PDF.svg
In [42]:
df.groupby('Species').plot.box();
LMS with Iris data¶
Create subset for 1d-regression¶
In [45]:
## join multiple boolean dataframes with | (or), & (and)
subset = df2[(df2.Species == 'Iris-setosa') | (df2.Species == 'Iris-virginica')]
## alternative: negate with ~
# subset = df2[~(df2.Species == 'Iris-versicolor')]
subset = subset[['Sepal Length', 'Class'] ]
subset
Out[45]:
Prepare data for regression¶
In [46]:
## make separate vectors for x and y, and make the class -1 or 1
x = subset['Sepal Length']
y = subset['Class'] - 1
plt.scatter(x, y);
In [48]:
## Put everything in one matrix, with constant column prepended
import numpy as np
examples = np.vstack([np.ones_like(x), x, y])
examples = examples.T
print(examples.shape)
In [51]:
examples[:5]
Out[51]:
The LMS Algorithm¶
We implement the basic LMS Algorithm, using RSS to measure the error.
Design decisions:
- Store examples in a (n, p+1)-matrix
- Last column is the response variable (class)
- Omit convergence check
$$
RSS(w) = \sum_{i=1}^n (y_i - y(x_i))^2 = \sum_{i=1}^n (y_i - w^Tx_i)^2
$$
In [54]:
## using numpy:
y_true = np.array([1, -1, 1, 1])
y_pred = np.array([.8, -.5, -.4, .6])
y_true - y_pred
Out[54]:
In [55]:
(y_true - y_pred)**2
Out[55]:
In [56]:
((y_true - y_pred)**2).sum()
Out[56]:
In [57]:
def rss(examples, w):
"""Compute the residual sum of squares for a linear model.
Arguments:
examples -- (n, p + 1)-matrix of predictors and response
w -- p-vector of linear model weights
"""
x = examples[:,:-1] ### First p columns: shape = (n, p)
y = examples[:,-1] ### Last column: shape = (n,)
y_pred = w.dot(x.T)
rss = (y - y_pred)**2
### Note: w.dot(x) has shape = (n,)
return rss.sum()
In [59]:
def lms(examples, eta, iterations, print_every=1000):
np.random.seed(2)
rows, columns = examples.shape
p = columns - 1 ### last column is the response variable
w = np.random.uniform(low=-1.0, high=1.0, size=p)
for iteration in range(iterations):
rand = np.random.randint(0, rows) ### select random index
x = examples[rand,:-1] ### Everything but the last column
c = examples[rand,-1:] ### The last column
y = w.dot(x)
error = c - y ### Error in the single chosen example
w += (eta * error * x)
if iteration % print_every == 0 or iteration == (iterations-1):
err = rss(examples, w)
print(f"Iteration: {iteration} RSS: {err:.2f}")
return w
Fitting the Model¶
In [60]:
%%time
w = lms(examples, eta=0.05, iterations=10000)
Examining our regression model¶
In [61]:
print(w, rss(examples, w))
In [182]:
line_x = np.array([min(examples[:,1]), max(examples[:,1])])
line_y = w.dot(np.array([np.ones_like(line_x), line_x]))
print('w =', w)
print('line_x =', line_x)
print('line_y =', line_y)
Visualizing the line of best fit¶
In [171]:
plt.scatter(examples[:,1], examples[:,2])
plt.xlabel("Sepal Length")
plt.ylabel("Species");
plt.ylim((-1.1,1.1))
plt.axhline(0, c='k', ls='--')
plt.axvline(-w[0]/w[1], c='g', ls='--')
plt.plot(line_x, line_y, 'r');
In [172]:
(np.sign(w.dot(examples[:,:2].T)) == examples[:,2]).sum()
Out[172]:
Seaborn: Statistical Data Visualization¶
- Produces advanced visualizations with much less work than using matplotlib directly
- Works together with pandas very well
- Documentation & tutorials: stanford.edu/~mwaskom/software/seaborn
In [66]:
import seaborn as sb
In [154]:
## Just put in a DataFrame object,
## seaborn does the right thing automatically:
sb.pairplot(df, hue='Species', height=3, diag_kind='hist');
In [174]:
# lmplot() Does linear regression automatically
sb.lmplot(x='Sepal Width', y='Petal Length', data=df[~(df.Species == 'Iris-setosa')]);
