Unsupervised Learning with Python

Unsupervised learning is a set of techniques que helps discover patterns in unlabeled data. For example, suppose you're looking at dogs - some dogs are tall, others may have long fur. Although you're not exactly sure what they're called, you may notice some repeating patterns and Similarities. Without Knowing what is a "Greyhound", or "German Shepherd" you've already learned Their important attributes! This is one type of unsupervised learning, called clustering. Unsupervised learning is powerful tool for detecting anomolies, finding useful features, and visualizing the structure of unlabeled data. In this workshop we will give you an introduction to unsupervised learning techniques in machine learning. After a crash course on unsupervised learning, we will cover a few strategies used by popular algorithms, including K-means and PCA. During the workshop, we will be using the popular Python machine learning library, scikit-learn. http://scikit-learn.org/

Transcript

1. Unsupervised  Learning  with  Python A  presentation  for  Python Brasil [12] by  Breandan Considine @breandan
2. Why  machine  learning? •ML  is  just  a  bunch  of  tools,  for,  example: •Prediction •Categorization •Anomaly  detection •Personalization •Adaptive  control •Playing  games
3. Types  of  machine  learning
4. Reinforcement  Learning •Agent  has  a  context  and  set  of  choices •Each  choice  has  an  (unknown)  reward •Goal:  Maximize  cumulative  reward
5. A  quick  taste  of  supervised  learning
10. 0 1
11. Cool  learning  algorithm def classify(datapoint, weights):
12. Cool  learning  algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights))
13. Cool  learning  algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1
14. Cool  learning  algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1
15. Cool  learning  algorithm def train(data_set):
16. Cool  learning  algorithm def train(data_set):
17. Cool  learning  algorithm def train(data_set): class Datum: def __init__(self, features, label): self.features = [1] + features self.label = label
18. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) [0, 0, 0]
19. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1
20. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
21. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
22. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
23. weights = [w + RATE * error * i for w, i in zip(weights, item.features)] Cool  learning  algorithm * 1 i1 i2 in * * * w0 w1 w2 wn Σ
24. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
25. Cool  learning  algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
29. A  brief  look  at  unsupervised  learning •Where  did  my  labels  go? •Mostly  clustering,  seperation,  association •Many  different  methods • Self  organizing  map • Expectation-­‐maximization • Association  rule  learning • Hierarchical  clustering
30. Cool  clustering  algorithm def cluster(data, k, max_it=1000):
31. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False
32. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels
33. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1
34. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1 old_centers = np.copy(centers)
35. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1 old_centers = np.copy(centers) update_labels(labeled, centers)
36. Cool  clustering  algorithm def update_labels(data, centers): for datum in data: datum[-1] = centers[0, -1] min = distance.euclidean(datum[:-1], centers[0, :-1]) for center in centers: dist = distance.euclidean(datum[:-1], center[:-1]) if dist < min: min = dist datum[-1] = center[-1]
37. Cool  clustering  algorithm def update_labels(data, centers): for datum in data: datum[-1] = centers[0, -1] min = distance.euclidean(datum[:-1], centers[0, :-1]) for center in centers: dist = distance.euclidean(datum[:-1], center[:-1]) if dist < min: min = dist datum[-1] = center[-1]
38. Cool  clustering  algorithm def update_labels(data, centers): for datum in data: datum[-1] = centers[0, -1] min = distance.euclidean(datum[:-1], centers[0, :-1]) for center in centers: dist = distance.euclidean(datum[:-1], center[:-1]) if dist < min: min = dist datum[-1] = center[-1]
39. Cool  clustering  algorithm def update_labels(data, centers): for datum in data: datum[-1] = centers[0, -1] min = distance.euclidean(datum[:-1], centers[0, :-1]) for center in centers: dist = distance.euclidean(datum[:-1], center[:-1]) if dist < min: min = dist datum[-1] = center[-1]
40. Cool  clustering  algorithm def update_labels(data, centers): for datum in data: datum[-1] = centers[0, -1] min = distance.euclidean(datum[:-1], centers[0, :-1]) for center in centers: dist = distance.euclidean(datum[:-1], center[:-1]) if dist < min: min = dist datum[-1] = center[-1]
41. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1 old_centers = np.copy(centers) update_labels(labeled, centers) update_centers(labeled, centers)
42. Cool  clustering  algorithm def update_centers(data, centers): k = len(centers) for i in range(1, k + 1): cluster = data[data[:, -1] == i, :-1] centers[i - 1, :-1] = np.mean(cluster, axis=0)
43. Cool  clustering  algorithm def update_centers(data, centers): k = len(centers) for i in range(1, k + 1): cluster = data[data[:, -1] == i, :-1] centers[i - 1, :-1] = np.mean(cluster, axis=0)
44. Cool  clustering  algorithm def update_centers(data, centers): k = len(centers) for i in range(1, k + 1): cluster = data[data[:, -1] == i, :-1] centers[i - 1, :-1] = np.mean(cluster, axis=0)
45. Cool  clustering  algorithm def update_centers(data, centers): k = len(centers) for i in range(1, k + 1): cluster = data[data[:, -1] == i, :-1] centers[i - 1, :-1] = np.mean(cluster, axis=0)
46. Cool  clustering  algorithm def update_centers(data, centers): k = len(centers) for i in range(1, k + 1): cluster = data[data[:, -1] == i, :-1] centers[i - 1, :-1] = np.mean(cluster, axis=0)
47. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1 old_centers = np.copy(centers) update_labels(labeled, centers) update_centers(labeled, centers) return labeled
48. Cool  clustering  algorithm def cluster(data, k, max_it=1000): labeled = np.append(data, np.zeros((len(data), 1)), axis=1) random_pts = np.random.choice(len(labeled), k, replace=False centers = labeled[random_pts] centers[:, -1] = range(1, k + 1) # Assign labels it = 0 old_centers = None while it < max_it and not np.array_equal(old_centers, centers): it += 1 old_centers = np.copy(centers) update_labels(labeled, centers) update_centers(labeled, centers) return labeled
55. Data  pre-­‐processing •Data  selection •Data  processing • Formatting &  Cleaning • Sampling •Data  transformation • Feature  scaling  &  Normalization • Decomposition  &  Aggregation • Dimensionality  reduction
56. Principal  Component  Analysis
57. Further  resources •Code  for  slides  github.com/breandan/ml-­‐exercises •Neural  Networks  Demystified,  Stephen  Welch, •Machine  Learning,  Andrew  Ng   https://www.coursera.org/learn/machine-­‐learnin •Awesome  public  data  sets   github.com/caesar0301/awesome-­‐public-­‐datasets
58. Special  Thanks Hanneli Tavante Ilma Rodriguez Vilma  Rodriguez