Leland McInnes
February 23, 2019
3.2k

# Learning Topology: topological methods for unsupervised learning

A whirlwind tour of how topological ideas and methods can provide powerful solutions to unsupervised learning problems.

## Leland McInnes

February 23, 2019

## Transcript

1. Learning
Topology
Topological Methods for
Unsupervised Learning

2. A whirlwind tour of some
topological data analysis
techniques

3. Sound theory
Practical application

4. A Topology Primer

5. Simplices

6. Theorem 1 (Nerve theorem). Let U = {Ui
}i2I be
a cover of a topological space X. If, for all ⇢ I
T
i2
Ui is either contractible or empty, then N(U)
is homtopically equivalent to X.

7. Functor
Limit
Colimit

8. Functor: A function
between domains of
discourse

equivalence between
domains of discourse

10. Limit: A solution to a
system of constraints

11. Colimit: Gluing together
a system of objects

12. Dimension
Reduction

13. If the data is uniformly
distributed on the manifold
then the cover will be “good”

14. When is data that
nicely behaved?

15. Assumption:
Data is uniformly
distributed on the manifold

16. Deﬁne a Riemannian
metric on the manifold to
make this assumption true

17. Assumption:
The manifold is locally
connected

18. But our local metrics
are all incompatible!

19. Glue things together
with colimits?

20. Theorem 2 (UMAP Adjunction). The
functors FinReal : sFuzz ! FinEPMet
and FinSing : FinEPMet ! sFuzz
form an adjunction FinReal a FinSing.

21. Suppose we were given a
low dimensional
representation

22. We can apply the same
process to get a
probabilistic graph!

23. X
a2A
µ(a) log

µ(a)
⌫(a)

+ (1 µ(a)) log

1 µ(a)
1 ⌫(a)

24. X
a2A
µ(a) log

µ(a)
⌫(a)

+ (1 µ(a)) log

1 µ(a)
1 ⌫(a)

Get the clumps right
Get the gaps right

25. UMAP on MNIST digits

26. UMAP on Fashion MNIST

27. UMAP on Kuzushiji-MNIST

28. Derived from Adam Bielski’s Siamese/Triplet repository:
Metric Learning

29. Word Embeddings

30. You shall know a word
by the company it keeps
— John Rupert Firth

31. Represent a word as a
multinomial
distribution of words
that co-occur with it

32. A very large and very
sparse matrix

33. Find the manifold on
which the words lie

34. Assumptions:
Uniform distribution
Locally connected

35. Use the correct metric
for multinomial
parameter space

36. Example embedding
of Yelp reviews

37. Clustering

38. What do we mean by a
cluster?

39. A cluster is …

40. A connected component of
a level set of the probability
density function of the
underlying (and unknown)
distribution from which our
data samples are drawn.

41. How do we compute
that without knowing
the PDF?

42. Assumption:
Data is distributed on the
manifold according to
some PDF

43. Choose a Riemannian
metric that preserves
the distribution

44. But our local metrics
may be incompatible…

45. Solve the system of
constraints using limits?

46. Theorem 2 (UMAP Adjunction). The
functors FinReal : sFuzz ! FinEPMet
and FinSing : FinEPMet ! sFuzz
form an adjunction FinReal a FinSing.

47. We have captured the
topology of the PDF

48. The connected
components functor p0
produces a fuzzy set of
connected components
π0

49. Exclude components
below a threshold cluster
size and sort points by
component membership

50. Conclusions

51. Topology and category
theory provide a
different language to
frame problems

52. Topological techniques
can provide powerful
solutions

53. Hopefully I have
motivated you to learn
more!
[email protected] @leland_mcinnes