Clojure and AI

Slides to the "Clojure and AI" talk on :clojureD 2018.

Transcript

1. Clojure and AI Michael Pershyn 2018-02-24 @pershyn
2. 2 • What is AI • AI Today • Brief history of AI and LISP • Paradigms of AI • Deep Learning Frameworks Landscape • Clojure and Symbolic AI • Clojure and Connectionist AI • Future Agenda
3. 3 What is AI?
4. 4 The defnition of AI is changing. Whenever we solve the problem, we don't call it AI anymore.
5. 5 What is Intelligence • Learning • Reasoning • Problem-solving • Perception • Language understanding
7. 7 „A computer would deserve to be called intelligent if it could deceive a human into believing that it was human.“
8. 8 Turing Test
10. 10 AI is ... • Intelligence, demonstrated by machines – Ability to learn, reason, solve problems, percieve and understand language • Area of computer science that emphasizes the creation of intelligent machines that work and react like humans *AI is frequently used to describe things that haven‘t been done yet
11. 11 AI is the “new electricity”
12. 12 AI Today: „Recent Advancements“ In case you missed it.
13. 13 11 May 1997 First computer program to defeat a world champion in a match under tournament regulations (3½–2½) https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov
14. 14 Self driving cars
15. 15 2011 IBM Watson competed on Jeopardy! against former winners winning the frst place prize.
16. 16 2014 Turing Test Passed (33% counts?) 'Eugene Goostman' simulates a 13 year old boy and 10 out of 30 judges were convinced he is a human https://en.wikipedia.org/wiki/Eugene_Goostman
17. 17 http://niessnerlab.org/papers/2016/1facetoface/thies2016face.pdf 2016 Face2Face: Real-time Face Capture and Reenactment of RGB Videos
18. 18 9–15 March 2016 AlphaGo won the match in a game with Lee Sedol, 18-time world champion. (4-1) https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
19. 19 2017: pix2pix https://github.com/afnelayer/pix2pix-tensorfow, https://github.com/phillipi/pix2pix
20. 20 2017: pix2pix https://github.com/afnelayer/pix2pix-tensorfow, https://github.com/phillipi/pix2pix
21. 21 https://www.ef.org/ai/metrics
22. 22 https://www.ef.org/ai/metrics
23. 23 https://www.ef.org/ai/metrics Speech recognition
24. 24 Other advancements • 2014 Facebook research published an article where it claims that its DeepFace face recognition system identifes faces with accuracy of 97.35% • Libratus (by Carnegie Mellon) won against four top players at no-limit Texas hold 'em (2017) • OpenAI defeats top players in Dota 2 tournament (2017) • Alibaba language processing AI outscores top humans at a Stanford University reading and comprehension test, scoring 82.44 against 82.304 on a set of 100,000 questions (2018)
26. 26 Brief History of AI and LISP
28. 28 "symbol manipulation" view of how the mind works
29. 29 Notable LISP Features • Conditionals • Function type & recursion • Symbols • Garbage collection • Syntax uniformity • Extensible • Rapid prototyping
30. 30 AI research evolved, several paradigms became established
31. 31 Paradigms of AI Paradigm Inspired by Symbolic Logic and Linguistic Connectionist Neuroscience Evolutionist Evolutionary Biology Statistical Probability, Statistics and Combinatorics
32. 32 Symbolic Approach http://thinkdiferent.typepad.com/edulog/2006/10/types_of_expert.html • Focus on logic (if-then rules, etc) • Focus on problem solving • Knowledge engineering
33. 33 Typical AI Areas for Computing with Symbols • computer algebra & theorem proving • planning systems • diagnosis • knowledge representation and reasoning • logic • languages • machine translation • expert systems • ...
34. 34 https://www.igcseict.info/theory/7_2/expert/
35. 35 Notable AI Systems Developed in LISP • Macsyma - frst large computer algebra system (still in use). • ACL2 - theorem prover, e.g. used by AMD (last release in 2017) • DART - logistics planner. Used during the frst Gulf war by the US military. – This Lisp application alone is said to have paid back for all US investments in AI research at that time. • SPIKE - the planning and scheduling application for the Hubble Space Telescope. Also used by several other large telescopes. • Dendral, CADUCEUS, MYCIN and others...
36. 36 Connectionist Approach • Inspired by neuroscience • Non-symbolic knowledge representation • Learning capability • Parallel computation
38. 38 Deep Learning Frameworks Landscape
39. 39 https://twitter.com/fchollet/status/732226599882383361
41. 41 Deep learning frameworks (2018-02) github.com stats Forks Open issues Contributors last commit tensorfow/tensorfow 58367 1195 1338 today BVLC/cafe 13968 590 260 7 days ago keras-team/keras 9365 1032 624 2 days ago apache/incubator-mxnet 4817 761 485 1 day ago deeplearning4j/deeplearning4j 4021 604 139 today Microsoft/CNTK 3668 399 169 3 days ago Theano/Theano 2394 527 328 3 days ago torch/torch7 2244 210 133 5 month ago tfearn/tfearn 1792 470 110 8 days ago chainer/chainer 923 124 155 today karpathy/convnetjs 1749 37 15 14 month ago Lasagne/Lasagne 914 112 62 2 month ago NervanaSystems/neon 767 60 78 10 days ago mila-udem/blocks 325 178 48 7 month ago autumnai/leaf 262 27 14 1 month ago thinktopic/cortex (clojure) 85 28 26 1 month ago hswick/jutsu.ai (clojure) 4 3 3 11 days ago bpiel/guildsman (clojure) 3 0 1 3 days ago
42. 42 Clojure and Symbolic AI
43. 43 Reminder. AI efect. Whenever we solve the problem, we don't call it AI anymore
44. 44 Expert Systems in Clojure – Clara Rules • Friendly and readable DSL to specify rules. • Designed to be interoperable with Java • https://github.com/cerner/clara-rules
45. 45 core.logic • Takes features from Prolog to do Logic Programming in Clojure • Logic Programming Paradigm – declaring logic variables – defning constraints on them – having a logic engine fgure out what values of the variables satisfy the constraints
46. 46 Symbolic AI in Clojure • Symbolic pattern matcher – functions and macros to iterate patterns over collections of data – defne methods which specialise on patterns – and various other features – https://github.com/cognesence/matcher • Breadth-frst search mechanism for applying simple STRIPS-style operators – https://github.com/cognesence/ops-search • Convert free text to the set of rules – cognesence importing a library called lkit, WIP
47. 47 Clojure inherited LISP strengths, also useful for Symbolic AI
48. 48 Clojure and Connectionist AI
49. 49 3 approaches to DL in Clojure • Call the deeplearning4j • Call the TensorFlow • Use native clojure (cortex)
50. 50 hswick/jutsu.ai • Eclipse Deeplearning4j – for Java and Scala. – Integrated with Hadoop and Spark • Wrapper – Declarative, manageable – Good alternative to java interop • https://github.com/hswick/jutsu.ai • https://deeplearning4j.org/clojure
51. 51 hswick/jutsu.ai Example confguration of a simple multilayer feedforward neural network
52. 52 bpiel/guildsman • Implementing the API – Clojure, C++ • WIP, contributors welcome • https://github.com/bpiel/guildsman
53. 53 The Gradient Gap in API https://www.youtube.com/watch?v=8_HOB62rpvw
54. 54 thinktopic/cortex • Specialized graph framework for machine learning in Clojure • Tries to provide the least amount of cognitive overhead for devs • Components – cortex – compute, gpu-compute – cafe, keras • https://github.com/thinktopic/cortex
55. 55 Neural Network Description in Cortex
56. 56 Future
57. 57 Technological evolution is million times faster than biological
58. 58 Creativity, curiosity, emotions, self-awareness ….
59. 59 "Any sufciently advanced technology is indistinguishable from magic." Arthur C. Clarke
60. 60 OgmaNeo (Feynman Machine) • Hierarchical sequence prediction – video prediction – anomaly detection – sequence and grammar prediction – … stay tuned • Article – Feynman Machine: The Universal Dynamical Systems Computer – Eric Laukien, Richard Crowder, Fergal Byrne – https://arxiv.org/abs/1609.03971 https://aaai.org/ocs/index.php/SSS/SSS17/paper/viewFile/15362/14605
61. 61 http://www.dwavesys.com/sites/default/fles/D-Wave%202000Q%20Tech%20Collateral_0117F.pdf D-Wave 2000Q 2k qubit processor (2017)
62. 62 Google‘s TPU (In beta since 12. Feb 2018) https://cloudplatform.googleblog.com/2018/02/Cloud-TPU-machine-learning-accelerators-now-available-in-beta.html
63. 63 Synergry of symbolic and non-symbolic AI approaches
66. 66 Hype inevitably leads to a sense of disappointment when big breakthroughs don’t happen, causing overvalued startups to fail and investment to dry up.
67. 67 Hype Cycle
68. 68 https://jamesclear.com/continuous-improvement
69. 69 Dear Community, Thank You! You are awesome! Keep rocking!