Gorgonia: Primitives for Building Neural Networks in Go
Lightning talk given at Golang Sydney September 2016. This are the slides that introduce Gorgonia which was published earlier that day.
Gorgonia: https://github.com/chewxy/gorgonia
Gorgonia ¡ Library that provides the primitives to dynamically build neural networks and assorted machine learning algorithms ¡ Is like Theano and Tensorflow, but written in Go. Follow @chewxy on Twitter
Using Gorgonia ¡ Symbolic differentiation cost
:=
Must(Sum(act))
//or
mean
if
you
do
batch
size
stuff
grads,
err
:=
Grad(cost,
w)
//
grads[0]
is
∂cost ∂w Follow @chewxy on Twitter
Using Gorgonia ¡ Executing Expression Graph ¡ Two VMs available: ¡ TapeMachine
¡ LispMachine
¡ Created for different purposes Follow @chewxy on Twitter
Using Gorgonia ¡ TapeMachine ¡ Optimized for compile-once-execute-many-times expression graphs ¡ LispMachine ¡ Good for dynamically changing expression graphs (like LSTMs or GRUs) ¡ Mix and match running modes! Follow @chewxy on Twitter
Using Gorgonia ¡ Executing a Expression Graph prog,
locMap
:=
Compile(g)
m
:=
NewTapeMachine()
Let(x,
xVal)
Let(w,
wVal)
//
or
could
be
init
along
the
Node
if
err
:=
m.RunAll();
err
!=
nil
{
log.Fatal(err)
} Follow @chewxy on Twitter
Using Gorgonia prog,
locMap
:=
Compile(g)
m
:=
NewTapeMachine()
Let(x,
xVal)
Let(w,
wVal)
if
err
:=
m.RunAll();
err
!=
nil
{
log.Fatal(err)
} f
=
theano.function(inputs=[x],
outputs=grads)
f(xVal)
More verbose, but more mental clarity with regards to the running mode of the expression* *pure opinion(probably biased) by the creator of Gorgonia Follow @chewxy on Twitter
Things I wrote with Gorgonia ¡ "Compose" music given inputs from music by John Williams, Murray Gold, and Bear McCreary ¡ Neural Tensor Machine ¡ Generative Models in general ¡ (also all the not-so-cool stuff like linear regression*) * The world's oldest machine learning algorithm, over 100years old and still going strong! Follow @chewxy on Twitter
The Ask ¡ Help out Gorgonia's project (released today!) ¡ Test coverage is abysmal at 50%. 90% would be nice. ¡ Weird ASM bug wrt []float32
¡ Add more fun Ops. ¡ Distributed Computing (tried to implement 3 times!!) ¡ Low-level Optimization (esp. re: malloc and gc) https://github.com/chewxy/gorgonia Follow @chewxy on Twitter