The Go-to Language for AI: Exploring Opportunities and Challenges

This session explores the unfolding potential of Go in AI, spotlighting pioneering ML/NLP frameworks and emerging LLM-oriented projects. Highlighting the shift towards compiled languages for more efficient AI solutions, Go emerges as a streamlined, high-performance choice. However, when juxtaposed against established frameworks or emerging stars written in C++ and Rust, which offer 'close to the metal' performance, Go faces challenges, particularly regarding raw performance on GPU. Despite these, Go’s unique strengths align well with the rising demand for local model execution, offering a strategic advantage to integrate AI capabilities into everyday projects. Through real-world scenarios, we’ll explore the pathway towards effective, local inference and single-executable deployment, positioning Go as a compelling contender in the evolving AI arena.

Transcript

1. Matteo Grella 2023 The Go-to Language for AI: Exploring Opportunities and Challenges GopherConAU 2023
2. Matteo Grella 2023 github.com/matteo-grella/gophercon-au-2023/ 2/30
3. Matteo Grella 2023 Matteo Grella • Head of Research & Artificial Intelligence at Crisis24 • Applying AI to Risk Intelligence for over 10 years • AI/ML/NLP Enthusiast (currently feeling overwhelmed) • Go/Ada/Lisp/Fortran Software Developer • Contributor to RWKV LLM (Fortran and Go port) • Creator of and related projects • Avid Onewheel rider and electric guitar enthusiast. 3/30 @GrellaMatteo | matteo-grella | Matteo Grella
4. Matteo Grella 2023 4/29 Key Question » Is Go the go-to language for AI? That is: what are the use cases for Go in the evolving AI arena?
5. Matteo Grella 2023 5/30 The Transformer Era's Influence on Technology
6. Matteo Grella 2023 6/30 (Neural) Large Language Models Westworld (TV series) LLMs learn simply by observing natural language! • LLMs excel at tasks like summarization, translation, coding, arithmetic, etc, due to their ability to GENERALIZE, REMEMBER, and ADAPT. • These skills arise from training on PREDICTING THE NEXT TOKEN in a sequence.
7. Matteo Grella 2023 7/30 LLM: A Financial Black Hole? • Costs for Training: Large GPT's training runs into millions. • Fine-Tuning Edge: Usually costs significantly less. • Quick Cost Drop for Inference: Notable 90% decrease for similar models since Dec 22. • Evolving Efficiency: Quantization advances promise further cost reductions.
8. Matteo Grella 2023 Hugging Face Hub: The Home of Open-Source LLMs 8/30
9. Matteo Grella 2023 9/30 LLM Benchmarks (Oct 2023) Zephyr 7B: a HuggingFace H4's fine- tuned model based on Mistral 7B that surpasses META Llama 2 70B https://arxiv.org/pdf/2310.16944.pdf
10. Matteo Grella 2023 Python: The De Facto Language for AI—For Now Outlines 〰 Guide the LLM text generation Deep Learning framework for CPU/GPU Distributed training and inference Multimodal AI services and pipelines Search and data extraction platform Agent-based applications Multimodal library with pretrained models Search across diverse data formats with LLMs 10/30
11. Matteo Grella 2023 11/30 High-Performance AI Tools in C/C++ and Rust • A Rust-based ML framework with PyTorch-like syntax, supports CPU/GPU acceleration backends and browser deployment via WASM. • It includes diverse models for language processing, text/image conversion, and computer vision, with support for BERT, LLaMA, Whisper and Stable Diffusion. • Compatible with various model file formats Candle • Plain C/C++ implementation to run selected LLMs with support for integer quantization • It is tailored for Apple silicon, leveraging ARM NEON, Accelerate, and also supports AVX through AVX512 for x86 architectures. • It works with F16/F32 precision and includes CUDA, Metal, and OpenCL backends for GPU • A Rust-based vector database and similarity search engine for neural network-based applications. • It offers a production-ready API for managing and searching vector points with payloads, • It serves as the vector database for Grok by @xai, enabling real-time access to world knowledge via the X platform, enhancing matching, searching, and recommendation.
12. Matteo Grella 2023 12/30 Go’s unique strengths Simplicity and Maintainability Go's straightforward syntax and toolchain reduces complexity and improves maintainability. Strong Concurrency Go's lighter weight goroutines allow efficient utilization of system resources. Cross-Platform Development Go supports cross-platform development and generates stand-alone executables. Stability Go has fewer dependencies compared to Python reducing devops time and increasing overall stability. Growing Developer Interest A growing developer community signifies potential for new opportunities and innovation.
13. Matteo Grella 2023 AI with Go: Unpacking Our Options Cons Pros Wrapping existing LLMs API to streamline their use in Go C++ bindings to utilize powerful LLMs in Go Pure Go ML models and algorithms Quick integration, access to state-of-the-art models Third-party dependency, API costs, limited customization Local usage for inference, performance gains from C++ libraries Complexity in maintenance, need for C++ expertise, Go's C FFI overhead (cgo) Local usage for training, fine-tuning, and inference, full control and optimization within Go's ecosystem Significant development effort, inherent performance ceiling due to language characteristics (?) 13/30
14. Matteo Grella 2023 14/30 A Spotlight on Relevant Go “AI” Repositories on GitHub LocalAI is a Go package that runs LLMs such as llama2, rwkv, vicuna, falcon, whisper, and others replicating the OpenAI APIs, via C/C++ bindings. Spago is a minimalistic ML framework in pure Go featuring a concurrent computational graph. It includes feed-forward, recurrent, and attention layers, gradient descent optimizers. 2-dim Array. No GPU. Cybertron is a pure Go NLP package built on spaGO, focused on inference for selected HuggingFace's pre- trained Transformer models, with potential for future fine- tuning—all within Go. Tensor offers efficient, generic n-dimensional arrays and common mathematical operations on CPU and GPU. It powers Gorgonia, a library aimed at ML with a focus on flexibility and correctness. Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more. LangChainGo is a work-in-progress Go implementation of LangChain, a framework designed to simplify the creation of applications using LLMs.
15. Matteo Grella 2023 15/30 Introducing spaGO
16. Matteo Grella 2023 1. package main 2. import ( 3. "fmt" 4. . "github.com/nlpodyssey/spago/ag" 5. "github.com/nlpodyssey/spago/mat" 6. ) 7. func main() { 8. w := mat.Scalar(3.) 9. b := mat.Scalar(1.) 10. x := mat.Scalar(2.) 11. y := Sigmoid(Add(Mul(w, x), b)) // y = σ(w*x + b) 12. fmt.Printf("y = %0.3f\n", y.Value().Item()) // y = 0.999 13. } Define-by-run Expression Graph 16/30
17. Matteo Grella 2023 Auto Gradient Backpropagation Loss (output, expected) Output Gradient Parameter Parameter Input 17/30 Δ Δ
18. Matteo Grella 2023 The Model 1. class Model(nn.Module): 2. def __init__(self, in_size, out_size): 3. super(Model, self).__init__() 4. self.w = nn.Parameter(torch.zeros(out_size, in_size)) 5. self.b = nn.Parameter(torch.zeros(out_size)) 6. def forward(self, x): 7. return torch.matmul(x, self.w.t()) + self.b 1. type Model struct { 2. nn.Module 3. W *nn.Param // weights 4. B *nn.Param // biases 5. } 6. func New[T float.DType](in, out int) *Model { 7. return &Model{ 8. W: nn.NewParam(mat.NewDense[T](mat.WithShape(out, in))), 9. B: nn.NewParam(mat.NewDense[T](mat.WithShape(out))), 10. } 11. } 12. func (m *Model) Forward(x mat.Tensor) mat.Tensor { 13. return ag.Add(ag.Mul(m.W, x), m.B) 14. } 18/30 Supports serialization methods for Gob, Python Pickle, and Safetensors.
19. Matteo Grella 2023 Introducing Waveny
20. Matteo Grella 2023 20/29 Waveny https://github.com/nlpodyssey/waveny • Implementation in Go of Neural Amp Modeler (NAM), a free and open-source technology for modeling guitar amplifiers and pedals using deep learning. • Training and inference in pure Go • Real-time mode with PortAudio (cgo)
21. Matteo Grella 2023 21/30 Waveny: Profile, Test, Live REAL-TIME INFERENCE TRAINING
22. Matteo Grella 2023 22/22 Live Demo Cybertron
23. Matteo Grella 2023 Masked Language Model func main() { m, err := tasks.Load[languagemodeling.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: "bert-large-cased", }) if err != nil { log.Fatalf("error loading model: %s", err) } fn := func(text string) error { result, err := m.Predict(ctx, text, languagemodeling.Parameters{K: 10}) if err != nil { return err } fmt.Println(MarshalJSON(result)) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > I'm so [MASK] to talk about this topic! I'm so excited to talk about this topic! > The [MASK] of this neural [MASK] is impressive. The efficiency of this neural network is impressive. > [MASK] is a programming language Python is a programming language > Berlin is the capital of [MASK] . Berlin is the capital of Germany . > [MASK] is the capital of Germany. Frankfurt is the capital of Germany. 23/29
24. Matteo Grella 2023 Named Entities Recognition func main() { m, err := tasks.Load[tokenclassification.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: "dslim/bert-base-NER, }) if err != nil { log.Fatalf("error loading model: %s", err) } params := tokenclassification.Parameters{ AggregationStrategy:tokenclassification.AggregationStrategySimple, } fn := func(text string) error { result, err := m.Classify(context.Background(), text, params) if err != nil { return err } fmt.Println(MarshalJSON(result)) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > Matteo Grella was born in Turin (Italy) on December 18, 1987. Matteo Grella -> PERSON Turin -> GPE Italy -> GPE December 18 , 1987 -> DATE > He is the Head of AI at EXOP GmbH. EXOP GmbH -> ORG 24/29
25. Matteo Grella 2023 Question-Answering func main() { m, err := tasks.Load[questionanswering.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: "deepset/bert-base-cased-squad2", }) if err != nil { log.Fatalf("error loading model: %s", err) } document := readFile(os.Args[2]) opts := &questionanswering.Options{} fn := func(question string) error { result, err := m.ExtractAnswer(ctx, question, document, opts) if err != nil { return err } fmt.Println(MarshalJSON(result)) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > What is Go? 0. a programming language [0.93] > When Go was created? 0. 2007 [0.96] > What is the purpose of Go? Sorry, I'm not sure. > Why Go was created? 0. to address criticism of other languages [0.53] > Who invented Go? 0. Robert Griesemer, Rob Pike, and Ken Thompson [0.96] > Where was Robert working when he created Go? 0. Google [1.00] Go is a programming language designed at Google in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson to address criticism of other languages. 25/29
26. Matteo Grella 2023 Zero-shot Text Classification func main() { m, err := tasks.Load[zeroshotclassifier.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: "valhalla/distilbart-mnli-12-3", }) if err != nil { log.Fatalf("error loading model: %v", err) } params := zeroshotclassifier.Parameters{ CandidateLabels: []string{"positive", "negative"}, HypothesisTemplate: zeroshotclassifier.DefaultHypothesisTemplate, MultiLabel: true, } fn := func(text string) error { result, err := m.Classify(context.Background(), text, params) if err != nil { return err } fmt.Println(MarshalJSON(result)) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > I got a promotion at work! 0. positive [0.98] 1. negative [0.02] > I've been working at a company but I got fired. 0. negative [0.90] 1. positive [0.10] > I am pleased with my new phone, but it has a short battery life. 0. positive [0.61] 1. negative [0.39] 26/29
27. Matteo Grella 2023 Cross-Lingual Text Similarity // /!\ Omit err check for brevity func main() { m, err := tasks.Load[textencoding.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: "sentence-transformers/all-MiniLM-L6-v2", }) targetVectors := make([]mat.Matrix, len(targets)) for i, text := range targets { // predefined sentences vector, err := m.Encode(context.Background(), text, int(bert.MeanPooling)) targetVectors[i] = vector.Vector.Normalize2() } fn := func(text string) error { result, err := m.Encode(context.Background(), text, int(bert.MeanPooling)) vector := result.Vector.Normalize2() hits := make([]Pair, len(targets)) for i, target := range targets { hits[i] = Pair{ Text: target, Score: vector.DotUnitary(targetVectors[i]).Item().F32()} } fmt.Println(MarshalJSON(Sort(hits))) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > I am very satisfied with my phone. 0. I really like my phone. [0.82] 1. I love my wife very much. [0.56] 2. I'm looking forward to dining with you. [0.39] 3. Did you get a new car? [0.22] 4. How old are you? [0.12] > ¿Qué edad tienes? 0. How old are you? [0.86] 1. Did you get a new car? [0.48] 2. Will it snow tomorrow? [0.37] 3. I love my wife very much. [0.22] 4. I really like my phone. [0.20] > Hai una macchina nuova? 0. Did you get a new car? [0.88] 1. Will it snow tomorrow? [0.46] 2. How old are you? [0.42] 3. I really like my phone. [0.33] 4. I love my wife very much. [0.26] 27/29
28. Matteo Grella 2023 Machine Translation func main() { m, err := tasks.Load[textgeneration.Interface](&tasks.Config{ ModelsDir: os.Args[1], ModelName: textgeneration.DefaultModelForMachineTranslation( os.Args[2], // source language e.g., “it” os.Args[3], // target language e.g., “en” ), }) if err != nil { log.Fatalf("error loading model: %v", err) } opts := textgeneration.DefaultOptions() fn := func(text string) error { result, err := m.Generate(context.Background(), text, opts) if err != nil { return err } fmt.Println(result.Texts[0]) return nil } if err = ForEachInput(os.Stdin, fn); err != nil { log.Fatal(err) } } > Questo è un traduttore automatico scritto in puro Go... ci puoi credere?! This is a machine translation written in pure Go... Can you believe it?! 28/29
29. Matteo Grella 2023 29/29 » Is Go the go-to language for AI? • Go is great for Go developers integrating ML and NLP in their projects • Rust or C++ pros won't find big advantages in switching (IMO) • Python lovers, GopherCon AU might not be your scene—just a bit of coder humor!
30. Matteo Grella 2023 Thank you! Questions?