The MLIR Framework A Brief Introduction to Deep Learning Compilers Dafna Mordechai 2021 

Hello! @Dafna_Mordechai ● Dafna Mordechai, BSc. in Computer Science, The Hebrew University of Jerusalem (2008) ● Senior RT Embedded Software Engineer ● LOVE technology, and LOVE sharing it with others 

AGENDA The LLVM Compiler Deep Learning MLIR 

The LLVM Compiler 

Compiler(Source Code) --> Machine Code C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC ● ● ● ● ● ● ● ● ● ● Translations + Transformations [semantic preserving] Compiler == ( ) 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC LLVM - Low Level Virtual Machine 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC ● ● ● ● ● ● ● ● ● ● Hardware agnostic operations Language agnostic operations 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Front-End Clang / Flang / Gollvm / rustc / … Back-End LLVM - An Open Source Modular Compiler Hardware agnostic operations Language agnostic operations Examples: • Machine instruction selection • Register allocation • Scheduling Examples: • Lexical analysis • Syntax analysis • Semantic analysis 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Front-End Back-End LLVM - An Open Source Modular Compiler Hardware agnostic operations Language agnostic operations ? 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC LLVM-IR - Standardized Intermediate Representation Hardware agnostic operations Language agnostic operations Middle-End LLVM-IR Standardized IR Front-End High Level Language to LLVM-IR Back-End LLVM-IR to Machine Code 

➔ Three-address code ➔ Typed ➔ Static single assignment (SSA) form LLVM-IR - Standardized Intermediate Representation 

C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Back-End LLVM - Three-Stage Compiler Hardware agnostic operations Language agnostic operations Front-End Middle-End LLVM Optimizer Examples: • Constant folding • Loop-invariant code motion • Dead code elimination 

LLVM - Low Level Virtual Machine Three-Stage Compiler ➢ Frontend ➢ Middle-end ➢ Backend ‐ Modularity ‐ Reusability ‐ Standardization of the IR ‐ Open-source ‐ Shared debug tools 

Deep Learning and MLIR 

Image by Gordon Johnson from Pixabay Deep Learning - A World of Dataflow Graphs ➔ Training ➔ Inference Time Series Data Numerical Data Categorical Data Images Categorical Data Categorical Data Text 

Example: GANILLA Image source: GANILLA’s paper: arXiv:2002.05638v2 

Inference edge-devices Deep Learning - Multiple Frameworks and Multiple Devices CPU GPU TPU ? 

Example: GANILLA GANILLA’s generator architecture (Image source: GANILLA’s paper: arXiv:2002.05638v2) Code written by Lior Dagan Leib, Liron Soffer and Dafna Mordechai. 

A Huge Amount of Data but Just A Few Lines of Code... Code written by Lior Dagan Leib, Liron Soffer and Dafna Mordechai. 

Inference edge-devices Deep Learning - Multiple Frameworks and Multiple Devices CPU GPU TPU Image by Gordon Johnson from Pixabay ? ➔ Operations definition and type handling ➔ Graph optimization ➔ Handling Memory related operation ➔ Handling model / data parallelism ➔ Code-Generation for multiple devices 

MLIR is a compiler intermediate representation that allows lowering dataflow graphs to target-specific machine code of high-performance data-parallel systems. MLIR: Multi Level Intermediate Representation 

PyTorch TensorFlow Mxnet Onnx Caffe … CPU GPU TPU Inference edge-devices MLIR - Multi Level Intermediate Representation Middle-End LLVM-IR MLIR LLVM - Instruction Based IR MLIR - Operation Based IR 

MLIR 

MLIR: Multi Level Intermediate Representation MLIR’s key concepts are operation, which gathers into Dialects. Compiler passes are used to transform operations, lowering one dialect to another. 

MLIR Operations An Operation represents a concept. A high-level concept like: function definitions, function calls, buffer allocations, view or slices of buffers, process creation, etc. A low-level concept like: target-independent arithmetic, target-specific instructions, configuration registers, logic gates, etc. Operation semantics can be described abstractly using Traits and Interfaces. 

No content

MLIR Dialects ➔ Dialects are the mechanism for extending the MLIR ecosystem. ➔ Dialects can define new operations, attributes, and types. ➔ In MLIR, dialects are interleaved. At any given time, the IR can be comprised of multiple dialects. ➔ Dialects don’t necessarily define hierarchies, but rather represent different abstraction levels. 

No content

Examples: Affine dialect - provides a powerful abstraction for affine operations and analyses. Standard dialect - a collection of operations such as: compare, casts, select, branch, conditional branch, return, assert, Atomic-RMW, fma, div, neg, or, and, etc. math dialect – sin, cos, exp, tanh, atan, pow, log, sqrt, rsqrt, etc. GPU dialect - middle-level abstractions for launching GPU kernels following a programming model that resembles CUDA or OpenCL. Vector dialect – operations such as load, store, shuffle, gather, transpose, reduction, etc. LLVM dialect - maps LLVM IR into MLIR by defining the corresponding operations and types. 

MLIR - Different Types of Transformation Middle-End LLVM-IR MLIR Importing Exporting Converting Translation Vs. Conversion 

Dialect Conversion ➔ MLIR support Partial Lowering (Also called Progressive Lowering) ➔ An operation of one dialect can be converted to another operation, or a group of operations, from the same dialect or other dialects. ➔ MLIR supports a pattern matching mechanism 

Partial Lowering / Progressive Lowering 

No content

MLIR: Multi Level Intermediate Representation Multi-Stage Compiler ➢ Import to MLIR ➢ Dialect conversion ➢ Export from MLIR ‐ Extendable operation-based IR - Designed for partial lowering ‐ Open source ‐ Part of the LLVM project, and can be easily lowered to LLVM-IR. 

Thank You! https://www.linkedin.com/in/dafna-mordechai @Dafna_Mordechai