Foreign Inline Code in Haskell (YOW! Lambda Jam 2014)

Slide 1

Slide 1 text

Manuel M T Chakravarty University of New South Wales Foreign Inline Code in Haskell mchakravarty α TacticalGrace TacticalGrace justtesting.org 1 30 minute time slot: 25min talking + 5min [15min The Problem; 5min TH+Quasiquoting; 5min Inline Objective-C]

Slide 2

Slide 2 text

λ Shiny new functional language 2 » Imagine, you have got a shiny, new functional language...

Slide 3

Slide 3 text

3 » ...and you want to use it to write a great new app...

Slide 4

Slide 4 text

λ 4 » ...then you will need to use many existing frameworks and libraries. » Luckily, any serious language will have a foreign function interface! » Haskell standard includes a simple, but versatile FFI

Slide 5

Slide 5 text

“Problem solved?” 5 » Does that solve the problem of language interoperability? » Let me explain that at an example...

Slide 6

Slide 6 text

“Problem solved?” No! 5 » Does that solve the problem of language interoperability? » Let me explain that at an example...

Slide 7

Slide 7 text

dumpURL :: String -> IO () dumpURL urlString = do urlData <- stringWithContentsOfUrl urlString putStr urlData What we want to write 6

Slide 8

Slide 8 text

dumpURL :: String -> IO () dumpURL urlString = do urlData <- stringWithContentsOfUrl urlString putStr urlData What we want to write char *stringWithContentsOfUrlCstub(char *urlString) { NSURL *url = [NSURL URLWithString:urlString]; [NSString stringWithContentsOfURL:url encoding:NSUTF8StringEncoding error:NULL]; } What we want to call (& need to put into an extra ﬁle) 6

Slide 9

Slide 9 text

What we have to write as well… foreign import stringWithContentsOfURLCstub :: CString -> IO CString stringWithContentsOfURL :: String -> IO String stringWithContentsOfURL url = withCString url $ \urlC -> do resultC <- stringWithContentsOfURLCstub urlC result <- peek resultC free resultC return result 7 » FFI code: If you find it confusing, that’s fine, as I want to argue that you shouldn’t write it in the first place. * Anything extensively relying on foreign frameworks will be a pain [Hybrid languages (eg, F# & Scala) have a different set of trade offs, but don’t solve it either.]

Slide 10

Slide 10 text

Bridging Libraries 1-to-1 transliteration of types & functions 8 * Bridging or binding libraries transliterate types & functions * Need to be maintained and documented; track multiple versions of base library * Lack of type safety * Tools and dynamic transliteration (by reﬂection) help

Slide 11

Slide 11 text

C➙Haskell c2hs A case study GTK+ 9 * c2hs: automation for bridging C libraries to Haskell * Implements large parts of a C compiler front-end * Bridge for the cross-platform Gnome GUI library GTK+

Slide 12

Slide 12 text

10 * AFAIK, currently the only fully featured and properly maintained Haskell GUI library * Haskell GTK+ library is used for realistic applications

Slide 13

Slide 13 text

“Does this approach scale?” 11 » Read question * It requires signiﬁcant resources & constant effort to track the original library * Application frameworks: enormous and growing footprint

Slide 14

Slide 14 text

February 2011 Introduction of GTK+ 3 12 * Base GTK+ development isn’t even particularly fast * Bridging libraries lag behind their base libraries * Another example: Haskell OpenGL library

Slide 15

Slide 15 text

February 2011 Introduction of GTK+ 3 December 2013 GTK+ 3 in Haskell Bridge 12 * Base GTK+ development isn’t even particularly fast * Bridging libraries lag behind their base libraries * Another example: Haskell OpenGL library

Slide 16

Slide 16 text

13 * Compared to modern application frameworks, GTK+ is small!

Slide 17

Slide 17 text

Bridging libraries don’t scale Too much weight! 14

Slide 18

Slide 18 text

“Interoperability is an old problem — maybe an old solution can help?” 15

Slide 19

Slide 19 text

16 * Modula-2 with inline 68000 assembly * Assembly code can symbolically refer to Modula-2 entities (e.g., variables)

Slide 20

Slide 20 text

16 * Modula-2 with inline 68000 assembly * Assembly code can symbolically refer to Modula-2 entities (e.g., variables)

Slide 21

Slide 21 text

16 * Modula-2 with inline 68000 assembly * Assembly code can symbolically refer to Modula-2 entities (e.g., variables)

Slide 22

Slide 22 text

“Inline C, C++, Objective-C, … in Haskell?” 17 * We don’t want to build front ends for a few more languages into GHC * How do we share entities between the languages?

Slide 23

Slide 23 text

Meta-programming Template Haskell 18 » Generic infrastructure for program manipulation NOTE: I'll run through this quickly; I'll explain the details at the workshop.

Slide 24

Slide 24 text

HASKELL WORKSHOP 2002 19 * Template Haskell: Haskell extension implemented by GHC * Useful for: deﬁning macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…

Slide 25

Slide 25 text

HASKELL WORKSHOP 2002 #define macros 19 * Template Haskell: Haskell extension implemented by GHC * Useful for: deﬁning macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…

Slide 26

Slide 26 text

HASKELL WORKSHOP 2002 #define macros [| … |] code generators 19 * Template Haskell: Haskell extension implemented by GHC * Useful for: deﬁning macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…

Slide 27

Slide 27 text

HASKELL WORKSHOP 2002 #define macros [| … |] code generators trafo (ConE name) = … code transformations 19 * Template Haskell: Haskell extension implemented by GHC * Useful for: deﬁning macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…

Slide 28

Slide 28 text

$(sel 1 3) (a, b, c) = a 20 * Meta function executed at splice point, generating spliced code

Slide 29

Slide 29 text

$(sel 1 3) (a, b, c) = a meta-programming function 20 * Meta function executed at splice point, generating spliced code

Slide 30

Slide 30 text

$(sel 1 3) (a, b, c) = a meta-programming function splice 20 * Meta function executed at splice point, generating spliced code

Slide 31

Slide 31 text

$(sel 1 3) (a, b, c) = a meta-programming function splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a 20 * Meta function executed at splice point, generating spliced code

Slide 32

Slide 32 text

$(sel 1 3) (a, b, c) = a meta-programming function splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a $(sel 5 5) (a, b, c, d, e) = e 20 * Meta function executed at splice point, generating spliced code

Slide 33

Slide 33 text

$(sel 1 3) (a, b, c) = a meta-programming function splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a $(sel 5 5) (a, b, c, d, e) = e \tup -> case tup of {(x, y, z, v, w) -> w} :: (a, b, c, d, e) -> e 20 * Meta function executed at splice point, generating spliced code

Slide 34

Slide 34 text

sel :: Int -> Int -> ExpQ sel i n = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop

Slide 35

Slide 35 text

sel :: Int -> Int -> ExpQ sel i n = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop

Slide 36

Slide 36 text

sel :: Int -> Int -> ExpQ sel i n = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions quasi-quotation 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop

Slide 37

Slide 37 text

sel :: Int -> Int -> ExpQ sel i n = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions quasi-quotation spliced expression (anti quote) 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop

Slide 38

Slide 38 text

Processing other languages Generic quasi-quotation 22 * Quoting arbitrary languages

Slide 39

Slide 39 text

HASKELL WORKSHOP 2007 Quasiquoting for any language you can provide a parser for 23

Slide 40

Slide 40 text

Language.C.Quote add n = [cfun| int addConstant(int x) { return x + $int:n; } |] 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C

Slide 41

Slide 41 text

Language.C.Quote add n = [cfun| int addConstant(int x) { return x + $int:n; } |] quasi-quotation identiﬁer 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C

Slide 42

Slide 42 text

Language.C.Quote add n = [cfun| int addConstant(int x) { return x + $int:n; } |] quasi-quotation identiﬁer splice identiﬁer 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C

Slide 43

Slide 43 text

mkMap dev aenv fun arr = return $ CUTranslSkel "map" [cunit| $esc:("#include ") extern "C" __global__ void map ($params:argIn, $params:argOut) { const int shapeSize = size(shOut); const int gridSize = $exp:(gridSize dev); int ix; for ( ix = $exp:(threadIdx dev) ; ix < shapeSize ; ix += gridSize ) { $items:(dce x .=. get ix) $items:(setOut "ix" .=. f x) } } |] where ... 25 * Accelerate: embedded high-performance array language for GPUs * Combinators as code skeletons (code templates with holes) * Yellow splices/anti-quotes are the holes (parameters) of the template * Doing this with strings or explicit AST construction would be much worse

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Inlining foreign languages Inline C Code 26

Slide 47

Slide 47 text

Language.C.Inline dumpURL :: String -> IO () dumpURL urlString = do urlData <- putStr urlData stringWithContentsOfUrl urlString 27 * Inline Objective-C * All the FFI code is generated automatically * Again, details in the workshop

Slide 48

Slide 48 text

Language.C.Inline dumpURL :: String -> IO () dumpURL urlString = do urlData <- putStr urlData $(objc ['urlString] ''String [cexp| [NSString stringWithContentsOfURL: [NSURL URLWithString:urlString] encoding:NSUTF8StringEncoding error:NULL] |]) inline Objective-C splice 27 * Inline Objective-C * All the FFI code is generated automatically * Again, details in the workshop

Slide 49

Slide 49 text

Inline C …is simple 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language

Slide 50

Slide 50 text

Inline C …is simple No bridging library needs to be maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language

Slide 51

Slide 51 text

Inline C …is simple No extra documentation No bridging library needs to be maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language

Slide 52

Slide 52 text

Inline C …is simple No extra documentation Some type safety No bridging library needs to be maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language

Slide 53

Slide 53 text

Language integration by language inlining types >< state languages 29 Outline workshop * Get familiar with TH and Language.C.Quote * Run through three exercises using inline Objective-C including a graphical REPL

Slide 54

Slide 54 text

Workshop Part 1 Template Haskell 30 * http://www.haskell.org/haskellwiki/Template_Haskell * http://hackage.haskell.org/package/template-haskell

Slide 55

Slide 55 text

Clone https://github.com/mchakravarty/ylj14-workshop 31

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) 33

Slide 61

Slide 61 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) data Format = D | S | L String parse :: String -> [Format] 33

Slide 62

Slide 62 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) data Format = D | S | L String parse :: String -> [Format] gen :: [Format] -> ExpQ gen [D] = [| \n -> show n |] gen [S] = [| \s -> s |] gen [L s] = [| s |] 33

Slide 63

Slide 63 text

Task ❶ A recursive generator 34

Slide 64

Slide 64 text

printf/ 35

Slide 65

Slide 65 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) [| "" |] printf/ 35

Slide 66

Slide 66 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ printf/ 35

Slide 67

Slide 67 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix printf/ 35

Slide 68

Slide 68 text

printf :: String -> ExpQ printf fmt = gen (parse fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix printf/ 35

Slide 69

Slide 69 text

Slide 70

Slide 70 text

Slide 71

Slide 71 text

Slide 72

Slide 72 text

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Workshop Part 2 Quasi-quotation of C variants 36 * http://hackage.haskell.org/package/language-c-quote

Slide 75

Slide 75 text

gensum/ 37

Slide 76

Slide 76 text

genSum :: Name -> Int -> BlockItem genSum arr n = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] gensum/ 37

Slide 77

Slide 77 text

genSum :: Name -> Int -> BlockItem genSum arr n = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] custom parser for C block items gensum/ 37

Slide 78

Slide 78 text

genSum :: Name -> Int -> BlockItem genSum arr n = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] custom parser for C block items splicing of a name as an identiﬁer gensum/ 37

Slide 79

Slide 79 text

Slide 80

Slide 80 text

Task ❷ Loop unrolling 38

Slide 81

Slide 81 text

Replace the for loop by a statement sequence [cstms| … |] quote a statement sequence $stms:… splice a statement sequence 39

Slide 82

Slide 82 text

genSum arr n = [citem| { int sum = 0; $stms:(additions 0) } |] where additions i | i == n = [] | otherwise = [cstms| sum += $id:(show arr)[ $int:i ]; $stms:(additions (i + 1)) |] Replace the for loop by a statement sequence [cstms| … |] quote a statement sequence $stms:… splice a statement sequence 39

Slide 83

Slide 83 text

Workshop Part 3 Inline Objective-C Code 40 * http://hackage.haskell.org/package/language-c-inline

Slide 84

Slide 84 text

{-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import [""] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit minimal/ 41 * Complete the skeleton in the exercise pack

Slide 85

Slide 85 text

{-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import [""] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit imports for the generated Objective-C code minimal/ 41 * Complete the skeleton in the exercise pack

Slide 86

Slide 86 text

{-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import [""] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit free variables imports for the generated Objective-C code minimal/ 41 * Complete the skeleton in the exercise pack

Slide 87

Slide 87 text

{-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import [""] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit free variables imports for the generated Objective-C code ﬁnalise inline code minimal/ 41 * Complete the skeleton in the exercise pack

Slide 88

Slide 88 text

Task ❸ Bridging classes 42

Slide 89

Slide 89 text

particle/ Particle.hs Represent Haskell record as ObjC class Main.hs Allocate instance and use it 43

Slide 90

Slide 90 text

particle/ Particle.hs Represent Haskell record as ObjC class Main.hs Allocate instance and use it Complete the ObjC class implementation 43

Slide 91

Slide 91 text

Task ❹ A simple application 44

Slide 92

Slide 92 text

app/ Interpreter.hs Haskell interface to package ghc AppDelegate.hs Haskell-Objective-C bridge 45

Slide 93

Slide 93 text

app/ Interpreter.hs Haskell interface to package ghc AppDelegate.hs Haskell-Objective-C bridge Complete AppDelegate.hs Run with open -a HSApp.app 45

Slide 94

Slide 94 text

Thank you! 46

Slide 95

Slide 95 text

Images from http://wikimedia.org http://openclipart.org GitHub repo https://github.com/mchakravarty/language-c-inline Megamax Modula-2 Screenshot by Dirk Steins