Foreign Inline Code in Haskell (YOW! Lambda Jam 2014)

Foreign Inline Code in Haskell (YOW! Lambda Jam 2014)

VIDEO: https://www.youtube.com/watch?v=52yvHv_Ahvg

Talk and workshop at YOW! Lambda Jam, Brisbane, 2014.

Template Haskell is a meta programming framework for Haskell implemented by the Glasgow Haskell Compiler (GHC), which is widely used as a template meta-programming system for Haskell, to define macros, code generators, or even code transformation engines. Subsequent support for the quasiquoting of arbitrary programming languages greatly simplified writing code generators in Haskell that produce complex C, CUDA, OpenCL, or Objective-C code by writing code templates in the syntax of the generated language.

Additionally, quasiquoting of C-like languages enables a purely library-based system for inline C code in Haskell. This dramatically simplifies language interoperability, and especially, the use of frameworks and libraries written in C-like languages from Haskell. It is, for example, helpful in applications based on native GUI libraries and projects integrating code written in multiple languages.

In this talk, I will explain the concepts of template meta-programming and quasiquoting and how they are used in Template Haskell. I will demonstrate quasiquoting by way of a few simple and intuitive examples. Finally, I will demonstrate the use of inline C code in Haskell and compare it to other forms of language interoperability as provided by Haskell and other functional languages.

This talk will explore the use, but not the implementation of quasiquoting and inline C & Objective-C code. Hence, the material should be accessible to anybody with an intermediate-level working knowledge of Haskell and C.

This slide set also contains the slides for the associated hands on workshop. If you are interested in going through the exercises, you can get the workshop source code templates and solutions from https://github.com/mchakravarty/ylj14-workshop and you can find details on the prerequisite software setup (which requires OS X) at https://gist.github.com/mchakravarty/fb8ecc1be86df1a23b33

2cc5323ccdfc09b921f1be34b3d78a69?s=128

Manuel Chakravarty

May 08, 2014
Tweet

Transcript

  1. Manuel M T Chakravarty University of New South Wales Foreign

    Inline Code in Haskell mchakravarty α TacticalGrace TacticalGrace justtesting.org 1 30 minute time slot: 25min talking + 5min [15min The Problem; 5min TH+Quasiquoting; 5min Inline Objective-C]
  2. λ Shiny new functional language 2 » Imagine, you have

    got a shiny, new functional language...
  3. 3 » ...and you want to use it to write

    a great new app...
  4. λ 4 » ...then you will need to use many

    existing frameworks and libraries. » Luckily, any serious language will have a foreign function interface! » Haskell standard includes a simple, but versatile FFI
  5. “Problem solved?” 5 » Does that solve the problem of

    language interoperability? » Let me explain that at an example...
  6. “Problem solved?” No! 5 » Does that solve the problem

    of language interoperability? » Let me explain that at an example...
  7. dumpURL :: String -> IO () dumpURL urlString = do

    urlData <- stringWithContentsOfUrl urlString putStr urlData What we want to write 6
  8. dumpURL :: String -> IO () dumpURL urlString = do

    urlData <- stringWithContentsOfUrl urlString putStr urlData What we want to write char *stringWithContentsOfUrlCstub(char *urlString) { NSURL *url = [NSURL URLWithString:urlString]; [NSString stringWithContentsOfURL:url encoding:NSUTF8StringEncoding error:NULL]; } What we want to call (& need to put into an extra file) 6
  9. What we have to write as well… foreign import stringWithContentsOfURLCstub

    :: CString -> IO CString stringWithContentsOfURL :: String -> IO String stringWithContentsOfURL url = withCString url $ \urlC -> do resultC <- stringWithContentsOfURLCstub urlC result <- peek resultC free resultC return result 7 » FFI code: If you find it confusing, that’s fine, as I want to argue that you shouldn’t write it in the first place. * Anything extensively relying on foreign frameworks will be a pain [Hybrid languages (eg, F# & Scala) have a different set of trade offs, but don’t solve it either.]
  10. Bridging Libraries 1-to-1 transliteration of types & functions 8 *

    Bridging or binding libraries transliterate types & functions * Need to be maintained and documented; track multiple versions of base library * Lack of type safety * Tools and dynamic transliteration (by reflection) help
  11. C➙Haskell c2hs A case study GTK+ 9 * c2hs: automation

    for bridging C libraries to Haskell <https://hackage.haskell.org/package/ c2hs> * Implements large parts of a C compiler front-end * Bridge for the cross-platform Gnome GUI library GTK+ <http://projects.haskell.org/ gtk2hs/>
  12. 10 * AFAIK, currently the only fully featured and properly

    maintained Haskell GUI library * Haskell GTK+ library is used for realistic applications
  13. “Does this approach scale?” 11 » Read question * It

    requires significant resources & constant effort to track the original library * Application frameworks: enormous and growing footprint
  14. February 2011 Introduction of GTK+ 3 12 * Base GTK+

    development isn’t even particularly fast * Bridging libraries lag behind their base libraries * Another example: Haskell OpenGL library
  15. February 2011 Introduction of GTK+ 3 December 2013 GTK+ 3

    in Haskell Bridge 12 * Base GTK+ development isn’t even particularly fast * Bridging libraries lag behind their base libraries * Another example: Haskell OpenGL library
  16. 13 * Compared to modern application frameworks, GTK+ is small!

  17. Bridging libraries don’t scale Too much weight! 14

  18. “Interoperability is an old problem — maybe an old solution

    can help?” 15
  19. 16 * Modula-2 with inline 68000 assembly * Assembly code

    can symbolically refer to Modula-2 entities (e.g., variables)
  20. 16 * Modula-2 with inline 68000 assembly * Assembly code

    can symbolically refer to Modula-2 entities (e.g., variables)
  21. 16 * Modula-2 with inline 68000 assembly * Assembly code

    can symbolically refer to Modula-2 entities (e.g., variables)
  22. “Inline C, C++, Objective-C, … in Haskell?” 17 * We

    don’t want to build front ends for a few more languages into GHC * How do we share entities between the languages?
  23. Meta-programming Template Haskell 18 » Generic infrastructure for program manipulation

    NOTE: I'll run through this quickly; I'll explain the details at the workshop.
  24. HASKELL WORKSHOP 2002 19 * Template Haskell: Haskell extension implemented

    by GHC * Useful for: defining macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…
  25. HASKELL WORKSHOP 2002 #define macros 19 * Template Haskell: Haskell

    extension implemented by GHC * Useful for: defining macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…
  26. HASKELL WORKSHOP 2002 #define macros [| … |] code generators

    19 * Template Haskell: Haskell extension implemented by GHC * Useful for: defining macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…
  27. HASKELL WORKSHOP 2002 #define macros [| … |] code generators

    trafo (ConE name) = … code transformations 19 * Template Haskell: Haskell extension implemented by GHC * Useful for: defining macros, code generators, code transformations… * Other languages have their own variants; eg., MetaOCaml » Let’s look at an example…
  28. $(sel 1 3) (a, b, c) = a 20 *

    Meta function executed at splice point, generating spliced code
  29. $(sel 1 3) (a, b, c) = a meta-programming function

    20 * Meta function executed at splice point, generating spliced code
  30. $(sel 1 3) (a, b, c) = a meta-programming function

    splice 20 * Meta function executed at splice point, generating spliced code
  31. $(sel 1 3) (a, b, c) = a meta-programming function

    splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a 20 * Meta function executed at splice point, generating spliced code
  32. $(sel 1 3) (a, b, c) = a meta-programming function

    splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a $(sel 5 5) (a, b, c, d, e) = e 20 * Meta function executed at splice point, generating spliced code
  33. $(sel 1 3) (a, b, c) = a meta-programming function

    splice \tup -> case tup of {(x, y, z) -> x} :: (a, b, c) -> a $(sel 5 5) (a, b, c, d, e) = e \tup -> case tup of {(x, y, z, v, w) -> w} :: (a, b, c, d, e) -> e 20 * Meta function executed at splice point, generating spliced code
  34. sel :: Int -> Int -> ExpQ sel i n

    = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop
  35. sel :: Int -> Int -> ExpQ sel i n

    = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop
  36. sel :: Int -> Int -> ExpQ sel i n

    = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions quasi-quotation 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop
  37. sel :: Int -> Int -> ExpQ sel i n

    = [| \tup -> case tup of {$pat -> $res} |] where pat = tupP (map varP names) res = varE (names !! (i - 1)) names = [mkName $ "v" ++ show i | i <- [1..n]] type of Haskell expressions quasi-quotation spliced expression (anti quote) 21 * Quasiquotations in [|..|] brackets * Explain TH in more detail in the workshop
  38. Processing other languages Generic quasi-quotation 22 * Quoting arbitrary languages

  39. HASKELL WORKSHOP 2007 Quasiquoting for any language you can provide

    a parser for 23
  40. Language.C.Quote add n = [cfun| int addConstant(int x) { return

    x + $int:n; } |] 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C
  41. Language.C.Quote add n = [cfun| int addConstant(int x) { return

    x + $int:n; } |] quasi-quotation identifier 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C
  42. Language.C.Quote add n = [cfun| int addConstant(int x) { return

    x + $int:n; } |] quasi-quotation identifier splice identifier 24 * QQ for C including some GNU extensions, parts of CUDA & OpenCL, and all of Objective-C
  43. mkMap dev aenv fun arr = return $ CUTranslSkel "map"

    [cunit| $esc:("#include <accelerate_cuda.h>") extern "C" __global__ void map ($params:argIn, $params:argOut) { const int shapeSize = size(shOut); const int gridSize = $exp:(gridSize dev); int ix; for ( ix = $exp:(threadIdx dev) ; ix < shapeSize ; ix += gridSize ) { $items:(dce x .=. get ix) $items:(setOut "ix" .=. f x) } } |] where ... 25 * Accelerate: embedded high-performance array language for GPUs * Combinators as code skeletons (code templates with holes) * Yellow splices/anti-quotes are the holes (parameters) of the template * Doing this with strings or explicit AST construction would be much worse
  44. mkMap dev aenv fun arr = return $ CUTranslSkel "map"

    [cunit| $esc:("#include <accelerate_cuda.h>") extern "C" __global__ void map ($params:argIn, $params:argOut) { const int shapeSize = size(shOut); const int gridSize = $exp:(gridSize dev); int ix; for ( ix = $exp:(threadIdx dev) ; ix < shapeSize ; ix += gridSize ) { $items:(dce x .=. get ix) $items:(setOut "ix" .=. f x) } } |] where ... 25 * Accelerate: embedded high-performance array language for GPUs * Combinators as code skeletons (code templates with holes) * Yellow splices/anti-quotes are the holes (parameters) of the template * Doing this with strings or explicit AST construction would be much worse
  45. mkMap dev aenv fun arr = return $ CUTranslSkel "map"

    [cunit| $esc:("#include <accelerate_cuda.h>") extern "C" __global__ void map ($params:argIn, $params:argOut) { const int shapeSize = size(shOut); const int gridSize = $exp:(gridSize dev); int ix; for ( ix = $exp:(threadIdx dev) ; ix < shapeSize ; ix += gridSize ) { $items:(dce x .=. get ix) $items:(setOut "ix" .=. f x) } } |] where ... 25 * Accelerate: embedded high-performance array language for GPUs * Combinators as code skeletons (code templates with holes) * Yellow splices/anti-quotes are the holes (parameters) of the template * Doing this with strings or explicit AST construction would be much worse
  46. Inlining foreign languages Inline C Code 26

  47. Language.C.Inline dumpURL :: String -> IO () dumpURL urlString =

    do urlData <- putStr urlData stringWithContentsOfUrl urlString 27 * Inline Objective-C * All the FFI code is generated automatically * Again, details in the workshop
  48. Language.C.Inline dumpURL :: String -> IO () dumpURL urlString =

    do urlData <- putStr urlData $(objc ['urlString] ''String [cexp| [NSString stringWithContentsOfURL: [NSURL URLWithString:urlString] encoding:NSUTF8StringEncoding error:NULL] |]) inline Objective-C splice 27 * Inline Objective-C * All the FFI code is generated automatically * Again, details in the workshop
  49. Inline C …is simple 28 * Lack of bridge saves

    a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language
  50. Inline C …is simple No bridging library needs to be

    maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language
  51. Inline C …is simple No extra documentation No bridging library

    needs to be maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language
  52. Inline C …is simple No extra documentation Some type safety

    No bridging library needs to be maintained 28 * Lack of bridge saves a lot of work; instant access to new versions * Bridges require familiarity with native libraries already; inline code leverages that directly * Haskell types are automatically mapped to types of the inline language
  53. Language integration by language inlining types >< state languages 29

    Outline workshop * Get familiar with TH and Language.C.Quote * Run through three exercises using inline Objective-C including a graphical REPL
  54. Workshop Part 1 Template Haskell 30 * http://www.haskell.org/haskellwiki/Template_Haskell * http://hackage.haskell.org/package/template-haskell

  55. Clone https://github.com/mchakravarty/ylj14-workshop 31

  56. 32

  57. 32

  58. 32

  59. 33

  60. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) 33
  61. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) data Format = D | S | L String parse :: String -> [Format] 33
  62. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) data Format = D | S | L String parse :: String -> [Format] gen :: [Format] -> ExpQ gen [D] = [| \n -> show n |] gen [S] = [| \s -> s |] gen [L s] = [| s |] 33
  63. Task ❶ A recursive generator 34

  64. printf/ 35

  65. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] printf/ 35
  66. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ printf/ 35
  67. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix printf/ 35
  68. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix printf/ 35
  69. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix = [| \n -> $(gen fmt [| $prefix ++ show n |]) |] printf/ 35
  70. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix = [| \n -> $(gen fmt [| $prefix ++ show n |]) |] gen (S : fmt) prefix printf/ 35
  71. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix = [| \n -> $(gen fmt [| $prefix ++ show n |]) |] gen (S : fmt) prefix = [| \s -> $(gen fmt [| $prefix ++ s |]) |] printf/ 35
  72. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix = [| \n -> $(gen fmt [| $prefix ++ show n |]) |] gen (S : fmt) prefix = [| \s -> $(gen fmt [| $prefix ++ s |]) |] gen (L s : fmt) prefix printf/ 35
  73. printf :: String -> ExpQ printf fmt = gen (parse

    fmt) [| "" |] gen :: [Format] -> ExpQ -> ExpQ gen [] prefix = prefix gen (D : fmt) prefix = [| \n -> $(gen fmt [| $prefix ++ show n |]) |] gen (S : fmt) prefix = [| \s -> $(gen fmt [| $prefix ++ s |]) |] gen (L s : fmt) prefix = gen fmt [| $prefix ++ s |] printf/ 35
  74. Workshop Part 2 Quasi-quotation of C variants 36 * http://hackage.haskell.org/package/language-c-quote

  75. gensum/ 37

  76. genSum :: Name -> Int -> BlockItem genSum arr n

    = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] gensum/ 37
  77. genSum :: Name -> Int -> BlockItem genSum arr n

    = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] custom parser for C block items gensum/ 37
  78. genSum :: Name -> Int -> BlockItem genSum arr n

    = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] custom parser for C block items splicing of a name as an identifier gensum/ 37
  79. genSum :: Name -> Int -> BlockItem genSum arr n

    = [citem| { int sum = 0; for (int i = 0; i++; i < $n) sum += $id:(show arr)[i]; } |] custom parser for C block items splicing of a name as an identifier default is to splice as an expression gensum/ 37
  80. Task ❷ Loop unrolling 38

  81. Replace the for loop by a statement sequence [cstms| …

    |] quote a statement sequence $stms:… splice a statement sequence 39
  82. genSum arr n = [citem| { int sum = 0;

    $stms:(additions 0) } |] where additions i | i == n = [] | otherwise = [cstms| sum += $id:(show arr)[ $int:i ]; $stms:(additions (i + 1)) |] Replace the for loop by a statement sequence [cstms| … |] quote a statement sequence $stms:… splice a statement sequence 39
  83. Workshop Part 3 Inline Objective-C Code 40 * http://hackage.haskell.org/package/language-c-inline

  84. {-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import

    ["<Foundation/Foundation.h>"] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit minimal/ 41 * Complete the skeleton in the exercise pack
  85. {-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import

    ["<Foundation/Foundation.h>"] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit imports for the generated Objective-C code minimal/ 41 * Complete the skeleton in the exercise pack
  86. {-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import

    ["<Foundation/Foundation.h>"] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit free variables imports for the generated Objective-C code minimal/ 41 * Complete the skeleton in the exercise pack
  87. {-# LANGUAGE TemplateHaskell, QuasiQuotes #-} import Language.C.Quote.ObjC import Language.C.Inline.ObjC objc_import

    ["<Foundation/Foundation.h>"] nslog :: String -> IO () nslog msg = $(objc ['msg] ''() [cexp| NSLog(@"Here is a message from Haskell: %@”, ! ! ! ! ! ! msg) |]) objc_emit free variables imports for the generated Objective-C code finalise inline code minimal/ 41 * Complete the skeleton in the exercise pack
  88. Task ❸ Bridging classes 42

  89. particle/ Particle.hs Represent Haskell record as ObjC class Main.hs Allocate

    instance and use it 43
  90. particle/ Particle.hs Represent Haskell record as ObjC class Main.hs Allocate

    instance and use it Complete the ObjC class implementation 43
  91. Task ❹ A simple application 44

  92. app/ Interpreter.hs Haskell interface to package ghc AppDelegate.hs Haskell-Objective-C bridge

    45
  93. app/ Interpreter.hs Haskell interface to package ghc AppDelegate.hs Haskell-Objective-C bridge

    Complete AppDelegate.hs Run with open -a HSApp.app 45
  94. Thank you! 46

  95. Images from http://wikimedia.org http://openclipart.org GitHub repo https://github.com/mchakravarty/language-c-inline Megamax Modula-2 Screenshot

    by Dirk Steins