Connecting the JVM to Haskell

Presented at scalatam2019. http://scalatam.com

If you feel torn between using Haskell and the JVM for your next project, ponder no more. This talk will show you techniques and challenges when combining the two. inline-java is a tool we use at Tweag IO to integrate Spark, a large Scala project, with Haskell programs.

inline-java is a Haskell library and compiler plugin that allows to embed fragments of Java code in Haskell modules. This ability offers a practical alternative to reimplementing Java libraries in Haskell. In this talk, I will discuss how inline-java works and what are its limitations. You will hear of using the compiler to check that values are marshaled correctly, about coordinating garbage collectors, and what the experience has been using it with Spark, a Scala framework for distributed computing made available in Haskell through a package called sparkle.

Transcript

1. Connecting the JVM to Haskell Facundo Dom´ ınguez May 3, 2018 Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 1 / 20
2. En esta presentaci´ on 1 Una soluci´ on para combinar los runtimes de Haskell y la JVM 2 Caso de estudio: Spark Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 2 / 20
3. Por qu´ e integrar Haskell con la JVM Haskell Application Java libraries Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 3 / 20
4. inline-java main = withJVM [] $ do correct <- challengeDialog 2 3 print correct Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 4 / 20
5. inline-java main = withJVM [] $ do correct <- challengeDialog 2 3 print correct Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 4 / 20
6. inline-java main = withJVM [] $ do correct <- challengeDialog 2 3 print correct challengeDialog :: Int32 -> Int32 -> IO Bool challengeDialog x y = [java| { String question = "¿Cu´ anto es " + $x + " + " + $y + "?"; String response = javax.swing.JOptionPane.showInputDialog(question); try { return Integer.parseInt(response) == $x + $y; } catch (NumberFormatException e) { return false; } } |] Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 4 / 20
7. Por qu´ e no usar bindings? String showInputDialog(Object message) Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 5 / 20
8. Por qu´ e no usar bindings? String showInputDialog(Object message) showInputDialog :: JObject -> IO String Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 5 / 20
9. Por qu´ e no usar bindings? String showInputDialog(Object message) showInputDialog :: JObject -> IO String Bindings Application Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 5 / 20
10. Por qu´ e no usar bindings? module A where challengeDialog :: ... Application class Challenge { boolean challengeDialog(...) { ... showInputDialog ... ... parseInt ... } } Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 6 / 20
11. Java Native Interface (JNI) defineClass :: ClassName -> ByteString -> IO JClass findClass :: ClassName -> IO JClass getStaticMethodID :: JClass -> MethodName -> MethodTypeSignature -> IO JMethodID callStaticMethod :: JClass -> JMethodID -> [...] -> IO ... Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 7 / 20
12. inline-java challengeDialog :: Int32 -> Int32 -> IO Bool challengeDialog x y = [java| { String question = "¿Cu´ anto es " + $x + " + " + $y + "?"; String response = javax.swing.JOptionPane.showInputDialog(question); try { return Integer.parseInt(response) == $x + $y; } catch (NumberFormatException e) { return false; } } |] Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 8 / 20
13. inline-java module A where ... [java| { foreign code } |] ... class ClassForModuleA { static boolean m0(...) { foreign code } } Bytecode de ClassForModuleA module A where ... JNI para darle el bytecode a la JVM e invocar m0 ... javac inline-java A.o Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 9 / 20
14. En resumen inline-java encuentra errores de tipo al compilar, reduce la necesidad de grandes librer´ ıas de bindings, y automatiza la compilaci´ on y enlazado de Java. Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 10 / 20
15. Spark https://spark.apache.org/ Spark la interfase de programaci´ on Spark el entorno para ejecutar computaciones distribuidas Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 11 / 20
16. Spark https://spark.apache.org/ Spark la interfase de programaci´ on Spark el entorno para ejecutar computaciones distribuidas Worker 1 Worker 2 Worker N Dataset Driver Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 11 / 20
17. sparkle https://github.com/tweag/sparkle Bindings para Spark Soluci´ on para ejecutar Haskell en una aplicaci´ on de Spark Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 12 / 20
18. sparkle https://github.com/tweag/sparkle Bindings para Spark Soluci´ on para ejecutar Haskell en una aplicaci´ on de Spark spark application.jar Main.class ... Other.class Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 12 / 20
19. sparkle https://github.com/tweag/sparkle Bindings para Spark Soluci´ on para ejecutar Haskell en una aplicaci´ on de Spark spark application.jar Main.class ... Other.class app.zip shared haskell library.so dependency1.so ... dependencyN.so Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 12 / 20
20. Bindings de Spark newtype Dataset a = Dataset (J ('Class "org.apache.spark.sql.Dataset")) count :: Dataset a -> IO Int64 map :: (a -> b) -> Dataset a -> IO (Dataset b) reduce :: (a -> a -> a) -> Dataset a -> IO a filter ::(a -> Bool) -> Dataset a -> IO (Dataset a) Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 13 / 20
21. Conversi´ on en bloque Convertir [Int] es menos costoso que convertir los enteros de a uno. Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 14 / 20
22. Conversi´ on en bloque Convertir [Int] es menos costoso que convertir los enteros de a uno. Convertir ([a], [b]) es m´ as barato que convertir [(a,b)]. Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 14 / 20
23. Conversi´ on en bloque Convertir [Int] es menos costoso que convertir los enteros de a uno. Convertir ([a], [b]) es m´ as barato que convertir [(a,b)]. Convertir [[a]] es m´ as barato que convertir las listas de a una. Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 14 / 20
24. Conversi´ on en bloque Convertir [Int] es menos costoso que convertir los enteros de a uno. Convertir ([a], [b]) es m´ as barato que convertir [(a,b)]. Convertir [[a]] es m´ as barato que convertir las listas de a una. [Compuesto] -> ([A], [B], [C], ...) --<language barrier>--> ([ForeignA], [ForeignB], ...) -> [ForeignComposed] Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 14 / 20
25. Conversi´ on en bloque Convertir [Int] es menos costoso que convertir los enteros de a uno. Convertir ([a], [b]) es m´ as barato que convertir [(a,b)]. Convertir [[a]] es m´ as barato que convertir las listas de a una. [Compuesto] -> ([A], [B], [C], ...) --<language barrier>--> ([ForeignA], [ForeignB], ...) -> [ForeignComposed] https: //github.com/tweag/inline-java/tree/master/jvm-batching https: //github.com/tweag/inline-java/tree/master/jvm-streaming Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 14 / 20
26. Bindings de Spark newtype Row a = Row (J ('Class "org.apache.spark.sql.Row")) newtype Column = Column (J ('Class "org.apache.spark.sql.Column")) select :: Dataset Row -> [Column] -> IO (Dataset Row) whereDS :: Dataset Row -> Column -> IO (Dataset Row) Examples: embelished <- select employees [col "name", col "wage" * 10] expensive <- whereDS employees (col "wage" > 10000) Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 15 / 20
27. Bindinds de Spark map :: (a -> b) -> Dataset a -> IO (Dataset b) reduce :: (a -> a -> a) -> Dataset a -> IO a filter ::(a -> Bool) -> Dataset a -> IO (Dataset a) Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 16 / 20
28. Recolecci´ on de basura ... coordinada ... Haskell JVM Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 17 / 20
29. Recolecci´ on de basura ... coordinada ... Recuerde borrar cada referencia luego de usarla Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 18 / 20
30. Recolecci´ on de basura ... coordinada ... Recuerde borrar cada referencia luego de usarla bracket createRef deleteRef useRef Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 18 / 20
31. Recolecci´ on de basura ... coordinada ... Recuerde borrar cada referencia luego de usarla bracket createRef deleteRef useRef Linear types https://github.com/ghc-proposals/ghc-proposals/pull/111 Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 18 / 20
32. Recolecci´ on de basura ... coordinada ... Recuerde borrar cada referencia luego de usarla bracket createRef deleteRef useRef Linear types https://github.com/ghc-proposals/ghc-proposals/pull/111 Notificaciones del recolector en la JVM Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 18 / 20
33. Impresiones finales sparkle permite un uso eficiente de Spark desde Haskell con algunas restricciones. Queda por explorar una soluci´ on mejor para la coordinaci´ on de GCs. Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 19 / 20
34. Referencias inline-java: https://github.com/tweag/inline-java Tutorial de inline-java: https://www.tweag.io/posts/ 2017-09-15-inline-java-tutorial.html Sobre la implementaci´ on de inline-java: https://www.tweag. io/posts/2017-09-22-inline-java-ghc-plugin.html Spark: https://spark.apache.org/ sparkle: https://github.com/tweag/sparkle Propuesta de tipos lineales: https://github.com/ghc-proposals/ghc-proposals/pull/111 Sobre coordinaci´ on de recolectores de basura: https://www.tweag.io/posts/2017-11-29-linear-jvm.html Facundo Dom´ ınguez (tweag.io) Connecting the JVM to Haskell May 3, 2018 20 / 20