The Real Real World of Haskell

Slides from my talk at DevoxxFR 2016

Transcript

1. The Real Real World The Real Real World of Haskell of Haskell Relations on the Land of Pure Functional Relations on the Land of Pure Functional Programming by a Mortal Being Programming by a Mortal Being Arnaud Bailly Arnaud Bailly April 22, 2016 April 22, 2016
2. Table of Contents Table of Contents How I Came to Reside in the Land of Haskell and Why It Matters On Various Aspects of Haskelland Civilisation and The Mores of Its Inhabitants Some Ways this Wonderful Journey Could Inspire the Reader
3. How I Came to Visit How I Came to Visit Haskelland Haskelland
5. Where the author dares to present himself Where the author dares to present himself
6. Where the author dares to present himself Where the author dares to present himself 20+ years of experience developing software… Has been coding in Haskell for side-projects since 2001 Had the opportunity to join Capital Match as Co-founder and CTO in 2014 Haskell was part of the plan since the beginning and the reason why I got involved I like theory but I prefere working software
7. On the Company which I travelled to On the Company which I travelled to Haskelland for Haskelland for
8. On the Company which I travelled to On the Company which I travelled to Haskelland for Haskelland for Capital Match is the leading plaform in Singapore for peer-to-peer lending to SMEs Backend system developed in Haskell, frontend in Clojurescript/Om since 2014 Core Development team of 4 persons
9. Where it is demonstrated why the interest Where it is demonstrated why the interest of the honorable reader might be aroused of the honorable reader might be aroused This is not a Haskell tutorial This is not a Monad tutorial Let’s stop thinking “Haskell is good for your brain but not for practical use” I will be happy if you end up thinking: I could try this cool stuff in my next microservice I will be even happier if you can put to a ac ct tu ua al l u us se e some of the stuff I present here
10. On Various Aspects of On Various Aspects of Haskelland Haskelland
12. Where the author tries to cover the most important aspects of Haskeller’s civilisation in an extremely short span of time, thus probably missing lot of important points and generating frustration in the informed reader, but nevertheless hoping to convey enough tips for the audience to be willing to adopt some of the customs of those strange people.
13. Language Language
14. On some minor differences between On some minor differences between Haskell Language and more Common ones Haskell Language and more Common ones
15. Haskell’s syntax favours terseness and strives to be close to mathematical language, compare: with: {(x, 2x)|x 㱨 [1, 100]㱸x mod 3 = 0} Haskellers don’t use lot of punctuation signs but favor using indentation to express nesting [ (x, x * 2) | x <- [ 0 .. 100 ], x `mod` 3 == 0 ]
16. On the importance of composition in On the importance of composition in Haskell Haskell
17. Haskell language is geared towards making composition of expressions straightforward Referential transparency allows one to factorize all kind of common sub-expressions to remove redundancy Partial application and composition of functions makes it easy to write function pipelines and build new functions out of old ones transactionsFromCashflows :: AccountId -> (CashFlow -> [Transaction]) -> [CashFlow] -> [Transaction] transactionsFromCashflows pivot generator = concatMap $ map (normalize . balance pivot) . generator
18. On the virtue of being lazy On the virtue of being lazy
19. Haskell’s evaluation semantics is lazy, non strict or call-by-need: Arguments to functions are evaluated only when needed This makes it practical to express infinite computations in the language This applies also to I/O operations which may not be evaluated hence have no side-effects if their result is not needed
20. On some of the less desirable aspects of the On some of the less desirable aspects of the language language
21. No support for first-class module which makes large scale development more cumbersome A natural tendency to abuse operators and be too terse and dense too the point of becoming cryptic Beginners should expect to be confused at first by how the system supports strings and numbers Lazy I/O can have annoying side-effects There are too many ways to handler exceptions
22. Philosophy Philosophy
23. On the importance of Types for Haskellers On the importance of Types for Haskellers
24. Haskellers seems to be extreme platonicists: Everything must have a well-defined type! Although types can be inferred by the compiler, they are an extremely important tool to design programs Haskell’s type system can be daunting at first but is conceptually elegant and compact
25. Where one discovers types do not impose Where one discovers types do not impose extra burden at runtime extra burden at runtime newtypes provide cheap encapsulation of other types as they are unpacked by compiler: One should never refrain from creating such new types, they provide more safety than type aliases newtype AccountNr = AccountNr { accountNr :: T.Text } deriving (Eq,Ord,Show,Read) instance IsString AccountNr where fromString = AccountNr . T.pack
26. Where it is shown types can even conjure Where it is shown types can even conjure the undead the undead Phantom types provide type-level annotation to add more information to other types Allow distinguishing between different types with identical representations Provide thread-safety by ensuring some type variables do not data Base64 data Hex newtype Encoded code = Encoded { encodedText :: Text } toBase64Text :: ByteString -> Encoded Base64 toHex :: ByteString -> Encoded Hex escape local scope
27. Where one discovers subtle differences Where one discovers subtle differences cause more trouble than obvious ones cause more trouble than obvious ones can be thought as way to define interfaces, asbtracting away implementation details Type classes can define not only functions, but values and types Type classes actually define constraints on types that can be accumulated in expressions and checked by the compiler Type classes class (Eq a) => Differ a where uniqueId :: a -> Id a constructor :: a -> Entity b equals :: a -> a -> Bool
28. Where one starts to wonder if there is an Where one starts to wonder if there is an end to recursion end to recursion Type families provide type-level functions, e.g. ways to compute types from types at compilation time Type families can be nested within data and class definitions to provide contextual types They can also be used at toplevel: class (ToJSON (Command a)) => BusinessModel a where data Event a :: * data Command a :: * type family Id a :: * type instance Id Account = AccountId type instance Id Transaction = TransactionId
29. Where it is shown how more constraints Where it is shown how more constraints makes one more Free makes one more Free data Gratis f a = Effectless a | forall x. Effectful (f x) (x -> Gratis f a) data EmailServiceF a where DoMail :: Emailer -> Email -> EmailServiceF EmailStatus GetAllEmails :: Confirmation -> EmailServiceF [EmailWithStatus] type MailService = Gratis EmailServiceF liftFF :: EmailServiceF a -> MailService a liftFF c = Effectful c pure doMail :: Emailer -> Email -> MailService EmailStatus doMail mailer mail = liftFF $ DoMail mailer mail doGetAllEmails :: Confirmation -> MailService [ EmailWithStatus ] doGetAllEmails = liftFF . GetAllEmails interpret :: MailService a -> ExceptT L.Text (WebStateM s l m) a interpret (Effectful (DoMail mailer mail) f) = lift (liftIO $ mailer mail >>= handleSendingResult) >>= interpret . f interpret (Effectful (GetAllEmails confirmation) f) = lift (runWithEmails $ doGetEmails confirmation) >>= interpret . f interpret (Effectless a) = return a
30. Mores & Daily Life Mores & Daily Life
31. Where the amazed foreigner discovers how Where the amazed foreigner discovers how much can be done with crude tools much can be done with crude tools
33. There is no one-stop Integrated Development Environment for Haskell: Best support is provided by integrating various extensions into E Em ma ac cs s There is a nice on how to setup emacs and Chris Done has provided some Makes it easy to setup remote pairing is a recent initiative to provide an interactive and easy to use programming environment in Haskell provides a viable alternative with “everything” preconfigured and packaged tutorial standard configuration Haskell for Mac Spacemacs
34. On the usefulness and efficiency of On the usefulness and efficiency of interacting with the machine interacting with the machine
35. Use the REPL, Luke! Provides much faster feedback loop than full-blown compilation and building, can be used for Type/Test-DD too GHC has a cool feature to handle h ho ol le es s: variables which are not in scope but typechecked so that one can use that to GHCi comes with a debugger but I have never used it: provides a Haskell kernel for IPython notebooks for interactive programming. Provides a nice alternative to text-only development in Haskell esp. for number crunching deduce needed type and implementation Debugging sucks, testing rocks IHaskell
37. On the typical day of a commoner in On the typical day of a commoner in Haskell land Haskell land Write a skeletal test file using , e.g. test/FooTest.hs 1. Start REPL in Emacs by loading file C-c C-l 2. See it fail to compile 3. :reload until it compiles 4. Run the test and see it fail: hspec myTest 5. Fill in code until test passes 6. Do a full compile and test run before pushing to CI 7. hspec
38. Government Government
39. Where we discover Haskellers also value Where we discover Haskellers also value experimentation experimentation
40. There are quite a few testing frameworks available to write unit or integration tests I favour which is one of the many rspec-inspired tools provides bindings to in Haskell Running tests is automated as part of build tools, e.g. cabal hspec hs-webdriver Selenium
41. Where we see empiricism going hand in Where we see empiricism going hand in hand with formalism hand with formalism is the property-based testing tool for Haskell Use it for defining formal properties of your code beyond what type system provides QuickCheck instance Arbitrary ScaleRatio where arbitrary = ... instance Arbitrary Transaction where arbitrary = ... prop_scaled_transaction_is_normalized :: Transaction -> ScaleRatio -> Bool prop_scaled_transaction_is_normalized tx (ScaleRatio ratio) = isNormalized tx' && isBalanced tx' where tx' = scale ratio tx
42. Where one discovers QuickCheck can be Where one discovers QuickCheck can be useful beside testing properties useful beside testing properties Generating sample data to test Generating sample data to test migration migration Generate sample data sets to be used as part of other kind of tests: We used it to generate test data for schema migration code sample_v8_Events :: [ByteString] sample_v8_Events = ["{\"tag\":\"AddedDocuments\",\"contents\":[\"c1f09023b3a9398a1d8a257c372392ab\",[ , "{\"tag\":\"UpdatedInvestor\",\"contents\":{\"invDocuments\":[], \"invBankAccount\":{\"bankBranch\":\"Ge\",\"bankAccountName\":\"1W\",\"bankAcco \"invId\":\"46ed3336bc60a5a423962e9b6343c003\",\"invReferralCode\":{\"refer\":\ \"invLegalEntity\":{\"tag\":\"Corporate\",\"primaryContactPosition\":{\"tag\":\ , "{\"tag\":\"NoEvent\",\"contents\":[]}" , "{\"tag\":\"AddedDocuments\",\"contents\":[\"b9973afdd569f11269fe0be736086049\"
43. Generating execution sequences Generating execution sequences representing typical scenarios representing typical scenarios genAdminActions = frequency [ (1, return []) , (1, do f <- NewListing <$> arbitrary <*> newFacility <*> rate (f:) <$> genAdminActions) , (10, (AcceptSomeFacility:) <$> genAdminActions) , (5, (LookupFacilities:) <$> genAdminActions) , (6, (AdvanceDay:) . (ConfirmRepayments:) <$> genAdminActions) ] provides needed tooling to embed data generation within effectful code Monadic QuickCheck
44. On the use of different level of testing On the use of different level of testing
45. Architecture Architecture
46. On the importance of laying out strong On the importance of laying out strong foundations foundations
47. GHC has become the standard compiler for Haskell, current version is 7.10.3 It includes a lot of optimisations and features from 20 years of research on programming languages and compilation Always be sure to turn on -Wall -Werror when compiling: Warnings are often signs of potential troubles C Ca av ve ea at t: Compilation can take a long time but still does not yet beat scalac’s slowness
48. On the intricacies of building complex On the intricacies of building complex software with a Cabal of scholars software with a Cabal of scholars
49. Standard package management tool is which has become infamous due to Cabal is not very good at managing complex package structures out-of-the-box greatly improves thing in order to provide fully reproducible builds and sandboxed environments, down to compiler versions provides a solution to package management in Haskell Cabal cabal hell stack nix
50. Diplomacy Diplomacy
51. Where we separate that which is pure Where we separate that which is pure from that which is impure from that which is impure
52. Type system provides a clean way to distinguish pure values from impure effects Staying pure has predictable semantics and opens the door to optimisations Effectful computations are one way portal to the real world
53. Where we learn we can have our cake and Where we learn we can have our cake and eat it too eat it too
54. Concurrency in Haskell is mostly based on composable operations STM do not mix with I/O operations hence cannot have observable side-effects but the upside is that runtime can detect deadlocks a high-level library to package asynchronous computations within cheap Haskell threading model We used actor-like queues to protect access to I/O resources, e.g. database. Haskell provide a lot of useful abstractions on top of STM and core concurrency features to build your own tools Software Transactional Memory async
55. How Haskellers do communicate with the How Haskellers do communicate with the REST of the world REST of the world
56. provides all the needed components to build high performance REST-based services in Haskell Initial development was done with which is a lightweight framework inspired by Sinatra Definitely need to have a look at which is a kind of “successor” to Scotty WAI Scotty Spock
57. Where types make mortal beings doubt Where types make mortal beings doubt about their sanity about their sanity Newer services are now developed with which provides a way to express APIs a at t t th he e t ty yp pe e l le ev ve el l Makes it possible to derive server, client or swagger specs from a single declaration Servant type CreateJob = ReqBody '[JSON] Job :> Post '[JSON] JobId type ListJobs = Get '[JSON] [Job] type RemoveJob = Capture "jobid" JobId :> Delete '[JSON] JobId type SchedulerApi = "api" :> "scheduler" :> "jobs" :> CreateJob :<|> "api" :> "scheduler" :> "jobs" :> ListJobs :<|> "api" :> "scheduler" :> "jobs" :> RemoveJob
58. On some ways data is persisted to stable On some ways data is persisted to stable storage storage
59. Haskell has bindings to most open-source or standard (R)DBMS but we did not used them Seems like perfect environment to use We developed in-house solution to store events generated by the application as a sequential log of events stored in a file on disk event sourcing
61. War War
62. Where peaceful Haskellers prepare Where peaceful Haskellers prepare themselves to wage war themselves to wage war
63. is a Haskell tool to replace make and orchestrate complex builds is our Continuous Integration tool We used both to build and deploy all components of the system as containers Both provides way to express complex build processes as type-safe declarative rules Shake bake docker
64. On the war-time organisation of On the war-time organisation of Haskell’s legions Haskell’s legions
65. is a configuration management tool in Haskell developed by of Debian fame Configuration is expressed as a Haskell program that is compiled and run on the target host It provides a nice gpg key based model to encrypt private data (e.g. ssh keys, authentication tokens, passwords…) that need to be deployed in the source directory Once again, it allows expressing configuration items in a type-safe way propellor Joey Hess
66. On the capabilities to wage war outside of On the capabilities to wage war outside of Haskelland Haskelland
67. is Haskell client for interacting with ’s API where we deploy all our containers There is thorough Haskell support for AWS API through and Google’s APIs through Monitoring is done through a hdo Digital Ocean Amazonka gogol riemann Haskell client
68. Conclusion Conclusion
69. What benefits one can expect from visiting What benefits one can expect from visiting Haskelland Haskelland
70. Strong type discipline and powerful compiler removes a whole class of programming errors Types are also great to design complex software systems and yield great opportunities for verification and optimizations All the best practices you learnt are still valid in Haskelland, only a bit differently practiced You can stay away from “hairy” stuff while still being productive and have fun
71. What the cautious traveller should be What the cautious traveller should be aware of when ashoring Haskelland aware of when ashoring Haskelland
72. Development environment can be tricky to get - and stay - “right” A lot of the material one can find is research focused which can be daunting Avoid partial functions at all cost It can be hard to find Haskell developers locally
73. Acknowledgements Acknowledgements
74. Credits Credits All the woodcuts illustrating this talk are drawn from the with the exception of: Encyclopédie de Diderot et d’Alembert , The , , . Durer’s “Artist drawing a couching woman” Austerlitz Pyramid Durer’s Rhinoceros Allégorie à la Paix d’Utrecht
75. Thanks Thanks To the Devoxx selection committee for having accepted that talk, To all the fine people at Capital Match who made all this possible, To Haskellers all over the world who fuel the fire with their wits and spirit, To Haskell B. Curry who, among others, laid out the foundations for our daily job.