$30 off During Our Annual Pro Sale. View details »

The Real Real World of Haskell

The Real Real World of Haskell

Slides from my talk at DevoxxFR 2016

Arnaud Bailly

April 22, 2016
Tweet

More Decks by Arnaud Bailly

Other Decks in Technology

Transcript

  1. The Real Real World The Real Real World of Haskell

    of Haskell Relations on the Land of Pure Functional Relations on the Land of Pure Functional Programming by a Mortal Being Programming by a Mortal Being Arnaud Bailly Arnaud Bailly April 22, 2016 April 22, 2016
  2. Table of Contents Table of Contents How I Came to

    Reside in the Land of Haskell and Why It Matters On Various Aspects of Haskelland Civilisation and The Mores of Its Inhabitants Some Ways this Wonderful Journey Could Inspire the Reader
  3. How I Came to Visit How I Came to Visit

    Haskelland Haskelland
  4. None
  5. Where the author dares to present himself Where the author

    dares to present himself
  6. Where the author dares to present himself Where the author

    dares to present himself 20+ years of experience developing software… Has been coding in Haskell for side-projects since 2001 Had the opportunity to join Capital Match as Co-founder and CTO in 2014 Haskell was part of the plan since the beginning and the reason why I got involved I like theory but I prefere working software
  7. On the Company which I travelled to On the Company

    which I travelled to Haskelland for Haskelland for
  8. On the Company which I travelled to On the Company

    which I travelled to Haskelland for Haskelland for Capital Match is the leading plaform in Singapore for peer-to-peer lending to SMEs Backend system developed in Haskell, frontend in Clojurescript/Om since 2014 Core Development team of 4 persons
  9. Where it is demonstrated why the interest Where it is

    demonstrated why the interest of the honorable reader might be aroused of the honorable reader might be aroused This is not a Haskell tutorial This is not a Monad tutorial Let’s stop thinking “Haskell is good for your brain but not for practical use” I will be happy if you end up thinking: I could try this cool stuff in my next microservice I will be even happier if you can put to a ac ct tu ua al l u us se e some of the stuff I present here
  10. On Various Aspects of On Various Aspects of Haskelland Haskelland

  11. None
  12. Where the author tries to cover the most important aspects

    of Haskeller’s civilisation in an extremely short span of time, thus probably missing lot of important points and generating frustration in the informed reader, but nevertheless hoping to convey enough tips for the audience to be willing to adopt some of the customs of those strange people.
  13. Language Language

  14. On some minor differences between On some minor differences between

    Haskell Language and more Common ones Haskell Language and more Common ones
  15. Haskell’s syntax favours terseness and strives to be close to

    mathematical language, compare: with: {(x, 2x)|x 㱨 [1, 100]㱸x mod 3 = 0} Haskellers don’t use lot of punctuation signs but favor using indentation to express nesting [ (x, x * 2) | x <- [ 0 .. 100 ], x `mod` 3 == 0 ]
  16. On the importance of composition in On the importance of

    composition in Haskell Haskell
  17. Haskell language is geared towards making composition of expressions straightforward

    Referential transparency allows one to factorize all kind of common sub-expressions to remove redundancy Partial application and composition of functions makes it easy to write function pipelines and build new functions out of old ones transactionsFromCashflows :: AccountId -> (CashFlow -> [Transaction]) -> [CashFlow] -> [Transaction] transactionsFromCashflows pivot generator = concatMap $ map (normalize . balance pivot) . generator
  18. On the virtue of being lazy On the virtue of

    being lazy
  19. Haskell’s evaluation semantics is lazy, non strict or call-by-need: Arguments

    to functions are evaluated only when needed This makes it practical to express infinite computations in the language This applies also to I/O operations which may not be evaluated hence have no side-effects if their result is not needed
  20. On some of the less desirable aspects of the On

    some of the less desirable aspects of the language language
  21. No support for first-class module which makes large scale development

    more cumbersome A natural tendency to abuse operators and be too terse and dense too the point of becoming cryptic Beginners should expect to be confused at first by how the system supports strings and numbers Lazy I/O can have annoying side-effects There are too many ways to handler exceptions
  22. Philosophy Philosophy

  23. On the importance of Types for Haskellers On the importance

    of Types for Haskellers
  24. Haskellers seems to be extreme platonicists: Everything must have a

    well-defined type! Although types can be inferred by the compiler, they are an extremely important tool to design programs Haskell’s type system can be daunting at first but is conceptually elegant and compact
  25. Where one discovers types do not impose Where one discovers

    types do not impose extra burden at runtime extra burden at runtime newtypes provide cheap encapsulation of other types as they are unpacked by compiler: One should never refrain from creating such new types, they provide more safety than type aliases newtype AccountNr = AccountNr { accountNr :: T.Text } deriving (Eq,Ord,Show,Read) instance IsString AccountNr where fromString = AccountNr . T.pack
  26. Where it is shown types can even conjure Where it

    is shown types can even conjure the undead the undead Phantom types provide type-level annotation to add more information to other types Allow distinguishing between different types with identical representations Provide thread-safety by ensuring some type variables do not data Base64 data Hex newtype Encoded code = Encoded { encodedText :: Text } toBase64Text :: ByteString -> Encoded Base64 toHex :: ByteString -> Encoded Hex escape local scope
  27. Where one discovers subtle differences Where one discovers subtle differences

    cause more trouble than obvious ones cause more trouble than obvious ones can be thought as way to define interfaces, asbtracting away implementation details Type classes can define not only functions, but values and types Type classes actually define constraints on types that can be accumulated in expressions and checked by the compiler Type classes class (Eq a) => Differ a where uniqueId :: a -> Id a constructor :: a -> Entity b equals :: a -> a -> Bool
  28. Where one starts to wonder if there is an Where

    one starts to wonder if there is an end to recursion end to recursion Type families provide type-level functions, e.g. ways to compute types from types at compilation time Type families can be nested within data and class definitions to provide contextual types They can also be used at toplevel: class (ToJSON (Command a)) => BusinessModel a where data Event a :: * data Command a :: * type family Id a :: * type instance Id Account = AccountId type instance Id Transaction = TransactionId
  29. Where it is shown how more constraints Where it is

    shown how more constraints makes one more Free makes one more Free data Gratis f a = Effectless a | forall x. Effectful (f x) (x -> Gratis f a) data EmailServiceF a where DoMail :: Emailer -> Email -> EmailServiceF EmailStatus GetAllEmails :: Confirmation -> EmailServiceF [EmailWithStatus] type MailService = Gratis EmailServiceF liftFF :: EmailServiceF a -> MailService a liftFF c = Effectful c pure doMail :: Emailer -> Email -> MailService EmailStatus doMail mailer mail = liftFF $ DoMail mailer mail doGetAllEmails :: Confirmation -> MailService [ EmailWithStatus ] doGetAllEmails = liftFF . GetAllEmails interpret :: MailService a -> ExceptT L.Text (WebStateM s l m) a interpret (Effectful (DoMail mailer mail) f) = lift (liftIO $ mailer mail >>= handleSendingResult) >>= interpret . f interpret (Effectful (GetAllEmails confirmation) f) = lift (runWithEmails $ doGetEmails confirmation) >>= interpret . f interpret (Effectless a) = return a
  30. Mores & Daily Life Mores & Daily Life

  31. Where the amazed foreigner discovers how Where the amazed foreigner

    discovers how much can be done with crude tools much can be done with crude tools
  32. None
  33. There is no one-stop Integrated Development Environment for Haskell: Best

    support is provided by integrating various extensions into E Em ma ac cs s There is a nice on how to setup emacs and Chris Done has provided some Makes it easy to setup remote pairing is a recent initiative to provide an interactive and easy to use programming environment in Haskell provides a viable alternative with “everything” preconfigured and packaged tutorial standard configuration Haskell for Mac Spacemacs
  34. On the usefulness and efficiency of On the usefulness and

    efficiency of interacting with the machine interacting with the machine
  35. Use the REPL, Luke! Provides much faster feedback loop than

    full-blown compilation and building, can be used for Type/Test-DD too GHC has a cool feature to handle h ho ol le es s: variables which are not in scope but typechecked so that one can use that to GHCi comes with a debugger but I have never used it: provides a Haskell kernel for IPython notebooks for interactive programming. Provides a nice alternative to text-only development in Haskell esp. for number crunching deduce needed type and implementation Debugging sucks, testing rocks IHaskell
  36. None
  37. On the typical day of a commoner in On the

    typical day of a commoner in Haskell land Haskell land Write a skeletal test file using , e.g. test/FooTest.hs 1. Start REPL in Emacs by loading file C-c C-l 2. See it fail to compile 3. :reload until it compiles 4. Run the test and see it fail: hspec myTest 5. Fill in code until test passes 6. Do a full compile and test run before pushing to CI 7. hspec
  38. Government Government

  39. Where we discover Haskellers also value Where we discover Haskellers

    also value experimentation experimentation
  40. There are quite a few testing frameworks available to write

    unit or integration tests I favour which is one of the many rspec-inspired tools provides bindings to in Haskell Running tests is automated as part of build tools, e.g. cabal hspec hs-webdriver Selenium
  41. Where we see empiricism going hand in Where we see

    empiricism going hand in hand with formalism hand with formalism is the property-based testing tool for Haskell Use it for defining formal properties of your code beyond what type system provides QuickCheck instance Arbitrary ScaleRatio where arbitrary = ... instance Arbitrary Transaction where arbitrary = ... prop_scaled_transaction_is_normalized :: Transaction -> ScaleRatio -> Bool prop_scaled_transaction_is_normalized tx (ScaleRatio ratio) = isNormalized tx' && isBalanced tx' where tx' = scale ratio tx
  42. Where one discovers QuickCheck can be Where one discovers QuickCheck

    can be useful beside testing properties useful beside testing properties Generating sample data to test Generating sample data to test migration migration Generate sample data sets to be used as part of other kind of tests: We used it to generate test data for schema migration code sample_v8_Events :: [ByteString] sample_v8_Events = ["{\"tag\":\"AddedDocuments\",\"contents\":[\"c1f09023b3a9398a1d8a257c372392ab\",[ , "{\"tag\":\"UpdatedInvestor\",\"contents\":{\"invDocuments\":[], \"invBankAccount\":{\"bankBranch\":\"Ge\",\"bankAccountName\":\"1W\",\"bankAcco \"invId\":\"46ed3336bc60a5a423962e9b6343c003\",\"invReferralCode\":{\"refer\":\ \"invLegalEntity\":{\"tag\":\"Corporate\",\"primaryContactPosition\":{\"tag\":\ , "{\"tag\":\"NoEvent\",\"contents\":[]}" , "{\"tag\":\"AddedDocuments\",\"contents\":[\"b9973afdd569f11269fe0be736086049\"
  43. Generating execution sequences Generating execution sequences representing typical scenarios representing

    typical scenarios genAdminActions = frequency [ (1, return []) , (1, do f <- NewListing <$> arbitrary <*> newFacility <*> rate (f:) <$> genAdminActions) , (10, (AcceptSomeFacility:) <$> genAdminActions) , (5, (LookupFacilities:) <$> genAdminActions) , (6, (AdvanceDay:) . (ConfirmRepayments:) <$> genAdminActions) ] provides needed tooling to embed data generation within effectful code Monadic QuickCheck
  44. On the use of different level of testing On the

    use of different level of testing
  45. Architecture Architecture

  46. On the importance of laying out strong On the importance

    of laying out strong foundations foundations
  47. GHC has become the standard compiler for Haskell, current version

    is 7.10.3 It includes a lot of optimisations and features from 20 years of research on programming languages and compilation Always be sure to turn on -Wall -Werror when compiling: Warnings are often signs of potential troubles C Ca av ve ea at t: Compilation can take a long time but still does not yet beat scalac’s slowness
  48. On the intricacies of building complex On the intricacies of

    building complex software with a Cabal of scholars software with a Cabal of scholars
  49. Standard package management tool is which has become infamous due

    to Cabal is not very good at managing complex package structures out-of-the-box greatly improves thing in order to provide fully reproducible builds and sandboxed environments, down to compiler versions provides a solution to package management in Haskell Cabal cabal hell stack nix
  50. Diplomacy Diplomacy

  51. Where we separate that which is pure Where we separate

    that which is pure from that which is impure from that which is impure
  52. Type system provides a clean way to distinguish pure values

    from impure effects Staying pure has predictable semantics and opens the door to optimisations Effectful computations are one way portal to the real world
  53. Where we learn we can have our cake and Where

    we learn we can have our cake and eat it too eat it too
  54. Concurrency in Haskell is mostly based on composable operations STM

    do not mix with I/O operations hence cannot have observable side-effects but the upside is that runtime can detect deadlocks a high-level library to package asynchronous computations within cheap Haskell threading model We used actor-like queues to protect access to I/O resources, e.g. database. Haskell provide a lot of useful abstractions on top of STM and core concurrency features to build your own tools Software Transactional Memory async
  55. How Haskellers do communicate with the How Haskellers do communicate

    with the REST of the world REST of the world
  56. provides all the needed components to build high performance REST-based

    services in Haskell Initial development was done with which is a lightweight framework inspired by Sinatra Definitely need to have a look at which is a kind of “successor” to Scotty WAI Scotty Spock
  57. Where types make mortal beings doubt Where types make mortal

    beings doubt about their sanity about their sanity Newer services are now developed with which provides a way to express APIs a at t t th he e t ty yp pe e l le ev ve el l Makes it possible to derive server, client or swagger specs from a single declaration Servant type CreateJob = ReqBody '[JSON] Job :> Post '[JSON] JobId type ListJobs = Get '[JSON] [Job] type RemoveJob = Capture "jobid" JobId :> Delete '[JSON] JobId type SchedulerApi = "api" :> "scheduler" :> "jobs" :> CreateJob :<|> "api" :> "scheduler" :> "jobs" :> ListJobs :<|> "api" :> "scheduler" :> "jobs" :> RemoveJob
  58. On some ways data is persisted to stable On some

    ways data is persisted to stable storage storage
  59. Haskell has bindings to most open-source or standard (R)DBMS but

    we did not used them Seems like perfect environment to use We developed in-house solution to store events generated by the application as a sequential log of events stored in a file on disk event sourcing
  60. None
  61. War War

  62. Where peaceful Haskellers prepare Where peaceful Haskellers prepare themselves to

    wage war themselves to wage war
  63. is a Haskell tool to replace make and orchestrate complex

    builds is our Continuous Integration tool We used both to build and deploy all components of the system as containers Both provides way to express complex build processes as type-safe declarative rules Shake bake docker
  64. On the war-time organisation of On the war-time organisation of

    Haskell’s legions Haskell’s legions
  65. is a configuration management tool in Haskell developed by of

    Debian fame Configuration is expressed as a Haskell program that is compiled and run on the target host It provides a nice gpg key based model to encrypt private data (e.g. ssh keys, authentication tokens, passwords…) that need to be deployed in the source directory Once again, it allows expressing configuration items in a type-safe way propellor Joey Hess
  66. On the capabilities to wage war outside of On the

    capabilities to wage war outside of Haskelland Haskelland
  67. is Haskell client for interacting with ’s API where we

    deploy all our containers There is thorough Haskell support for AWS API through and Google’s APIs through Monitoring is done through a hdo Digital Ocean Amazonka gogol riemann Haskell client
  68. Conclusion Conclusion

  69. What benefits one can expect from visiting What benefits one

    can expect from visiting Haskelland Haskelland
  70. Strong type discipline and powerful compiler removes a whole class

    of programming errors Types are also great to design complex software systems and yield great opportunities for verification and optimizations All the best practices you learnt are still valid in Haskelland, only a bit differently practiced You can stay away from “hairy” stuff while still being productive and have fun
  71. What the cautious traveller should be What the cautious traveller

    should be aware of when ashoring Haskelland aware of when ashoring Haskelland
  72. Development environment can be tricky to get - and stay

    - “right” A lot of the material one can find is research focused which can be daunting Avoid partial functions at all cost It can be hard to find Haskell developers locally
  73. Acknowledgements Acknowledgements

  74. Credits Credits All the woodcuts illustrating this talk are drawn

    from the with the exception of: Encyclopédie de Diderot et d’Alembert , The , , . Durer’s “Artist drawing a couching woman” Austerlitz Pyramid Durer’s Rhinoceros Allégorie à la Paix d’Utrecht
  75. Thanks Thanks To the Devoxx selection committee for having accepted

    that talk, To all the fine people at Capital Match who made all this possible, To Haskellers all over the world who fuel the fire with their wits and spirit, To Haskell B. Curry who, among others, laid out the foundations for our daily job.