Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Real Real World of Haskell

The Real Real World of Haskell

Slides from my talk at DevoxxFR 2016

Arnaud Bailly

April 22, 2016
Tweet

More Decks by Arnaud Bailly

Other Decks in Technology

Transcript

  1. The Real Real World
    The Real Real World
    of Haskell
    of Haskell
    Relations on the Land of Pure Functional
    Relations on the Land of Pure Functional
    Programming by a Mortal Being
    Programming by a Mortal Being
    Arnaud Bailly
    Arnaud Bailly
    April 22, 2016
    April 22, 2016

    View Slide

  2. Table of Contents
    Table of Contents
    How I Came to Reside in the Land of Haskell and Why It
    Matters
    On Various Aspects of Haskelland Civilisation and The Mores
    of Its Inhabitants
    Some Ways this Wonderful Journey Could Inspire the Reader

    View Slide

  3. How I Came to Visit
    How I Came to Visit
    Haskelland
    Haskelland

    View Slide

  4. View Slide

  5. Where the author dares to present himself
    Where the author dares to present himself

    View Slide

  6. Where the author dares to present himself
    Where the author dares to present himself
    20+ years of experience developing software…
    Has been coding in Haskell for side-projects since 2001
    Had the opportunity to join Capital Match as Co-founder and
    CTO in 2014
    Haskell was part of the plan since the beginning and the
    reason why I got involved
    I like theory but I prefere working software

    View Slide

  7. On the Company which I travelled to
    On the Company which I travelled to
    Haskelland for
    Haskelland for

    View Slide

  8. On the Company which I travelled to
    On the Company which I travelled to
    Haskelland for
    Haskelland for
    Capital Match is the leading plaform in Singapore for
    peer-to-peer lending to SMEs
    Backend system developed in Haskell, frontend in
    Clojurescript/Om since 2014
    Core Development team of 4 persons

    View Slide

  9. Where it is demonstrated why the interest
    Where it is demonstrated why the interest
    of the honorable reader might be aroused
    of the honorable reader might be aroused
    This is not a Haskell tutorial
    This is not a Monad tutorial
    Let’s stop thinking “Haskell is good for your brain but not for
    practical use”
    I will be happy if you end up thinking: I could try this cool stuff in
    my next microservice
    I will be even happier if you can put to a
    ac
    ct
    tu
    ua
    al
    l u
    us
    se
    e some of the
    stuff I present here

    View Slide

  10. On Various Aspects of
    On Various Aspects of
    Haskelland
    Haskelland

    View Slide

  11. View Slide

  12. Where the author tries to cover the most important aspects of
    Haskeller’s civilisation in an extremely short span of time, thus
    probably missing lot of important points and generating
    frustration in the informed reader, but nevertheless hoping to
    convey enough tips for the audience to be willing to adopt some
    of the customs of those strange people.

    View Slide

  13. Language
    Language

    View Slide

  14. On some minor differences between
    On some minor differences between
    Haskell Language and more Common ones
    Haskell Language and more Common ones

    View Slide

  15. Haskell’s syntax favours terseness and strives to be close to
    mathematical language, compare:
    with:
    {(x, 2x)|x 㱨 [1, 100]㱸x mod 3 = 0}
    Haskellers don’t use lot of punctuation signs but favor using
    indentation to express nesting
    [ (x, x * 2) | x <- [ 0 .. 100 ], x `mod` 3 == 0 ]

    View Slide

  16. On the importance of composition in
    On the importance of composition in
    Haskell
    Haskell

    View Slide

  17. Haskell language is geared towards making composition of
    expressions straightforward
    Referential transparency allows one to factorize all kind of
    common sub-expressions to remove redundancy
    Partial application and composition of functions makes it easy to
    write function pipelines and build new functions out of old ones
    transactionsFromCashflows :: AccountId
    -> (CashFlow -> [Transaction])
    -> [CashFlow] -> [Transaction]
    transactionsFromCashflows pivot generator =
    concatMap $ map (normalize . balance pivot) . generator

    View Slide

  18. On the virtue of being lazy
    On the virtue of being lazy

    View Slide

  19. Haskell’s evaluation semantics is lazy, non strict or call-by-need:
    Arguments to functions are evaluated only when needed
    This makes it practical to express infinite computations in the
    language
    This applies also to I/O operations which may not be
    evaluated hence have no side-effects if their result is not
    needed

    View Slide

  20. On some of the less desirable aspects of the
    On some of the less desirable aspects of the
    language
    language

    View Slide

  21. No support for first-class module which makes large scale
    development more cumbersome
    A natural tendency to abuse operators and be too terse and dense
    too the point of becoming cryptic
    Beginners should expect to be confused at first by how the
    system supports strings and numbers
    Lazy I/O can have annoying side-effects
    There are too many ways to handler exceptions

    View Slide

  22. Philosophy
    Philosophy

    View Slide

  23. On the importance of Types for Haskellers
    On the importance of Types for Haskellers

    View Slide

  24. Haskellers seems to be extreme platonicists: Everything must
    have a well-defined type!
    Although types can be inferred by the compiler, they are an
    extremely important tool to design programs
    Haskell’s type system can be daunting at first but is
    conceptually elegant and compact

    View Slide

  25. Where one discovers types do not impose
    Where one discovers types do not impose
    extra burden at runtime
    extra burden at runtime
    newtypes provide cheap encapsulation of other types as they
    are unpacked by compiler:
    One should never refrain from creating such new types, they
    provide more safety than type aliases
    newtype AccountNr = AccountNr { accountNr :: T.Text }
    deriving (Eq,Ord,Show,Read)
    instance IsString AccountNr where
    fromString = AccountNr . T.pack

    View Slide

  26. Where it is shown types can even conjure
    Where it is shown types can even conjure
    the undead
    the undead
    Phantom types provide type-level annotation to add more
    information to other types
    Allow distinguishing between different types with identical
    representations
    Provide thread-safety by ensuring some type variables do not
    data Base64
    data Hex
    newtype Encoded code = Encoded { encodedText :: Text }
    toBase64Text :: ByteString -> Encoded Base64
    toHex :: ByteString -> Encoded Hex
    escape local scope

    View Slide

  27. Where one discovers subtle differences
    Where one discovers subtle differences
    cause more trouble than obvious ones
    cause more trouble than obvious ones
    can be thought as way to define interfaces,
    asbtracting away implementation details
    Type classes can define not only functions, but values and
    types
    Type classes actually define constraints on types that can be
    accumulated in expressions and checked by the compiler
    Type classes
    class (Eq a) => Differ a where
    uniqueId :: a -> Id a
    constructor :: a -> Entity b
    equals :: a -> a -> Bool

    View Slide

  28. Where one starts to wonder if there is an
    Where one starts to wonder if there is an
    end to recursion
    end to recursion
    Type families provide type-level functions, e.g. ways to compute
    types from types at compilation time
    Type families can be nested within data and class
    definitions to provide contextual types
    They can also be used at toplevel:
    class (ToJSON (Command a)) => BusinessModel a where
    data Event a :: *
    data Command a :: *
    type family Id a :: *
    type instance Id Account = AccountId
    type instance Id Transaction = TransactionId

    View Slide

  29. Where it is shown how more constraints
    Where it is shown how more constraints
    makes one more Free
    makes one more Free
    data Gratis f a = Effectless a
    | forall x. Effectful (f x) (x -> Gratis f a)
    data EmailServiceF a where
    DoMail :: Emailer -> Email -> EmailServiceF EmailStatus
    GetAllEmails :: Confirmation -> EmailServiceF [EmailWithStatus]
    type MailService = Gratis EmailServiceF
    liftFF :: EmailServiceF a -> MailService a
    liftFF c = Effectful c pure
    doMail :: Emailer -> Email -> MailService EmailStatus
    doMail mailer mail = liftFF $ DoMail mailer mail
    doGetAllEmails :: Confirmation -> MailService [ EmailWithStatus ]
    doGetAllEmails = liftFF . GetAllEmails
    interpret :: MailService a -> ExceptT L.Text (WebStateM s l m) a
    interpret (Effectful (DoMail mailer mail) f) =
    lift (liftIO $ mailer mail >>= handleSendingResult) >>= interpret . f
    interpret (Effectful (GetAllEmails confirmation) f) =
    lift (runWithEmails $ doGetEmails confirmation) >>= interpret . f
    interpret (Effectless a) = return a

    View Slide

  30. Mores & Daily Life
    Mores & Daily Life

    View Slide

  31. Where the amazed foreigner discovers how
    Where the amazed foreigner discovers how
    much can be done with crude tools
    much can be done with crude tools

    View Slide

  32. View Slide

  33. There is no one-stop Integrated Development Environment for
    Haskell: Best support is provided by integrating various
    extensions into E
    Em
    ma
    ac
    cs
    s
    There is a nice on how to setup emacs and Chris Done
    has provided some
    Makes it easy to setup remote pairing
    is a recent initiative to provide an interactive
    and easy to use programming environment in Haskell
    provides a viable alternative with “everything”
    preconfigured and packaged
    tutorial
    standard configuration
    Haskell for Mac
    Spacemacs

    View Slide

  34. On the usefulness and efficiency of
    On the usefulness and efficiency of
    interacting with the machine
    interacting with the machine

    View Slide

  35. Use the REPL, Luke! Provides much faster feedback loop
    than full-blown compilation and building, can be used for
    Type/Test-DD too
    GHC has a cool feature to handle h
    ho
    ol
    le
    es
    s: variables which are
    not in scope but typechecked so that one can use that to
    GHCi comes with a debugger but I have never used it:
    provides a Haskell kernel for IPython notebooks for
    interactive programming. Provides a nice alternative to
    text-only development in Haskell esp. for number crunching
    deduce needed type and implementation
    Debugging sucks, testing rocks
    IHaskell

    View Slide

  36. View Slide

  37. On the typical day of a commoner in
    On the typical day of a commoner in
    Haskell land
    Haskell land
    Write a skeletal test file using , e.g. test/FooTest.hs
    1.
    Start REPL in Emacs by loading file C-c C-l
    2.
    See it fail to compile
    3.
    :reload until it compiles
    4.
    Run the test and see it fail: hspec myTest
    5.
    Fill in code until test passes
    6.
    Do a full compile and test run before pushing to CI
    7.
    hspec

    View Slide

  38. Government
    Government

    View Slide

  39. Where we discover Haskellers also value
    Where we discover Haskellers also value
    experimentation
    experimentation

    View Slide

  40. There are quite a few testing frameworks available to write unit
    or integration tests
    I favour which is one of the many rspec-inspired tools
    provides bindings to in Haskell
    Running tests is automated as part of build tools, e.g. cabal
    hspec
    hs-webdriver Selenium

    View Slide

  41. Where we see empiricism going hand in
    Where we see empiricism going hand in
    hand with formalism
    hand with formalism
    is the property-based testing tool for Haskell
    Use it for defining formal properties of your code beyond
    what type system provides
    QuickCheck
    instance Arbitrary ScaleRatio where arbitrary = ...
    instance Arbitrary Transaction where arbitrary = ...
    prop_scaled_transaction_is_normalized :: Transaction -> ScaleRatio
    -> Bool
    prop_scaled_transaction_is_normalized tx (ScaleRatio ratio) =
    isNormalized tx' && isBalanced tx'
    where
    tx' = scale ratio tx

    View Slide

  42. Where one discovers QuickCheck can be
    Where one discovers QuickCheck can be
    useful beside testing properties
    useful beside testing properties
    Generating sample data to test
    Generating sample data to test
    migration
    migration
    Generate sample data sets to be used as part of other kind of
    tests: We used it to generate test data for schema migration code
    sample_v8_Events :: [ByteString]
    sample_v8_Events =
    ["{\"tag\":\"AddedDocuments\",\"contents\":[\"c1f09023b3a9398a1d8a257c372392ab\",[
    , "{\"tag\":\"UpdatedInvestor\",\"contents\":{\"invDocuments\":[],
    \"invBankAccount\":{\"bankBranch\":\"Ge\",\"bankAccountName\":\"1W\",\"bankAcco
    \"invId\":\"46ed3336bc60a5a423962e9b6343c003\",\"invReferralCode\":{\"refer\":\
    \"invLegalEntity\":{\"tag\":\"Corporate\",\"primaryContactPosition\":{\"tag\":\
    , "{\"tag\":\"NoEvent\",\"contents\":[]}"
    , "{\"tag\":\"AddedDocuments\",\"contents\":[\"b9973afdd569f11269fe0be736086049\"

    View Slide

  43. Generating execution sequences
    Generating execution sequences
    representing typical scenarios
    representing typical scenarios
    genAdminActions =
    frequency [ (1, return [])
    , (1, do
    f <- NewListing <$> arbitrary <*> newFacility <*> rate
    (f:) <$> genAdminActions)
    , (10, (AcceptSomeFacility:) <$> genAdminActions)
    , (5, (LookupFacilities:) <$> genAdminActions)
    , (6, (AdvanceDay:) . (ConfirmRepayments:) <$> genAdminActions)
    ]
    provides needed tooling to embed data
    generation within effectful code
    Monadic QuickCheck

    View Slide

  44. On the use of different level of testing
    On the use of different level of testing

    View Slide

  45. Architecture
    Architecture

    View Slide

  46. On the importance of laying out strong
    On the importance of laying out strong
    foundations
    foundations

    View Slide

  47. GHC has become the standard compiler for Haskell, current
    version is 7.10.3
    It includes a lot of optimisations and features from 20 years of
    research on programming languages and compilation
    Always be sure to turn on -Wall -Werror when compiling:
    Warnings are often signs of potential troubles
    C
    Ca
    av
    ve
    ea
    at
    t: Compilation can take a long time but still does not yet
    beat scalac’s slowness

    View Slide

  48. On the intricacies of building complex
    On the intricacies of building complex
    software with a Cabal of scholars
    software with a Cabal of scholars

    View Slide

  49. Standard package management tool is which has
    become infamous due to
    Cabal is not very good at managing complex package
    structures out-of-the-box
    greatly improves thing in order to provide fully
    reproducible builds and sandboxed environments, down to
    compiler versions
    provides a solution to package management in Haskell
    Cabal
    cabal hell
    stack
    nix

    View Slide

  50. Diplomacy
    Diplomacy

    View Slide

  51. Where we separate that which is pure
    Where we separate that which is pure
    from that which is impure
    from that which is impure

    View Slide

  52. Type system provides a clean way to distinguish pure values
    from impure effects
    Staying pure has predictable semantics and opens the door to
    optimisations
    Effectful computations are one way portal to the real world

    View Slide

  53. Where we learn we can have our cake and
    Where we learn we can have our cake and
    eat it too
    eat it too

    View Slide

  54. Concurrency in Haskell is mostly based on composable
    operations
    STM do not mix with I/O operations hence cannot have
    observable side-effects but the upside is that runtime can detect
    deadlocks
    a high-level library to package asynchronous
    computations within cheap Haskell threading model
    We used actor-like queues to protect access to I/O resources,
    e.g. database. Haskell provide a lot of useful abstractions on
    top of STM and core concurrency features to build your own
    tools
    Software Transactional Memory
    async

    View Slide

  55. How Haskellers do communicate with the
    How Haskellers do communicate with the
    REST of the world
    REST of the world

    View Slide

  56. provides all the needed components to build high
    performance REST-based services in Haskell
    Initial development was done with which is a
    lightweight framework inspired by Sinatra
    Definitely need to have a look at which is a kind of
    “successor” to Scotty
    WAI
    Scotty
    Spock

    View Slide

  57. Where types make mortal beings doubt
    Where types make mortal beings doubt
    about their sanity
    about their sanity
    Newer services are now developed with which provides
    a way to express APIs a
    at
    t t
    th
    he
    e t
    ty
    yp
    pe
    e l
    le
    ev
    ve
    el
    l
    Makes it possible to derive server, client or swagger specs from
    a single declaration
    Servant
    type CreateJob = ReqBody '[JSON] Job :> Post '[JSON] JobId
    type ListJobs = Get '[JSON] [Job]
    type RemoveJob = Capture "jobid" JobId :> Delete '[JSON] JobId
    type SchedulerApi = "api" :> "scheduler" :> "jobs" :> CreateJob
    :<|> "api" :> "scheduler" :> "jobs" :> ListJobs
    :<|> "api" :> "scheduler" :> "jobs" :> RemoveJob

    View Slide

  58. On some ways data is persisted to stable
    On some ways data is persisted to stable
    storage
    storage

    View Slide

  59. Haskell has bindings to most open-source or standard
    (R)DBMS but we did not used them
    Seems like perfect environment to use
    We developed in-house solution to store events generated by
    the application as a sequential log of events stored in a file on disk
    event sourcing

    View Slide

  60. View Slide

  61. War
    War

    View Slide

  62. Where peaceful Haskellers prepare
    Where peaceful Haskellers prepare
    themselves to wage war
    themselves to wage war

    View Slide

  63. is a Haskell tool to replace make and orchestrate
    complex builds
    is our Continuous Integration tool
    We used both to build and deploy all components of the
    system as containers
    Both provides way to express complex build processes as
    type-safe declarative rules
    Shake
    bake
    docker

    View Slide

  64. On the war-time organisation of
    On the war-time organisation of
    Haskell’s legions
    Haskell’s legions

    View Slide

  65. is a configuration management tool in Haskell
    developed by of Debian fame
    Configuration is expressed as a Haskell program that is
    compiled and run on the target host
    It provides a nice gpg key based model to encrypt private data
    (e.g. ssh keys, authentication tokens, passwords…) that need to
    be deployed in the source directory
    Once again, it allows expressing configuration items in a
    type-safe way
    propellor
    Joey Hess

    View Slide

  66. On the capabilities to wage war outside of
    On the capabilities to wage war outside of
    Haskelland
    Haskelland

    View Slide

  67. is Haskell client for interacting with ’s API
    where we deploy all our containers
    There is thorough Haskell support for AWS API through
    and Google’s APIs through
    Monitoring is done through a
    hdo Digital Ocean
    Amazonka gogol
    riemann Haskell client

    View Slide

  68. Conclusion
    Conclusion

    View Slide

  69. What benefits one can expect from visiting
    What benefits one can expect from visiting
    Haskelland
    Haskelland

    View Slide

  70. Strong type discipline and powerful compiler removes a whole
    class of programming errors
    Types are also great to design complex software systems and
    yield great opportunities for verification and optimizations
    All the best practices you learnt are still valid in Haskelland, only
    a bit differently practiced
    You can stay away from “hairy” stuff while still being
    productive and have fun

    View Slide

  71. What the cautious traveller should be
    What the cautious traveller should be
    aware of when ashoring Haskelland
    aware of when ashoring Haskelland

    View Slide

  72. Development environment can be tricky to get - and stay -
    “right”
    A lot of the material one can find is research focused which can be
    daunting
    Avoid partial functions at all cost
    It can be hard to find Haskell developers locally

    View Slide

  73. Acknowledgements
    Acknowledgements

    View Slide

  74. Credits
    Credits
    All the woodcuts illustrating this talk are drawn from the
    with the exception of:
    Encyclopédie de Diderot et d’Alembert
    ,
    The ,
    ,
    .
    Durer’s “Artist drawing a couching woman”
    Austerlitz Pyramid
    Durer’s Rhinoceros
    Allégorie à la Paix d’Utrecht

    View Slide

  75. Thanks
    Thanks
    To the Devoxx selection committee for having accepted that
    talk,
    To all the fine people at Capital Match who made all this
    possible,
    To Haskellers all over the world who fuel the fire with their
    wits and spirit,
    To Haskell B. Curry who, among others, laid out the
    foundations for our daily job.

    View Slide