Share
Embed
Transcript
75 slides
-
The Real Real World The Real Real World of Haskell
of Haskell Relations on the Land of Pure Functional Relations on the Land of Pure Functional Programming by a Mortal Being Programming by a Mortal Being Arnaud Bailly Arnaud Bailly April 22, 2016 April 22, 2016 -
Table of Contents Table of Contents How I Came to
Reside in the Land of Haskell and Why It Matters On Various Aspects of Haskelland Civilisation and The Mores of Its Inhabitants Some Ways this Wonderful Journey Could Inspire the Reader -
How I Came to Visit How I Came to Visit
Haskelland Haskelland -
Where the author dares to present himself Where the author
dares to present himself -
Where the author dares to present himself Where the author
dares to present himself 20+ years of experience developing software… Has been coding in Haskell for side-projects since 2001 Had the opportunity to join Capital Match as Co-founder and CTO in 2014 Haskell was part of the plan since the beginning and the reason why I got involved I like theory but I prefere working software -
On the Company which I travelled to On the Company
which I travelled to Haskelland for Haskelland for -
On the Company which I travelled to On the Company
which I travelled to Haskelland for Haskelland for Capital Match is the leading plaform in Singapore for peer-to-peer lending to SMEs Backend system developed in Haskell, frontend in Clojurescript/Om since 2014 Core Development team of 4 persons -
Where it is demonstrated why the interest Where it is
demonstrated why the interest of the honorable reader might be aroused of the honorable reader might be aroused This is not a Haskell tutorial This is not a Monad tutorial Let’s stop thinking “Haskell is good for your brain but not for practical use” I will be happy if you end up thinking: I could try this cool stuff in my next microservice I will be even happier if you can put to a ac ct tu ua al l u us se e some of the stuff I present here -
Where the author tries to cover the most important aspects
of Haskeller’s civilisation in an extremely short span of time, thus probably missing lot of important points and generating frustration in the informed reader, but nevertheless hoping to convey enough tips for the audience to be willing to adopt some of the customs of those strange people. -
On some minor differences between On some minor differences between
Haskell Language and more Common ones Haskell Language and more Common ones -
Haskell’s syntax favours terseness and strives to be close to
mathematical language, compare: with: {(x, 2x)|x 㱨 [1, 100]㱸x mod 3 = 0} Haskellers don’t use lot of punctuation signs but favor using indentation to express nesting [ (x, x * 2) | x <- [ 0 .. 100 ], x `mod` 3 == 0 ] -
On the importance of composition in On the importance of
composition in Haskell Haskell -
Haskell language is geared towards making composition of expressions straightforward
Referential transparency allows one to factorize all kind of common sub-expressions to remove redundancy Partial application and composition of functions makes it easy to write function pipelines and build new functions out of old ones transactionsFromCashflows :: AccountId -> (CashFlow -> [Transaction]) -> [CashFlow] -> [Transaction] transactionsFromCashflows pivot generator = concatMap $ map (normalize . balance pivot) . generator -
On the virtue of being lazy On the virtue of
being lazy -
Haskell’s evaluation semantics is lazy, non strict or call-by-need: Arguments
to functions are evaluated only when needed This makes it practical to express infinite computations in the language This applies also to I/O operations which may not be evaluated hence have no side-effects if their result is not needed -
On some of the less desirable aspects of the On
some of the less desirable aspects of the language language -
No support for first-class module which makes large scale development
more cumbersome A natural tendency to abuse operators and be too terse and dense too the point of becoming cryptic Beginners should expect to be confused at first by how the system supports strings and numbers Lazy I/O can have annoying side-effects There are too many ways to handler exceptions -
On the importance of Types for Haskellers On the importance
of Types for Haskellers -
Haskellers seems to be extreme platonicists: Everything must have a
well-defined type! Although types can be inferred by the compiler, they are an extremely important tool to design programs Haskell’s type system can be daunting at first but is conceptually elegant and compact -
Where one discovers types do not impose Where one discovers
types do not impose extra burden at runtime extra burden at runtime newtypes provide cheap encapsulation of other types as they are unpacked by compiler: One should never refrain from creating such new types, they provide more safety than type aliases newtype AccountNr = AccountNr { accountNr :: T.Text } deriving (Eq,Ord,Show,Read) instance IsString AccountNr where fromString = AccountNr . T.pack -
Where it is shown types can even conjure Where it
is shown types can even conjure the undead the undead Phantom types provide type-level annotation to add more information to other types Allow distinguishing between different types with identical representations Provide thread-safety by ensuring some type variables do not data Base64 data Hex newtype Encoded code = Encoded { encodedText :: Text } toBase64Text :: ByteString -> Encoded Base64 toHex :: ByteString -> Encoded Hex escape local scope -
Where one discovers subtle differences Where one discovers subtle differences
cause more trouble than obvious ones cause more trouble than obvious ones can be thought as way to define interfaces, asbtracting away implementation details Type classes can define not only functions, but values and types Type classes actually define constraints on types that can be accumulated in expressions and checked by the compiler Type classes class (Eq a) => Differ a where uniqueId :: a -> Id a constructor :: a -> Entity b equals :: a -> a -> Bool -
Where one starts to wonder if there is an Where
one starts to wonder if there is an end to recursion end to recursion Type families provide type-level functions, e.g. ways to compute types from types at compilation time Type families can be nested within data and class definitions to provide contextual types They can also be used at toplevel: class (ToJSON (Command a)) => BusinessModel a where data Event a :: * data Command a :: * type family Id a :: * type instance Id Account = AccountId type instance Id Transaction = TransactionId -
Where it is shown how more constraints Where it is
shown how more constraints makes one more Free makes one more Free data Gratis f a = Effectless a | forall x. Effectful (f x) (x -> Gratis f a) data EmailServiceF a where DoMail :: Emailer -> Email -> EmailServiceF EmailStatus GetAllEmails :: Confirmation -> EmailServiceF [EmailWithStatus] type MailService = Gratis EmailServiceF liftFF :: EmailServiceF a -> MailService a liftFF c = Effectful c pure doMail :: Emailer -> Email -> MailService EmailStatus doMail mailer mail = liftFF $ DoMail mailer mail doGetAllEmails :: Confirmation -> MailService [ EmailWithStatus ] doGetAllEmails = liftFF . GetAllEmails interpret :: MailService a -> ExceptT L.Text (WebStateM s l m) a interpret (Effectful (DoMail mailer mail) f) = lift (liftIO $ mailer mail >>= handleSendingResult) >>= interpret . f interpret (Effectful (GetAllEmails confirmation) f) = lift (runWithEmails $ doGetEmails confirmation) >>= interpret . f interpret (Effectless a) = return a -
Where the amazed foreigner discovers how Where the amazed foreigner
discovers how much can be done with crude tools much can be done with crude tools -
There is no one-stop Integrated Development Environment for Haskell: Best
support is provided by integrating various extensions into E Em ma ac cs s There is a nice on how to setup emacs and Chris Done has provided some Makes it easy to setup remote pairing is a recent initiative to provide an interactive and easy to use programming environment in Haskell provides a viable alternative with “everything” preconfigured and packaged tutorial standard configuration Haskell for Mac Spacemacs -
On the usefulness and efficiency of On the usefulness and
efficiency of interacting with the machine interacting with the machine -
Use the REPL, Luke! Provides much faster feedback loop than
full-blown compilation and building, can be used for Type/Test-DD too GHC has a cool feature to handle h ho ol le es s: variables which are not in scope but typechecked so that one can use that to GHCi comes with a debugger but I have never used it: provides a Haskell kernel for IPython notebooks for interactive programming. Provides a nice alternative to text-only development in Haskell esp. for number crunching deduce needed type and implementation Debugging sucks, testing rocks IHaskell -
On the typical day of a commoner in On the
typical day of a commoner in Haskell land Haskell land Write a skeletal test file using , e.g. test/FooTest.hs 1. Start REPL in Emacs by loading file C-c C-l 2. See it fail to compile 3. :reload until it compiles 4. Run the test and see it fail: hspec myTest 5. Fill in code until test passes 6. Do a full compile and test run before pushing to CI 7. hspec -
Where we discover Haskellers also value Where we discover Haskellers
also value experimentation experimentation -
There are quite a few testing frameworks available to write
unit or integration tests I favour which is one of the many rspec-inspired tools provides bindings to in Haskell Running tests is automated as part of build tools, e.g. cabal hspec hs-webdriver Selenium -
Where we see empiricism going hand in Where we see
empiricism going hand in hand with formalism hand with formalism is the property-based testing tool for Haskell Use it for defining formal properties of your code beyond what type system provides QuickCheck instance Arbitrary ScaleRatio where arbitrary = ... instance Arbitrary Transaction where arbitrary = ... prop_scaled_transaction_is_normalized :: Transaction -> ScaleRatio -> Bool prop_scaled_transaction_is_normalized tx (ScaleRatio ratio) = isNormalized tx' && isBalanced tx' where tx' = scale ratio tx -
Where one discovers QuickCheck can be Where one discovers QuickCheck
can be useful beside testing properties useful beside testing properties Generating sample data to test Generating sample data to test migration migration Generate sample data sets to be used as part of other kind of tests: We used it to generate test data for schema migration code sample_v8_Events :: [ByteString] sample_v8_Events = ["{\"tag\":\"AddedDocuments\",\"contents\":[\"c1f09023b3a9398a1d8a257c372392ab\",[ , "{\"tag\":\"UpdatedInvestor\",\"contents\":{\"invDocuments\":[], \"invBankAccount\":{\"bankBranch\":\"Ge\",\"bankAccountName\":\"1W\",\"bankAcco \"invId\":\"46ed3336bc60a5a423962e9b6343c003\",\"invReferralCode\":{\"refer\":\ \"invLegalEntity\":{\"tag\":\"Corporate\",\"primaryContactPosition\":{\"tag\":\ , "{\"tag\":\"NoEvent\",\"contents\":[]}" , "{\"tag\":\"AddedDocuments\",\"contents\":[\"b9973afdd569f11269fe0be736086049\" -
Generating execution sequences Generating execution sequences representing typical scenarios representing
typical scenarios genAdminActions = frequency [ (1, return []) , (1, do f <- NewListing <$> arbitrary <*> newFacility <*> rate (f:) <$> genAdminActions) , (10, (AcceptSomeFacility:) <$> genAdminActions) , (5, (LookupFacilities:) <$> genAdminActions) , (6, (AdvanceDay:) . (ConfirmRepayments:) <$> genAdminActions) ] provides needed tooling to embed data generation within effectful code Monadic QuickCheck -
On the use of different level of testing On the
use of different level of testing -
On the importance of laying out strong On the importance
of laying out strong foundations foundations -
GHC has become the standard compiler for Haskell, current version
is 7.10.3 It includes a lot of optimisations and features from 20 years of research on programming languages and compilation Always be sure to turn on -Wall -Werror when compiling: Warnings are often signs of potential troubles C Ca av ve ea at t: Compilation can take a long time but still does not yet beat scalac’s slowness -
On the intricacies of building complex On the intricacies of
building complex software with a Cabal of scholars software with a Cabal of scholars -
Standard package management tool is which has become infamous due
to Cabal is not very good at managing complex package structures out-of-the-box greatly improves thing in order to provide fully reproducible builds and sandboxed environments, down to compiler versions provides a solution to package management in Haskell Cabal cabal hell stack nix -
Where we separate that which is pure Where we separate
that which is pure from that which is impure from that which is impure -
Type system provides a clean way to distinguish pure values
from impure effects Staying pure has predictable semantics and opens the door to optimisations Effectful computations are one way portal to the real world -
Where we learn we can have our cake and Where
we learn we can have our cake and eat it too eat it too -
Concurrency in Haskell is mostly based on composable operations STM
do not mix with I/O operations hence cannot have observable side-effects but the upside is that runtime can detect deadlocks a high-level library to package asynchronous computations within cheap Haskell threading model We used actor-like queues to protect access to I/O resources, e.g. database. Haskell provide a lot of useful abstractions on top of STM and core concurrency features to build your own tools Software Transactional Memory async -
How Haskellers do communicate with the How Haskellers do communicate
with the REST of the world REST of the world -
provides all the needed components to build high performance REST-based
services in Haskell Initial development was done with which is a lightweight framework inspired by Sinatra Definitely need to have a look at which is a kind of “successor” to Scotty WAI Scotty Spock -
Where types make mortal beings doubt Where types make mortal
beings doubt about their sanity about their sanity Newer services are now developed with which provides a way to express APIs a at t t th he e t ty yp pe e l le ev ve el l Makes it possible to derive server, client or swagger specs from a single declaration Servant type CreateJob = ReqBody '[JSON] Job :> Post '[JSON] JobId type ListJobs = Get '[JSON] [Job] type RemoveJob = Capture "jobid" JobId :> Delete '[JSON] JobId type SchedulerApi = "api" :> "scheduler" :> "jobs" :> CreateJob :<|> "api" :> "scheduler" :> "jobs" :> ListJobs :<|> "api" :> "scheduler" :> "jobs" :> RemoveJob -
On some ways data is persisted to stable On some
ways data is persisted to stable storage storage -
Haskell has bindings to most open-source or standard (R)DBMS but
we did not used them Seems like perfect environment to use We developed in-house solution to store events generated by the application as a sequential log of events stored in a file on disk event sourcing -
Where peaceful Haskellers prepare Where peaceful Haskellers prepare themselves to
wage war themselves to wage war -
is a Haskell tool to replace make and orchestrate complex
builds is our Continuous Integration tool We used both to build and deploy all components of the system as containers Both provides way to express complex build processes as type-safe declarative rules Shake bake docker -
On the war-time organisation of On the war-time organisation of
Haskell’s legions Haskell’s legions -
is a configuration management tool in Haskell developed by of
Debian fame Configuration is expressed as a Haskell program that is compiled and run on the target host It provides a nice gpg key based model to encrypt private data (e.g. ssh keys, authentication tokens, passwords…) that need to be deployed in the source directory Once again, it allows expressing configuration items in a type-safe way propellor Joey Hess -
On the capabilities to wage war outside of On the
capabilities to wage war outside of Haskelland Haskelland -
is Haskell client for interacting with ’s API where we
deploy all our containers There is thorough Haskell support for AWS API through and Google’s APIs through Monitoring is done through a hdo Digital Ocean Amazonka gogol riemann Haskell client -
What benefits one can expect from visiting What benefits one
can expect from visiting Haskelland Haskelland -
Strong type discipline and powerful compiler removes a whole class
of programming errors Types are also great to design complex software systems and yield great opportunities for verification and optimizations All the best practices you learnt are still valid in Haskelland, only a bit differently practiced You can stay away from “hairy” stuff while still being productive and have fun -
What the cautious traveller should be What the cautious traveller
should be aware of when ashoring Haskelland aware of when ashoring Haskelland -
Development environment can be tricky to get - and stay
- “right” A lot of the material one can find is research focused which can be daunting Avoid partial functions at all cost It can be hard to find Haskell developers locally -
Credits Credits All the woodcuts illustrating this talk are drawn
from the with the exception of: Encyclopédie de Diderot et d’Alembert , The , , . Durer’s “Artist drawing a couching woman” Austerlitz Pyramid Durer’s Rhinoceros Allégorie à la Paix d’Utrecht -
Thanks Thanks To the Devoxx selection committee for having accepted
that talk, To all the fine people at Capital Match who made all this possible, To Haskellers all over the world who fuel the fire with their wits and spirit, To Haskell B. Curry who, among others, laid out the foundations for our daily job.