アルプのEff独自エフェクト集 / Alp-original ’Eff’ pearls

Now in its fourth year at Alp, Inc, Eff has been in continuous use since the startup's foundation.
In this talk, I will introduce our original Effects that originated from practical needs, and those we envision in the future.

Transcript

1. Alp-original ’Eff’ pearls Scala Matsuri 2022 @ma2k8 Ξϧϓͷ& ff ಠࣗ& ff FDUू
2. About me > Tsubasa Matsukawa - SWE at Alp, Inc. @wing_007 ࣗݾ঺հ @ma2k8 https://ma2k8.hateblo.jp/
3. Our Products > Scalebase This is software for launch up and managing and revenue management their subscription- based services. ΞϧϓͰ͸4DBMFCBTFͱ͍͏αϒεΫϦϓγϣϯ؅ཧɺܦӦ෼ ੳͷ4BB4ϓϩμΫτΛఏڙ͍ͯ͠·͢
4. Before I get to the point… ຊ୊ʹೖΔલʹʜ
5. ಠࣗఆٛ& ff FDUͷϝϦοτͱͯ͠ɺࡉ͔͍&SSPS)BOEMJOH΍ɺ & ff FDUͷӨڹൣғͷہॴԽͳͲ͕͋Γ·͕͢ɾɾɾ > Detailed error handling for each effect. > Modularity of effect by ‘Eff’. > Split expression and execution pros of proprietary definition effects
6. I'll explain the problems that can arise when you stack too many effects in ‘Eff'. & ff Ͱ͸ޮՌΛͨ͘͞ΜੵΉͱ
   ໰୊ʹͳΔ఺͕͋Γ·͢
7. ”Effect wiring problem” & ff FDUͷ഑ઢ໰୊Ͱ͢
8. What is ”Effect wiring problem”? & ff FDUͷ഑ઢ໰୊ͱ͸ʁ
9. First, ‘Eff’ separates the definition of an expression from its execution. ·ͣલఏͱͯ͠ɺ& ff Ͱ͸ࣜͷఆٛͱ࣮ߦ͕
   ෼͔Ε͍ͯΔͱ͍͏ੑ࣭͕͋Γ·͢
10. ‘expression’ is no interpret effect and use effect Logics. bࣜͷఆٛ`͸ΠϯλϓϦλʹධՁ͞ΕΔ·Ͱ͸
    ͳΜͷ࡞༻΋࣋ͪ·ͤΜ
11. When executing an Effect-A, you may want to use the properties of another Effect-B. *OUFSQSFUFSʹΑΔ& ff FDUͷ࣮ߦ࣌ʹɺ
    ผͷ& ff FDUͷಛੑΛར༻ͨ͘͠ͳΔ͜ͱ͕Α͋͘Γ·͢ Effect-A Use Effect-B
12. ͋Δ& ff FDU"͕& ff FDU#ͷੑ࣭΋ར༻͢Δ৔߹ɺ
    & ff FDU4UBDL͔ΒऔΓग़͢ॱ൪͕ݻఆ͞Ε·͢ When an effect-A also uses effect-B, the order of extraction from the EffectStack is fixed. Effect-Stack Effect-A Effect-B Deps Effect-A Interpreter Effect-B Interpreter 1. Pick effect-A 2. Run effect-A and use effect-B 3. Stack effet-B 4. Pick effect-B 5. Run effect-B
13. In particular, ”Either” and “Async” appears in various Effect-interpreters. ಛʹ&JUIFSͱඇಉظܥ 5BTL 'VUVSF ͸
    ͍Ζ͍Ζͳ& ff FDUͷΠϯλϓϦλʹग़͖ͯ·͢
14. This constraint becomes more complex as the number of effects increases. ͜ͷ੍໿͸& ff FDUͷ਺͕ଟ͘ͳΔʹͭΕ΍΍͘͜͠ͳΓɺ
    ৗʹ& ff FDUͷґଘؔ܎Λҙࣝͯ͠ίʔυΛॻ͘ඞཁ͕͋Γ·͢
15. How to execute? Effect-B Effect-C Effect-F Effect-A Effect-E Effect-D Effect-Stack Ͳ͏࣮ߦ͢Δ͔ʁ
16. Alp, Inc. addresses this problem by always running all effects. ΞϧϓͰ͸ৗʹશͯͷF ff FDUΛSVO͢Δख๏Ͱ
    ͜ͷ഑ઢ໰୊Λରॲ͍ͯ͠·͢
17. ”run-all” SVOBMMͱ͸
18. ͜ͷ༷ʹ& ff FDU͕4UBDL͞Ε͕ͨࣜͰ͖·͢ Effect-A Effect-B Effect-C Expression with A, B, C stacked
19. ೉͍͠ͷ͸ࣜΛධՁ͢Δॱ൪Ͱ͢ Effect-A Effect-B Effect-C The hard part is the order in which the 'expressions' are evaluated. Effect-B Effect-C Effect-C Run A Run B
20. SVOBMMͰ͸ࣜʹશͯͷ& ff FDU͕ೖͬͨલఏʹ͢Δ͜ͱͰɺ
    ࣮ߦͷॱ൪Λݻఆ͍ͯ͠·͢ Effect-A Effect-C In run-all, all effects are assumed to be included in the expression. The order of execution is fixed. Run A Effect-C Run B Effect-C B doesn't exist, but it runs. Expression without e ff ect B
21. & ff FDU4UBDLͷਪҠΛ໌ࣔతʹॻ͘͜ͱͰ࣮ݱ͍ͯ͠·͢ Write the EffectStack transitions explicitly. implicit class AllOps[R, A](effects: Eff[R, A]) { def runAll[U1, U2, U3, U4, U5, U6, U7, U8, U9, U10](withTx: Boolean)( implicit clockmAux: Member.Aux[ClockM, R, U1], idGenmAux: Member.Aux[IdGenM, U1, U2], authzIOAux: Member.Aux[AuthzIO, U2, U3], authzIOMs: _authzScope[U3], authzIOM1: _ppErrorEither[U3], authzIOM2: _task[U3], auditLogMAux: Member.Aux[AuditLogM, U3, U4], auditLogM1: _task[U4], auditLogM2: _ppErrorEither[U4], auditLogM3: _kfirehoseio[U4], kinesisFirehoseIOAux: Member.Aux[KinesisFirehoseIO, U4, U5], kinesisFirehoseIOM1: _task[U5], kinesisFirehoseIOM2: _ppErrorEither[U5], ɹ ……
22. Is it too slowly process? ॲཧ͕஗͘ͳΒͳ͍ͷͰ͠ΐ͏͔ʁ
23. ͭΤϑΣΫτ͕ੵ·Ε͍ͯΔࣜΛɺͭͷSVOͰ࣮ߦ͢ΔKVTU ͱͭͷSVOͰ࣮ߦ͢ΔBMMͰϕϯνϚʔΫΛऔΓ·ͨ͠ Benchmark def justBench() = { type R = Fx.fx3[Task, IdGenM, PpErrorEither] def program[R: _task: _idgenm: _ppErrorEither]: Eff[R, Unit] = { for { _ <- fromTask[R, Unit](Task.delay((): Unit)) _ <- OperatorId.generate[R] _ <- fromPpError[R, Unit](Right((): Unit)) } yield () } val s: Scheduler = Scheduler.global Await.result(program[R].runIdGenM().runPpError.runAsync.runToFuture(s), Duration.Inf) } def allBench() = { type R = Fx.fx6[Task, IdGenM, ClockM, PpErrorEither, Option, List] def program[R: _task: _idgenm: _ppErrorEither: _option: _list]: Eff[R, Unit] = { for { _ <- fromTask[R, Unit](Task.delay((): Unit)) _ <- OperatorId.generate[R] _ <- fromPpError[R, Unit](Right((): Unit)) } yield () } val s: Scheduler = Scheduler.global Await.result(program[R].runIdGenM().runClockM().runOption.runList.runPpError.runAsync.runToFuture(s), Duration.Inf) } •Benchmarked an expression with 3 effects stacked •'justBench' with 3 runs •‘allBench' with 6 runs
24. [info] Benchmark Mode Cnt Score Error Units [info] AllEffectBench.all sample 1091700 3.557 ± 0.045 us/op [info] AllEffectBench.all:all·p0.00 sample 3.000 us/op [info] AllEffectBench.all:all·p0.50 sample 3.208 us/op [info] AllEffectBench.all:all·p0.90 sample 3.540 us/op [info] AllEffectBench.all:all·p0.95 sample 3.748 us/op [info] AllEffectBench.all:all·p0.99 sample 4.872 us/op [info] AllEffectBench.all:all·p0.999 sample 9.408 us/op [info] AllEffectBench.all:all·p0.9999 sample 844.452 us/op [info] AllEffectBench.all:all·p1.00 sample 1859.584 us/op [info] AllEffectBench.just sample 822171 2.677 ± 0.150 us/op [info] AllEffectBench.just:just·p0.00 sample 1.790 us/op [info] AllEffectBench.just:just·p0.50 sample 1.958 us/op [info] AllEffectBench.just:just·p0.90 sample 2.164 us/op [info] AllEffectBench.just:just·p0.95 sample 2.416 us/op [info] AllEffectBench.just:just·p0.99 sample 7.288 us/op [info] AllEffectBench.just:just·p0.999 sample 65.386 us/op [info] AllEffectBench.just:just·p0.9999 sample 913.987 us/op [info] AllEffectBench.just:just·p1.00 sample 18612.224 us/op Φʔόʔϔου͸খ͍͞ Little overhead Small overhead because the interpreter is not processed, only the effects are extracted
25. Here's the thing. ͔͜͜Β͕ຊ୊
26. Original-effect in Alp.Inc ΞϧϓͰ࢖͍ͬͯΔಠࣗ& ff FDUΛҰ෦঺հ͠·͢
27. *E(FO. IdGenM
28. IdGenM > Simple IdGenerator. > Returns domain object's id. γϯϓϧʹ66*%Λੜ੒ͯͦ͠ΕͧΕͷυϝΠϯϞσϧͷ*%ܕ ʹ٧Ίͯฦ͢& ff FDUͰ͢
29. ClockM $MPDL.
30. ClockM > Perform date & time operations. ࣌ؒͷૢ࡞Λߦ͏& ff FDU
31. CryptoM $SZQUP.
32. CryptoM > execute crypto & decrypt. > use library ‘tink’. > use kms to store the crypto key. ҉߸Խɺ෮߸Λߦ͏& ff FDUͰ͢
33. HashM )BTI.
34. ϋογϡԽͱϋογϡͷൺֱΛߦ͏& ff FDUͰ͢ HashM > use library ‘tink’. > execute generate & verify.
35. AuditLogM "VEJU-PH.
36. AuditLogM > It is always evaluated at the end of the process. > Publish system audit log to s3 via kinesis. > Use the SafeEffect of atnos-eff. ɹ> Since we also need to log processing failures, we can use SafeEffect, to run "finally processing”. γεςϜ؂ࠪϩάͷ& ff FDUͰ͢
    ,JOFTJTܦ༝Ͱ4΁อଘ͠·͢
37. TransactionTask 5SBOTBDUJPO5BTL
38. TransactionTask > Manage DB Access & Transaction. > Use Scalikejdbc. > Extracts the Either effect from the EffectStack and controls the rollback if Left is included. > There is dryrun mechanism that performs a rollback after the query is executed. %#
    "DDFTT 5SBOTBDUJPOΛ؅ཧ͠·͢
    &JIUFSͷ݁ՌΛ֬ೝͯ͠SPMMCBDL͢Δػೳ΍ɺESZSVO͢Δ࢓૊Έ͕͋Γ·͢ Case-study materials: https://speakerdeck.com/kaelaela/api-design-by-clean-architecture-and-effɹ
39. Effects being implemented at Alp Inc. "MQͰ࣮૷தͷ& ff FDUΛ঺հ͠·͢
40. AuthZ "VUI;
41. Authorization is complex and coverage is wide-ranging Presenter Controller Repository(DB etc..) UseCase Domain Masking item Execute endpoint Filter resource read/write auhorization Execute UseCase Ramification domainLogic Execute domainLogic ೝՄ͸ద༻ൣғ͕޿͘ɺཁ݅࣍ୈͰ͸1SFTFOUFSd%PNBJO·Ͱ
    Өڹ͠͏Δɺͱͯ΋ෳࡶͳ֓೦Ͱ͢
42. Here is a good implementation that takes advantage of ’Eff’ property of separating expression and execution. ೝՄͷ࢓૊ΈΛ& ff ͷࣜͱ࣮ߦΛ෼཭͢Δੑ࣭Λར༻͠ɺ
    ྑ͍ײ͡ʹ࣮૷Ͱ͖ͨͷͰ͝঺հ͠·͢
43. Principal & Resource ೝՄ͸1SJODJQBMͱ3FTPVDF͔Βߏ੒͞Ε·͢ Principal Resource
44. Principal? Resource? 1SJODJQBM
    
    ೝՄର৅
    3FTPVSDF
    
    ೝՄର৅ͷૢ࡞ʹӨڹ͢Δର৅ > Principal > Authorization target ex. User, AcessToken etc.. > Resource > object that affect principal operation. ex.Repository,domainLogic,UseCase etc.. > What is treated as Principal may also be treated as Resource.
45. Principal & Resource has Authz attribute ͜ΕΒʹ͸ͦΕͧΕೝՄʹ࢖͏ଐੑΛ͍࣋ͨͤͯ·͢ "#"$ Principal Resource Policy Scope
46. Policy ? Scope ? 1PMJDZ͸1SJODJQBMʹ෇༩͞ΕΔೝՄଐੑ
    4DPQF͸3FTPVSDFʹ෇༩͞ΕΔೝՄ৘ใ > Policy - Authorization attribute assigned to Principal - In addition to authorization information, it has resoruce id and other information used for id-based filters > Scope - Authorization attribute assigned to Resource.
47. Resource exists in various places, making it difficult to handle in a unified handling. ೝՄͷର৅Ͱ͋Δ3FTPVSDFͷ4DPQF͕͞·͟·ͳ৔ॴʹ
    ଘࡏ͢ΔͨΊɺ౷ҰతͳϋϯυϦϯά͕೉͍͠໰୊͕͋Γ·͢
48. Solutions ͜ΕΛએݴతͳ4DPQFઃఆɺ4UBUFϞφυɺΠϯλϓϦλ࣮ߦ ࣌ʹೝՄνΣοΫΛڧ੍తʹ૊ΈࠐΉ͜ͱͰղܾͰ͖·ͨ͠ > Declarative scope setting > State monad > Enforce authorization checks at interpreter execute
49. Declarative scope setting υϝΠϯϞσϧͷ*%౳ʹείʔϓΛ෇༩͢Δ Assign scope to domain model IDs, etc.
50. State monad ಠࣗΤϑΣΫτͰɺ
    ઃఆͨ͠είʔϓΛ4UBUFʹQVU͠·͢ Put the scope set by the original effect to State
51. State monad είʔϓ͸ࢦఆͷܕʹแΊ͹
    ࣗಈͰΤϑΣΫτελοΫʹੵ·ΕΔΑ͏ʹ͠·ͨ͠ 
 TDBMB fi YͰܕͷࢦఆ͕࿙Ε͍ͯΕ͹ίϯύΠϧΤϥʔʹ͠·͢ > By setting the return value of the interface to the specified type, the scope is automatically loaded onto the effects stack. > If the type specification is omitted, ’scalafix’ will generate a compile error.
52. State monad ͜ΕʹΑ֤ͬͯ૚ʹލͬͨೝՄείʔϓΛ
    ͭͷ4UBUFͰ؅ཧͰ͖·͢ > This allows for the management of authorization scopes across all tiers in a single State. Presenter Controller Repository(DB etc..) UseCase Domain Set Scope A Set Scope B Set Scope C,D Set Scope E Set Scope F State[List[A,B,C,D,E,F], X]
53. Enforce authorization checks at interpreter execute & ff FDUΛΠϯλϓϦλʹ౉͢खલͰɺ
    1SJODJQBMͷݖݶϦΫΤετΛڬΈ·͢ Before executing your own effects Interrupt the authorization request of the Principal
54. Enforce authorization checks at interpreter execute όονͳͲɺݖݶνΣοΫΛলུ͍ͨ͠έʔεͰ͸
    লུͰ͖·͢ Can be omitted in cases where you want to omit permission checks, such as in batches
55. Enforce authorization flow ೝՄͷద༻ϑϩʔ 1. Expression Assembly Phase Define a command to set the required scope to State at any point in the code base. You will have an expression with the scope you need to execute it. 2. Call interpreter Phase 3. Run interpreter Phase program.run() authzInterpreter.run( program <<requestAuthzScope(principalId) ) val program: Eff[R, A] Stack State[List[ActionCompose], A] & others(DB,Task,Either, etc..) Passes an expression with scope to the interpreter. Incorporate authorization request commands when passing. This allows us to enforce the authorization request when obtaining ’Eff[R,A] => A’. When the ‘requestAuthzScope' command is evaluated, a allow/ deny decision is made based on the scope set and permissions associated with the principal. If the decision is "deny", an Either.left is added to the effect stack and the computation is stopped. for { state <- get[U, List[ActionComposing]] attachedPolicy <- showPolicy(principalId) attachedAction = attachedPolicy.policies.map(_.action) …
56. ͜ΕΒͷϝϦοτΛڗडͰ͖ΔΑ͏ʹͳΓ·ͨ͠ Pros > Scopes set up in various places can be collected by State monad. > Can create a situation where an authorization check is mandatory to evaluate an expression. > By placing the permission check before execution, it is no longer necessary to separate implementations based on whether permission checks are required or not.
57. ͜͜Ͱఏࣔͨ͠΋ͷΑΓࡉ͔͍ೝՄϋϯυϦϯά͸
    "VUI[*0
    & ff FDUͷ௚઀1PMJDZΛऔΓѻ͏ίϚϯυͰରԠ More detailed authorization handling > Handled by 'AuthzIO Effects' direct policy handling commands.
58. Effects to be implemented in Alp, Inc. "MQͰ࣮૷༧ఆͷ& ff FDU΋গ͠঺հ͠·͢
59. Compensation Transaction > Implementing the ‘Saga Pattern’. > Could be used to notify multiple elements of a transaction status update. ิঈτϥϯβΫγϣϯ͸& ff FDUͱͷ૬ੑ͕ྑͦ͞͏Ͱ͢
60. Akka-Stream > Use in paging control. > Use for chunk control in batch. "LLB4USFBNͷ& ff FDU͸
    ϖʔδϯάॲཧ΍όονͷνϟϯΫ੍ޚͳͲʹ࢖͑ͦ͏Ͱ͢
61. Macro-string > Macro-string generation and parsing effects are being implemented. > Provide users with a way to embed expressions and variables > Unlike the effects introduced so far, these effects are closer to the domain. Ϣʔβʔ͕จࣈྻʹࣜ΍ม਺ΛຒΊࠐΊΔػೳͷఏڙ΋ 
 & ff FDUͰߦ͏Α͏։ൃதͰ͢
62. Logic switch per provider > If the SaaS needs to behave slightly differently for a particular customer, the interpreter can be used to switch to a specific logic. 4BB4ͷ༷ͳαʔϏεܗଶͰ͸ɺސ٬ຖͷݻ༗ϩδοΫͷ࣮ݱ ͸೉͍͕͠ɺΠϯλϓϦλ੾Γସ͑Ͱ͋Δఔ౓ରԠͰ͖Δ
63. ‘Eff’ is Good! 👍👍 & ff ͸͍͍ͧʂ
64. Thanks! ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ