Take Flight

The Scala implementation of an Object Spaces kernel written in Scala and using Futures and Promises

1. Take Flight : Fly Object Space Nigel.Warren@underscoreconsulting.com
2. Themes •  Spaces •  Fly Object Space •  Flight •  Fly and Play Demo •  Roundup https://github.com/fly-object-space google : Fly Object Space
3. Spaces Tuple Spaces – Linda - a co-ordination language Presents a boundary for signals between … Threads - Processes – Machines - Systems Minimal Interface 3 Essential Operations Timely – Lease based Immutable Only
4. Fly Operations op write(entry, lease ): lease op read(template, lease): Option[entry] op take(template, lease): Option[entry] Query By Template Template Entry Car(Some("Red"), None) == Car(Some("Red"), Some(5)) Car(None, Some(7)) != Car(Some("Red"), Some(5)) Car(None, None) == Car(Some("Red"), Some(2))
5. def write(entry: AnyRef, lease: Long) : Long Write Op
6. def read[T<: AnyRef](template: T, lease: Long): Option[T] Read Op
7. def take[T<: AnyRef](template: T, lease: Long): Option[T] Take Op
8. Time Passes Leases Expire
9. Design Diamond Minimise Interface Minimise Uses Complexity
12. Flight Idea : Advances in core Scala libraries can be applied to Fly Futures Ops are time constrained Ops may succeed or fail FiniteDurations Express only finite leases in nanos (prev millis) Make a JVM local ( inter-thread ) version that has the new features as a test bed.
13. Flight Image : Experimental Aircarft - Woolf Sverak – www.redbubble.com Experimental
14. Flight write[T <: AnyRef](entry: T, lease: FiniteDuration) : Future[FiniteDuration] = … read[T <: AnyRef](template: T, lease: FiniteDuration) : Future[T] = … take[T <: AnyRef](template: T, lease: FiniteDuration) : Future[T] = …
15. Flight – Example import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent._ import scala.concurrent.duration._ import scala.language.postfixOps import com.zink.fly.{ Flight => flt } import com.zink.fly.examples.Price //case class Price(symbol: Option[String], value: Option[Int])
16. Flight – Example val offer = Price(Some("ARM"),Some(123)) flt.write(offer, 100 seconds) onSuccess { case lease => println(s"Offer written for $lease") } "Offer written for 100 seconds"
17. Flight – Example val tmpl = Price(Some("ARM"), None) flt.read(tmpl, 10 seconds) onComplete { case Success(entry) => println(entry) case Failure(t) => println("Sorry: " + t.getMessage) } flt.take(tmpl, 10 seconds) onSuccess { case ent => println(s"Take matched $ent") }
18. Flight – Prop val tmplArm = Price(Some("ARM"),Some(123)) val tmplWlf = Price(Some("WLF"),Some(321)) val lse = 10 seconds val futArm = flt.read(tmplArm, lse) val futWlf = flt.read(tmplWlf, lse) (futArm zip futWlf) onComplete { case Success(e) => println("Deal") case Failure(t) => println("No Deal") }
19. Fly – Demo
20. Fly – Summary •  Timely Distributed Computing •  Core Scala Library only •  Work with everything else •  Flight (Experimental) Interface •  FiniteDurations •  Futures https://github.com/fly-object-space google : Fly Object Space
21. Thanks www.flyobjectspace.com www.twitter.com/flyobjectspace > <---- ---------> <---- >
22. Playing with Fly Asher Glynn asher.glynn@burrowingdeep.com
23. Overview !  Used Play 2.0 framework last year for startup !  Simple as possible – database and app servers !  Adapted to leverage Fly
24. Starting out with play !  My journey !  Copy an example !  Write logic in Action handlers !  Write DAO layer !  Discover you need an API !  Rewrite with quite a lot of swearing
25. Abstracting the logic def Story[Request, Response](request: Request) (logic: Connection => Response) (implicit validators: Seq[Request => Connection => Option[StoryError]], reactors: Request => Response => Seq[Notification]) : Either[StoryError, Response] = { … // Handle transaction }
26. Notes on Story abstraction !  Use request/response to (potentially) allow for Akka distribution !  Handles overall transaction !  Validators execute sequentially prior to logic !  Reactor executes afterwards and asynchronously writes notifications (if any) outside transaction context !  Might have gone too far with implicits
27. Notifications – with DB !  Put an adapter in but never did anything with them !  Inserting into DB easy !  Querying reasonably painful !  NoSQL + Message Bus non trivial + complex in fast moving startup !  Eventually disabled clients
28. Notifications - With Fly !  Easy filters !  Get notifications without polling with notifyWrite !  Can listen for interesting notifications out !  Trivial to implement via writeMany
29. Closing auctions !  Auction system needs once only transaction !  Late auction is a bad one (unlikely to replay in case of long outage) !  Only winning bids translate to transactions !  All nodes capable of closing transaction (for redundancy)
30. Closing auction - database !  Various options – easiest holding write lock through a select for update def pin(auctionId: UUID) = SQL ("Select * from TA_LOCKS where LOCKABLE = 'CLOSER' FOR UPDATE").execute !  Locks out other nodes !  Could be finer grained !  Needs to have extra timeout code setup to execute predictably !  Only once by virtue of changed state in DB (need to check)
31. Closing auction - Fly !  Contrast in Fly fly.take(new models.auction.Auction(id = auctionId), 1000L) !  Timeouts “for free” !  Only once by virtue of getting the take
32. Integrating Fly !  Upgrade to current version of Play !  Add fly-java lib to /lib folder !  Add to dependencies in Build.sbt "com.flyobjectspace" %% "flyscala" % "2.1.0- SNAPSHOT” !  And you are away
33. Modifications for Fly !  Modify constructor of class for null fields case class Auction ( id : UUID = null, accountId : UUID = null, !  Add trait to indicate well formedness trait WellFormed { def isWellFormed : Boolean }
34. Modifications for Fly !  And add check for WellFormedness to each story !  Add to validation chain !  Add to main part of story !  Use notifications to invalidate cache !  Write auctions to Fly, use take for once only
35. Mistakes integrating with Fly !  Going nuts with Akka !  Messing up lease times and objects disappearing !  Lease time too short !  Adding 2PC complexity to thinking rather than working within the Fly idiom
36. Running out of talent… !  Use either presence of object in Space to indicate lock – or absence! !  For Auction – absence of auction object !  For Account modifications – presence of object !  What if there are two !  What if the owner disappears !  What if the ownership changes on an edge (split brain)
37. Longer term modifications !  Modify Story abstraction to work with Futures !  Modify to work cleanly with account balances !  Keep database but redesign to work nicely with Fly !  Keep notifications out of database !  Flatten notification structure
38. Final note on notifications !  Currently using a deep class case class Notice( id: UUID, originatorId: UUID, timestamp: Long, attributes: Map[String, String], to: Seq[UUID], system: Boolean) !  Will flatten to help leverage Fly filters case class Notice (subject: String = null, _predicate: String = null, _object = String) And correlate on receivers
39. Thanks