E-commerce under the hood

Portland Java User Group
February 21, 2017
Portland Oregon
keywords: Java, Scala, Ruby, Postgres, MongoDB, DynamoDB, Gilt

Transcript

1. E-commerce:
   under the hood
   Sean Sullivan
   February 21, 2017
   Portland Java User Group
   

   View Slide

2. • software engineer
   • 20 years Java
   • 6 years at Gilt
   • back office systems @ Gilt
   About me
   

   View Slide

3. www.gilt.com
   

   View Slide

4. View Slide

5. Case studies
   • order processing
   • logistics invoicing
   • payment processing
   • checkout
   

   View Slide

6. Case studies
   • order processing
   • logistics invoicing
   • payment processing
   • checkout
   

   View Slide

7. 2008
   order processing
   

   View Slide

8. 2017
   discount
   service
   payment
   service
   ship calculator
   service
   order processing
   

   View Slide

9. Gilt order
   

   View Slide

10. Order items
    

    View Slide

11. ORDERS
    ORDER_ITEMS
    Domain tables
    

    View Slide

12. ORDERS
    orders_audit_trigger
    AFTER INSERT OR DELETE OR UPDATE ON orders
    FOR EACH ROW
    EXECUTE PROCEDURE orders_audit_function()
    ORDER_ITEMS
    order_items_audit_trigger
    AFTER INSERT OR DELETE OR UPDATE ON order_items
    FOR EACH ROW
    EXECUTE PROCEDURE order_items_audit_function()
    Triggers
    

    View Slide

13. ORDERS_AUDIT_LOG
    ORDER_ITEMS_AUDIT_LOG
    Audit tables
    

    View Slide

14. select id,status,submitted_at,shipped_at
    from orders
    where id = 85352432;
    -[ RECORD 1 ]+---------------------------
    id | 85352432
    status | x
    submitted_at | 2017-02-13 20:57:25+00
    shipped_at | 2017-02-15 05:30:16.376+00
    

    View Slide

15. select count(*)
    from orders_audit_log
    where order_id = 85352432;
    -[ RECORD 1 ]
    count | 12
    

    View Slide

16. select order_id,change_type,changed_at,
    old_shipped_at,new_shipped_at
    from orders_audit_log
    where order_id = 85352432
    and old_shipped_at is null
    and new_shipped_at is not null;
    -[ RECORD 1 ]--+------------------------------
    order_id | 85352432
    change_type | u
    changed_at | 2017-02-15 14:12:26.668751+00
    old_shipped_at |
    new_shipped_at | 2017-02-15 05:30:16.376+00
    

    View Slide

17. More information
    https://wiki.postgresql.org/wiki/Audit_trigger
    https://wiki.postgresql.org/wiki/Audit_trigger_91plus
    https://github.com/2ndQuadrant/audit-trigger
    https://github.com/pgaudit/pgaudit
    

    View Slide

18. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout
    

    View Slide

19. http://www.recode.net/2014/2/4/11623056/gilts-new-business-running-logistics-for-other-e-commerce-sites
    February 2014
    

    View Slide

20. February 2014
    

    View Slide

21. warehouse cloud
    WMS
    invoicing
    service
    messaging
    service
    

    View Slide

22. invoicing service
    • Scala
    • Akka
    • Squeryl
    • Postgres
    • db-journaling
    

    View Slide

23. db-journaling
    https://github.com/gilt/db-journaling
    

    View Slide

24. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout
    

    View Slide

25. Checkout
    service
    Order
    processing
    Payment
    service
    

    View Slide

26. OPM (for MongoDB)
    https://github.com/gilt/opm
    

    View Slide

27. “OPM is a Scala library for
    managing the value of an object
    over time as a timeline of changes”
    https://github.com/gilt/opm
    

    View Slide

28. Demo!
    payment history endpoint
    

    View Slide

29. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout
    

    View Slide

30. Web
    Checkout
    Android
    Checkout
    iPhone
    Checkout
    Checkout
    service
    

    View Slide

31. https://martinfowler.com/eaaDev/EventSourcing.html
    

    View Slide

32. https://martinfowler.com/eaaDev/EventSourcing.html
    “Capture all changes to an
    application state as a
    sequence of events”
    

    View Slide

33. https://martinfowler.com/eaaDev/EventSourcing.html
    “Event Sourcing ensures that all
    changes to application state are
    stored as a sequence of events”
    

    View Slide

34. https://martinfowler.com/eaaDev/EventSourcing.html
    “The key to Event Sourcing is that we
    guarantee that all changes to the domain
    objects are initiated by the event objects"
    

    View Slide

35. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    

    View Slide

36. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    "event sourcing involves
    storing the sequence of events
    that produce the current view
    of the entity"
    

    View Slide

37. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    "an event represents an operation
    or transformation from
    one view of the entity to another"
    

    View Slide

38. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    “events are therefore tied
    to a starting view or snapshot,
    thus making any
    transformation idempotent”
    

    View Slide

39. https://github.com/gilt/lib-event-sourcing/
    

    View Slide

40. import com.gilt.eventsourcing.Transform

    import com.gilt.eventsourcing.Transform.{Delete, Insert, Update}

    import com.gilt.eventsourcing.dynamodb.DynamoDbMappingEventSource
    import com.gilt.eventsourcing.dynamodb.DynamoMappings.Mappings

    import com.gilt.eventsourcing.dynamodb.client.{AttributeDefinition, _}

    import com.gilt.eventsourcing.dynamodb.util.TableHelper

    lib-event-sourcing
    

    View Slide

41. object DemoDynamoDBEventSource
    extends DynamoDbMappingEventSource[JsValue]
    class DemoDynamoCommits(client: AmazonDynamoDBClient)
    extends DynamoCommits(client)
    class DemoMappingAPI(val store: Storage[Task])
    extends EventSourceAPI[Task]
    lib-event-sourcing
    

    View Slide

42. val key = LogicalKey(uid)
    eventSourceApi.save(
    key,
    Some(ContentKey(Json.toJson(thing)))).run
    val result = eventSourceApi.get(key)
    lib-event-sourcing
    

    View Slide

43. def put(order: Order): Task[Try[Unit]] = {
    val checkoutGuid: CheckoutSessionGuid = order.sessionGuid
    eventSourceApi.save(
    LogicalKey(checkoutGuid.toString()),
    Some(ContentKey(Json.toJson(order)))).
    map(_ => Success(())).
    handle {
    case e: Exception => Failure(e)
    }
    }
    Gilt checkout service
    

    View Slide

44. The end
    

    View Slide

45. View Slide

46. Bonus slides
    

    View Slide

47. https://www.youtube.com/watch?v=EKdV1IgAaFc
    “Database as a Value”
    

    View Slide