$30 off During Our Annual Pro Sale. View Details »

E-commerce under the hood

sullis
February 21, 2017

E-commerce under the hood

Portland Java User Group
February 21, 2017
Portland Oregon
keywords: Java, Scala, Ruby, Postgres, MongoDB, DynamoDB, Gilt

sullis

February 21, 2017
Tweet

More Decks by sullis

Other Decks in Technology

Transcript

  1. E-commerce:
    under the hood
    Sean Sullivan
    February 21, 2017
    Portland Java User Group

    View Slide

  2. • software engineer
    • 20 years Java
    • 6 years at Gilt
    • back office systems @ Gilt
    About me

    View Slide

  3. www.gilt.com

    View Slide

  4. View Slide

  5. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout

    View Slide

  6. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout

    View Slide

  7. 2008
    order processing

    View Slide

  8. 2017
    discount
    service
    payment
    service
    ship calculator
    service
    order processing

    View Slide

  9. Gilt order

    View Slide

  10. Order items

    View Slide

  11. ORDERS
    ORDER_ITEMS
    Domain tables

    View Slide

  12. ORDERS
    orders_audit_trigger
    AFTER INSERT OR DELETE OR UPDATE ON orders
    FOR EACH ROW
    EXECUTE PROCEDURE orders_audit_function()
    ORDER_ITEMS
    order_items_audit_trigger
    AFTER INSERT OR DELETE OR UPDATE ON order_items
    FOR EACH ROW
    EXECUTE PROCEDURE order_items_audit_function()
    Triggers

    View Slide

  13. ORDERS_AUDIT_LOG
    ORDER_ITEMS_AUDIT_LOG
    Audit tables

    View Slide

  14. select id,status,submitted_at,shipped_at
    from orders
    where id = 85352432;
    -[ RECORD 1 ]+---------------------------
    id | 85352432
    status | x
    submitted_at | 2017-02-13 20:57:25+00
    shipped_at | 2017-02-15 05:30:16.376+00

    View Slide

  15. select count(*)
    from orders_audit_log
    where order_id = 85352432;
    -[ RECORD 1 ]
    count | 12

    View Slide

  16. select order_id,change_type,changed_at,
    old_shipped_at,new_shipped_at
    from orders_audit_log
    where order_id = 85352432
    and old_shipped_at is null
    and new_shipped_at is not null;
    -[ RECORD 1 ]--+------------------------------
    order_id | 85352432
    change_type | u
    changed_at | 2017-02-15 14:12:26.668751+00
    old_shipped_at |
    new_shipped_at | 2017-02-15 05:30:16.376+00

    View Slide

  17. More information
    https://wiki.postgresql.org/wiki/Audit_trigger
    https://wiki.postgresql.org/wiki/Audit_trigger_91plus
    https://github.com/2ndQuadrant/audit-trigger
    https://github.com/pgaudit/pgaudit

    View Slide

  18. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout

    View Slide

  19. http://www.recode.net/2014/2/4/11623056/gilts-new-business-running-logistics-for-other-e-commerce-sites
    February 2014

    View Slide

  20. February 2014

    View Slide

  21. warehouse cloud
    WMS
    invoicing
    service
    messaging
    service

    View Slide

  22. invoicing service
    • Scala
    • Akka
    • Squeryl
    • Postgres
    • db-journaling

    View Slide

  23. db-journaling
    https://github.com/gilt/db-journaling

    View Slide

  24. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout

    View Slide

  25. Checkout
    service
    Order
    processing
    Payment
    service

    View Slide

  26. OPM (for MongoDB)
    https://github.com/gilt/opm

    View Slide

  27. “OPM is a Scala library for
    managing the value of an object
    over time as a timeline of changes”
    https://github.com/gilt/opm

    View Slide

  28. Demo!
    payment history endpoint

    View Slide

  29. Case studies
    • order processing
    • logistics invoicing
    • payment processing
    • checkout

    View Slide

  30. Web
    Checkout
    Android
    Checkout
    iPhone
    Checkout
    Checkout
    service

    View Slide

  31. https://martinfowler.com/eaaDev/EventSourcing.html

    View Slide

  32. https://martinfowler.com/eaaDev/EventSourcing.html
    “Capture all changes to an
    application state as a
    sequence of events”

    View Slide

  33. https://martinfowler.com/eaaDev/EventSourcing.html
    “Event Sourcing ensures that all
    changes to application state are
    stored as a sequence of events”

    View Slide

  34. https://martinfowler.com/eaaDev/EventSourcing.html
    “The key to Event Sourcing is that we
    guarantee that all changes to the domain
    objects are initiated by the event objects"

    View Slide

  35. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/

    View Slide

  36. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    "event sourcing involves
    storing the sequence of events
    that produce the current view
    of the entity"

    View Slide

  37. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    "an event represents an operation
    or transformation from
    one view of the entity to another"

    View Slide

  38. http://blogs.atlassian.com/2014/08/replayable-transactions-event-sourcing-dynamodb/
    “events are therefore tied
    to a starting view or snapshot,
    thus making any
    transformation idempotent”

    View Slide

  39. https://github.com/gilt/lib-event-sourcing/

    View Slide

  40. import com.gilt.eventsourcing.Transform

    import com.gilt.eventsourcing.Transform.{Delete, Insert, Update}

    import com.gilt.eventsourcing.dynamodb.DynamoDbMappingEventSource
    import com.gilt.eventsourcing.dynamodb.DynamoMappings.Mappings

    import com.gilt.eventsourcing.dynamodb.client.{AttributeDefinition, _}

    import com.gilt.eventsourcing.dynamodb.util.TableHelper

    lib-event-sourcing

    View Slide

  41. object DemoDynamoDBEventSource
    extends DynamoDbMappingEventSource[JsValue]
    class DemoDynamoCommits(client: AmazonDynamoDBClient)
    extends DynamoCommits(client)
    class DemoMappingAPI(val store: Storage[Task])
    extends EventSourceAPI[Task]
    lib-event-sourcing

    View Slide

  42. val key = LogicalKey(uid)
    eventSourceApi.save(
    key,
    Some(ContentKey(Json.toJson(thing)))).run
    val result = eventSourceApi.get(key)
    lib-event-sourcing

    View Slide

  43. def put(order: Order): Task[Try[Unit]] = {
    val checkoutGuid: CheckoutSessionGuid = order.sessionGuid
    eventSourceApi.save(
    LogicalKey(checkoutGuid.toString()),
    Some(ContentKey(Json.toJson(order)))).
    map(_ => Success(())).
    handle {
    case e: Exception => Failure(e)
    }
    }
    Gilt checkout service

    View Slide

  44. The end

    View Slide

  45. View Slide

  46. Bonus slides

    View Slide

  47. https://www.youtube.com/watch?v=EKdV1IgAaFc
    “Database as a Value”

    View Slide