Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A story about distributed computing before the cloud arrived - Rogier Lommers - bol.com

6e3ea86995d93d35c0fadf2694bca773?s=47 GoDays
January 23, 2020

A story about distributed computing before the cloud arrived - Rogier Lommers - bol.com

What do you do when you need a lot of hardware for a short period, but you are unable to get it the normal way? Well, think out-of-the-box... During this presentation I will tell you about how we applied distributed processing in a very unconventional way: by making use of our colleagues laptops and a little bit of Go's cross-platform compiling.

6e3ea86995d93d35c0fadf2694bca773?s=128

GoDays

January 23, 2020
Tweet

Transcript

  1. A story about distributed computing before the cloud arrived Re-design

    for the cloud Rogier Lommers software architect
  2. who is bol.com? 2004-2007 2008-2009 2010 2011 2012 2013 2014

    2015 1999-2003 2016 2017 Selling secondhand Books Entertainment Electronics Small household items Baby Baby & Pregnancy clothing Toys Animal Jewelry, Watches & Bags Businesses selling through bol.com Reading digitally Special, right? Started in a port-a-cabin Active in Belgium Takeover by Ahold Logistics through bol.com bol.com app Unlimited reading Cooking & Dining Garden & DIY Big household items Beauty & Health Save Constantly 2018 Alcohol Shoes Health Fusion Ahold Delhaize Expansion bol.com fulfilment center 2019 Living & Sleeping Sport & Leisure Opening offices Belgium Audio books Sustainable packaging First steps sustainable products Introduction
  3. None
  4. None
  5. None
  6. None
  7. 102 million in database 12 million incoming URLs, each day

    Assets
  8. Mobile website / Mobile weblog / Mobile marketing eBook apps

    External apps Multi-channel approach
  9. None
  10. the asset pipeline fetcher event-queue asset processing part renderer matcher

    serving part original assets renditions { "height": 1056, "width": 342, "fit": "fill", "fill_color": "white", "trim": "auto", "output_format": "image/jpeg” } synchronous
  11. size does matter…

  12. google introduced guetzli - a simple, standalone CLI tool, available

    for linux, macOS and Windows - results are fantastic: 20% / 45% reduction in jpeg file sizes - no loss in perceived quality a new open source jpeg encoder
  13. 87 kb

  14. None
  15. 87 kb 64 kb BEFORE GUETZLI AFTER GUETZLI

  16. the asset pipeline fetcher event-queue asset processing part renderer matcher

    serving part original assets renditions { "height": 1056, "width": 342, "fit": "fill", "fill_color": "white", "trim": "auto", "output_format": "image/jpeg” } synchronous guetzli queue
  17. None
  18. None
  19. None
  20. the asset pipeline fetcher asset processing part renderer event-queue matcher

    serving part original assets renditions { "height": 1056, "width": 342, "fit": "fill", "fill_color": "white", "trim": "auto", "output_format": "image/jpeg” } synchronous guetzli queue colleague 1 colleague n GET /work POST /work colleague 1 colleague n colleague 1 colleague n colleague 1 colleague n
  21. the “miracle”-client - written in Go - CLI (linux, mac

    and windows) - one binary, easy installation (opt-in) - guetzli binary embedded (github.com/jteeuwen/go-bindata) - auto-update (github.com/inconshreveable/go-update) - opt-in for colleagues - company software portal was able to push the client to the opted-in-people
  22. how it works colleague renderer GET /work guetzli queue renditions

    POST /work GUETZLI
  23. the game is on

  24. None
  25. conclusion

  26. thanks! Rogier Lommers rlommers@bol.com