The Prepared Mind

With so much information constantly being published, how can we possibly hope to keep up? My personal answer is to focus on fundamentals and frontiers. Reading research papers can be a wonderful way to do this. In an attempt to persuade you of this fact, we'll also take a look at some of the fundamentals and frontiers of enterprise IT today.

Transcript

1. Adrian Colyer | Accel Partners @adriancolyer The Prepared Mind
2. Information overload
3. You can never bridge the gap.. Credit: Simon Wardley
4. Louis Pasteur, 1854 “Chance favours the prepared mind” "Louis Pasteur". Licensed under Public Domain via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Louis_Pasteur.j pg#/media/File:Louis_Pasteur.jpg
5. Fundamentals & Frontiers
7. Becoming Finer-Grained
8. Finer-grained • Lifecycles • Deployment units • Organisations • Data partitions • Supporting Runtimes
9. openmirage.org
10. “Brutally hard and painful” - Asim Aslam • On designing and deploying internet scale services - Hamilton '07 • Life Beyond Distributed Transactions – Helland '07 • Building on Quicksand - Helland & Campbell '09 • Data on the Inside vs Data on the Outside – Helland '09 • Immutability Changes Everything – Helland '15 Towards Systems as Organisms
11. Distributed Snapshots • Chandy-Lamport '85
12. Thinking about Data • Goes around comes around – Stonebraker & Hellerstein '05 • Consistency, Availability, and Convergence – Mahajan '14 • The Declarative Imperative – Hellerstein '10 • Consistency analysis in Bloom, a CALM and collected approach – Alvaro et al. '11
13. Exploiting application invariants • Coordination avoidance in database systems – Bailis et al. '14 25x !
14. News you can use... $ 30% + 30% 32x !
15. Reduce Storage costs by 30%
16. Improve throughput by 30% • Cross-layer scheduling in cloud systems – Alkaff et al. '15
17. Do 32x less work! • ApproxHadoop: Bringing Approximations to MapReduce Frameworks – Goiri et al. '15
19. x:y:z
20. mem:stg:net
21. txt:voice:img
23. Fundamentals & Frontiers
24. the morning paper Papers announced on twitter: @adriancolyer Blog at: http://blog.acolyer.org Or subscribe to mailing list.
25. References • “The Tail at Scale” (micro-partitioning) http://blog.acolyer.org/2015/01/15/the-tail-at-scale/ • “Unikernels: Library operating systems for the cloud” http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems- for-the-cloud/ • “OSv – Optimizing the Operating System for Virtual Machines”, Kivity et al. 2014 • “On Designing and Deploying Internet Scale Services” - Hamilton 2007 • “Life Beyond Distributed Transactions” http://blog.acolyer.org/2014/11/20/life-beyond-distributed-transactions/ • “Building on Quicksand” http://blog.acolyer.org/2015/03/23/building-on-quicksand/
26. References • “Data on the outside vs Data on the inside” - Helland 2009 • “Immutability changes Everything” - Helland 2015 • “Distributed Snapshots: Determining Global States of Distributed Systems” http://blog.acolyer.org/?p=796 • “Goes Around Comes Around” - Stonebraker & Hellerstein 2005 • “The Declarative Imperative” http://blog.acolyer.org/2014/11/13/the-declarative-imperative-experience s-and-conjectures-in-distributed-logic/ • “Consistency Analysis in Bloom: A CALM and collected approach” http://blog.acolyer.org/2015/03/16/consistency-analysis-in-bloom-a-cal m-and-collected-approach/
27. References • “Consistency, Availability, and Convergence” http://blog.acolyer.org/2015/03/17/consistency-availability-and-converg ence-cops/ • “Coordination Avoidance in Database Systems” http://blog.acolyer.org/2015/03/19/coordination-avoidance-in-database- systems/ • “f4: Facebook’s warm blob storage system” http://blog.acolyer.org/2014/12/16/f4-facebooks-warm-blob-storage-sys tem/ • “A Hitch-hikers guide to fast and efficient data reconstruction in erasure coded data centers” http://blog.acolyer.org/2014/12/17/a-hitchhikers-guide-to-fast-and-effic ient-data-reconstruction-in-erasure-coded-data-centers/
28. References • “Cross-layer scheduling in Cloud Systems” http://blog.acolyer.org/2015/04/15/cross-layer-scheduling-in-cloud-syst ems/ • “ApproxHadoop: Bringing Approximations to MapReduce Frameworks” http://blog.acolyer.org/2015/04/16/approxhadoop-bringing-approximati ons-to-mapreduce-frameworks/ • “BlinkDB: Queries with bounded errors and bounded response times on very large datasets” - Agarwal et al. 2013 • “Mojim: A Reliable and Highly-Available Non-Volatile Memory System” http://blog.acolyer.org/2015/04/14/mojim-a-reliable-and-highly-availabl e-non-volatile-memory-system/
29. References • “Sirius: An open end-to-end voice and vision personal assistant and implications for future warehouse-scale computers” http://blog.acolyer.org/2015/04/13/sirius-an-open-end-to-end-voice-and- vision-personal-assistant-and-its-implications-for-future-warehouse-sca le-computers/
30. Additional Image Credits • Dust cloud - http://commons.wikimedia.org/wiki/File%3ARAF_Merlin_Helicopter_Creates_a_'Brownout'_Dust_Cloud_ Landing_in_Afghanistan_MOD_45153504.jpg • "Stonehenge Closeup". Licensed under Public Domain via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Stonehenge_Closeup.jpg#/media/File:Stonehenge_Closeup.jpg • "Stapel bakstenen - Pile of bricks 2005 Fruggo". Licensed under CC BY 1.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Stapel_bakstenen_- _Pile_of_bricks_2005_Fruggo.jpg#/media/File:Stapel_bakstenen_-_Pile_of_bricks_2005_Fruggo.jpg