Maximizing Velocity with SOA at Yammer

Transcript

1. Maximizing Velocity with SOA @ Yammer Mike Ihbe Senior Infrastructure Engineer twitter.com/mikeihbe github.com/mikejihbe Thursday, May 24, 2012
2. Maximizing Velocity with SOA @ Yammer Mike Ihbe Senior Infrastructure Engineer twitter.com/mikeihbe github.com/mikejihbe Thursday, May 24, 2012
3. What is Velocity? Thursday, May 24, 2012
4. What is SOA? Technology Software design paradigm enabling organizational flexibility through composable, loosely coupled, fault tolerant business services with clearly defined, published interfaces. Thursday, May 24, 2012
5. What is SOA? Technology Software design paradigm enabling organizational flexibility through composable, loosely coupled, fault tolerant business services with clearly defined, published interfaces. + People A decentralized human-organizational structure enabling small, directed, autonomous, extremely effective engineering teams. Thursday, May 24, 2012
6. Technology + People Thursday, May 24, 2012
7. VS the monolith • Large, complex, unfocused • Difficult to reason about • Shared state/data Thursday, May 24, 2012
8. OOP Similarities • Encapsulation • Focused functionality • Clear interfaces • Reusability Thursday, May 24, 2012
9. Organizationally “Organizations...are constrained to produce designs which are copies of the communication structures of these organizations.” -Conway’s law Thursday, May 24, 2012
10. Panacea? Thursday, May 24, 2012
11. Costs Effort ὎ Complexity ὎ Thursday, May 24, 2012
12. Costs Effort ὎ Complexity ὎ Monolith: Thursday, May 24, 2012
13. Costs Effort ὎ Complexity ὎ Monolith: SOA: Thursday, May 24, 2012
14. When is SOA worth it? Thursday, May 24, 2012
15. Babies Thursday, May 24, 2012
16. Baby Tech • Web framework • Database • @Yammer • Rails • Postgres • Normal Architecture • 1 Web / 1 Db Thursday, May 24, 2012
17. 1. Get to Market 2. Find a fit Maximize Velocity Thursday, May 24, 2012
18. Terrible Twos Thursday, May 24, 2012
19. Two Tech • Caching • Feature services • Solr / Lucene / Redis / whatever • @Yammer • Memcached • Solr/tsearch/Lucene • Redis • Normal Architecture • n webs • Master-slave dbs Thursday, May 24, 2012
20. 1. Best tool for job 2. Focused team & tools Maximize Velocity Thursday, May 24, 2012
21. Bootstrapping SOA @Yammer - Core Services team to own service creation & maintenance Thursday, May 24, 2012
22. Learning to Speak “MTTR” “Reliability” “Uptime” “Disaster Recovery” “MTBF” Thursday, May 24, 2012
23. Speaking Tech • MTTR - (semi) Automated Recovery • Monitoring • Backups • @Yammer • Smorgasbord :-( • Normal Architecture • Ganglia / Nagios / Cacti / whatever • In-house recovery scripts Thursday, May 24, 2012
24. 1. Monitor 2. Automate Recovery 3. Standard provisioning Maximize Velocity Thursday, May 24, 2012
25. Adolescence Thursday, May 24, 2012
26. Adolescence Thursday, May 24, 2012
27. Adolescence Thursday, May 24, 2012
28. Adolescence Thursday, May 24, 2012
29. Thursday, May 24, 2012
30. Thursday, May 24, 2012
31. Thursday, May 24, 2012
32. Cross-functional Teams Thursday, May 24, 2012
33. Cross-functional Teams Dexie Flatterie Query Cursor based Denormalizing Index building Search Search Infrastructure Thursday, May 24, 2012
34. Cross-functional Teams Completie Autocomplete Dexie Flatterie Query Cursor based Denormalizing Index building Search Search Infrastructure Thursday, May 24, 2012
35. Cross-functional Teams Completie Autocomplete Slurpie Data Export Dexie Flatterie Query Cursor based Denormalizing Index building Search Search Infrastructure Thursday, May 24, 2012
36. DISTRUST Thursday, May 24, 2012
37. SOA Technology XML SOAP JSON WSDL REST UDDI RPC ProtoBuf Thrift Thursday, May 24, 2012
38. Adolescent Tech • Varies widely - Tooling, Provisioning • @Yammer • Dropwizard - https://github.com/codahale/dropwizard - Jetty, Jersey, Jackson, Metrics, Guava, Log4j, Hibernate Validator • Quickly deploy monitored, alerting, metrics reporting, logging fat Jars • Diploymacy - Deploy artifacts from CI • Partie - Framework for distributed BDB-HA KV store Thursday, May 24, 2012
39. 1. Standardize tools 2. Minimize time to production Maximize Velocity Thursday, May 24, 2012
40. Adulthood Thursday, May 24, 2012
41. Adult Tech • @Yammer - Specialized Domain Services • Feedie - 10+ Billion messages, 45 node cluster • http://vimeo.com/41062751 • Streamie - Activity stream deliveries, Riak • http://basho.com/blog/technical/2011/03/28/Riak-and-Scala-at-Yammer/ Thursday, May 24, 2012
42. Thursday, May 24, 2012
43. SOA Setup Tips • Internalize SOA • Direct, Decentralize, Divide & Conquer • Monitor Everything • Limit dependencies • Handle Failure Modes • Keep it simple Thursday, May 24, 2012
44. is hiring Mike Ihbe Senior Infrastructure Engineer twitter.com/mikeihbe github.com/mikejihbe Thursday, May 24, 2012