How Yahoo Finance Android scales (AnDevCon 2017, DC)

I spoke about the various techniques we employ around understanding our users, planning, CI/CD, experimentation, coding, culture and performance to deliver high quality software to our users at AnDevCon 2017 Washington DC.

Transcript

1. @vikramb0
2. How Finance Android scales Vikram Bodicherla @vikramb0
3. A great idea won’t get you there!
4. A great idea won’t get you there! Great code Excellent feature set State of the art UI
6. Convert ideas to an MVP
7. Launch product and get feedback
8. Iterate on ideas from data
10. • Focus on learning • Minimize time through the loop
11. Users have high expectations • Fine-tuned, personalized solutions • Delightful, state of the art • Stable and fast
22. • Be ready to pivot • Spend only as much effort as you need for the given iteration
23. • Design for engineering
24. • Plan well • Talking deadlines
25. • Continuous integration • Deployable partial features
26. • Continuous Deployments • Automate your Playstore deployments
27. • Engaging with your users
28. Agenda • What does a good feature definition look like? • Practicing good scrum • CD all the way • Culture for winning • Architect for awesomeness
29. A good feature definition Know one when you see it
30. Meet John • Joined last minute because a friend dropped out • Knows nothing about football. Probably cares even less. • Drafts Roberto Aguayo in the 2nd round because Tom told him he’s good (screw Tom) • 12% of fantasy football players. Over 60% are in their first season
31. Building personas • Important attributes of your users • Personas define the user at the center of the cluster • Focusing on fewer personas help create focus • Cluster as a means to explanation
32. Know when you have succeeded • You need to know where the goal post is • Vanity metrics – DAUs/Downloads etc. won’t tell you how your feature is doing • Clarity metrics – engagement of the feature – how your feature is doing • http://firstround.com/review/im-sorry-but-those-are-vanity-metrics/
33. Data, not opinions • Wrong decisions invalidated quickly > no decisions • Ward off HiPPO attacks • An experiment should have • a hypothesis • an owner • a deadline
35. Practicing good scrum
36. Plan • Get a sense of time, scale and complexity • Identify blockers and dependencies • Establish owners • Aids distribution and parallelization of work
37. Many ways to plan • Waterfall • rigorous planning • sequential
38. Many ways to plan • Waterfall • rigorous planning • sequential • Agile • working software • people-centric • iterative, not change-averse • Read the agile manifesto (http://agilemanifesto.org)
39. What’s a good scrum story? • A part of a feature that adds user value • A feature is a collection of stories Ex: add stock to watchlist by tapping on star • User story: Value is perceivable to the user • Engineering story: Value not directly perceivable to the user
40. What’s a good scrum story? Story: A user should be able to add a stock to their watchlist by tapping on the star on a quote page Acceptance criterion • If a stock is not in the watchlist, tapping should add it • If a stock is in the watchlist, tapping should remove it • Add the following analytics event for star tap • Add the necessary Material-design compliant animations
41. Scrum • Grooming • Understand • Point
42. Grooming • Familiarize yourself with the backlog • Ensure stories are clear • Point using poker • Ensure >1 sprint’s worth stories are available
43. Grooming Best practices • User stories with reasonable detail • Stories have acceptance criterion • Planning poker to point • More stories, the better • Timebox at 1 hour Anti-patterns • Rushing to story point • Writing stories in the meeting • “I know we don’t have all details, but let’s point it anyways”
44. Scrum • Grooming • Sprint planning • Impl. details • Tasks
45. Sprint planning • Discuss how to implement • Break down stories into tasks
46. Sprint planning Best practices • Tasks < 1 day • Have a goal for the sprint • Plan to release at end of sprint • Discuss so product/design/UX can get out of the room first Anti-patterns • Team struggles to identify tasks • Tasks > 1 day • Discussions around what a story means
47. Scrum • Grooming • Sprint planning
48. Scrum • Grooming • Sprint planning • Standup • On track? • Blockers
49. Standups • Ensure team on track • Share impediments to blockers • This is NOT a status updates/knowledge sharing meeting
50. Standups Best practices • Everyone updates stories prior • Be explicit – what’s done, what’s coming up, blockers • Parking lot for long discussions • Agree on defn. of ‘Done’ • Focus of accomplishing everything Anti-patterns • Debate product functionalities • Doing involved demos • Providing a lot of detail • Trying to resolve issues/blockers • Standups > 10 min • “This story is done, except for code review and testing”
51. Standups Best practices • Everyone updates stories prior • Be explicit – what’s done, what’s coming up, blockers • Parking lot for long discussions • Agree on defn. of ‘Done’ • Focus of accomplishing everything Anti-patterns • The big boss needs some clarifications
52. Scrum • Grooming • Sprint planning • Standup • Demo • Finished stories
53. Scrum • Grooming • Sprint planning • Standup • Demo • Retrospective • Wins • Improvments
54. Demos and retrospective • Demo all stories that are done • Celebrate wins • Updates on experiments • What can be better?
55. Demo and retrospective Best practices • 2 action items for what should change • Demo only fully complete stories; no partial credit • Prepare upfront – projector, emulators, building your code Anti-patterns • The “What didn’t go well” list repeats i.e. no follow through • The team obsesses over the negatives • “This is not complete, but let me show what I’ve done”
56. Demo and retrospective Best practices • 2 action items for what should change • Demo only fully complete stories; no partial credit • Prepare upfront – projector, emulators, building your code Anti-patterns • Wasting time setting up for the demo
57. If you are not practicing scrum in spirit, you are better off not doing it
58. CD all the way
59. Continuous Delivery is • Minimizing time between “code complete” → “deployable” • deployable ≠ deploy • Based on a solid Continuous Integration (CI) pipeline
60. Continuous Delivery is • Deploying right away • will get us instant feedback • you won’t accumulate debt • Not the best thing for your users • Find your sweet spot
61. Continuous Delivery • Code for a feature is written
62. Continuous Delivery • Code is reviewed and merged • Code below quality bar halts
63. Continuous Delivery • Automated tests are run • Failed code halts
64. Continuous Delivery • Code is deployed to a pre-production user base • Internal users • Playstore alpha/beta testing
65. Continuous Delivery • Code is deployed to production user base • This step can be manual!
66. Continuous Delivery
67. A simpler branch-management model v1.0 v1.1 develop release
68. Protect important branches
69. Merging code • Reviewing code is very important • You don’t want your developer performing tasks that can be outsourced • Find the ‘best’ reviewer – facebook/mention-bot
70. Merging code • SonarQube • Static analysis and lint checks
72. Merging code • Android test results • Make sure you run tests across all your variants! • CodeCov (http://codecov.io) • Ensure code coverage stays up to bar
74. Merging code • Danger (http://danger.systems) • Automate your code review chores • Keep PRs to <500 lines • Good descriptive messages/ticker number for your history
75. # Ensure a descriptive PR body if (github.pr_body == nil || github.pr_body.length < 25) warn(’PR message not descriptive enough (should be >25 chars)') end
76. Merging code • Appetize (http://appetize.io) • Emulator in an iframe on the pull request • Removes the need to compile and install
79. APK storage and deployment • Use an artifact repository • Publish artifacts from either every commit/nightly builds • Bitnami has an AWS image for Artifactory • Automate your playstore updates • fastlane/fastlane provides excellent tools to automate screenshot capture and APK publishing
80. Culture for winning How you think, act and interact
81. Team rules of engagement • Not how a result is to be achieved, but will indicate what is acceptable and what is not • Setting expectations upfront
82. Team rules of engagement Own and drive plan Ex: Always Communicate the deviations/compromises. Never be in a state where someone learns about something, especially bad, from someone else or by chance
83. Team rules of engagement Maintain high quality product Ex: Proactively address app quality issues. Ex: if there’s an outage, check, triage, announce and work towards resolving it
84. Team rules of engagement Be a great team player Ex: if you are coming in late, let people know and mark it on your calendar; if you need to run an errand, mark it on your calendar
85. Goals
86. Goals • Build a new search experience to increase engagement by 10% • Increase ad revenue by showing more ads by 3%
87. Goals • Decrease crashes introduced by my code by 99% • Finish my 7th half marathon
88. Goals • Decrease app startup time to 2000 ms • Refactor query poller and increase test coverage to 100%
89. Objectives and Key Results • Objective: Increase ad revenue by showing more ads by 3% • Key results • Identify slots for ad placement • Devise ad unit to place at each ad slot • Implement the top ad placements to achieve expected revenue increase • Deploy experience to beta users and compute run rate over 1 month Target 70%
90. Meeting etiquette • Everyone needs to derive value • It’s OK to reject meeting invites • Enter with an agenda and goals, step out with action items • Start and end on time
91. Everyone is a leader • Engineering happiness • Bug duty • Feedback duty • Release duty
92. Architect for awesomeness
93. Code is a liability • Value = user revenue; cost = code • Writing, changing and reading code is expensive • Some components stabilize, some don’t • Building on bad code goes into a death spiral Stabilize or kill!
94. https://github.com/erikbern/git-of-theseus
95. Half life of code Amount of time in which half your code is replaced
96. https://github.com/erikbern/git-of-theseus
97. What’s the half-life of the code you are working on now?
98. yahoo-finance ~ 5 months angular ~ 4 months kubernetes~ 5 months httpd ~ 5.4 years git ~ 6 years linux ~ 6.6 years Rapidly evolving components Stable components https://erikbern.com/2016/12/05/the-half-life-of-code.html
99. Component design considerations • Investing time in stabilizing • Write clean, replaceable components • Capture history & knowledge
100. Architecture Documentation Records (ADRs) • Capture significant architectural decisions and the whys • Each significant component has a ADR file in git Ex: adrs/database.MD • Capture the what and the why of the change
101. Example Architecture Documentation Record
102. Do you really need • Fragments over Activityes? • That fancy DI framework? • Annotation with JSON parser? • Optimize what matters • Do not prematurely optimize
103. Prefer robustness over simplicity public class Application { public void onCreate() { WeatherSDK.initialize(true, "Washington DC"); } }
104. Prefer robustness over simplicity public class Application { public void onCreate() { WeatherSDK sdk = new WeatherSDK(); sdk.cacheWeather(true); sdk.trackWeather("Washington DC"); } }
105. Provide a way to accept shared resources public class Application { public void onCreate() { WeatherSDK sdk = new WeatherSDK.Builder() .withOkHttpClient(okHttpClient).build(); sdk.trackWeather("Washington DC"); } }
106. All AsyncTasks share a single thread pool private static final int CORE_POOL_SIZE = Math.max(2, Math.min(CPU_COUNT - 1, 4)); private static final int MAXIMUM_POOL_SIZE = CPU_COUNT * 2 + 1;
107. Actively report errors and crashes back FlurryAgent.onError("API_Resp", "FetchWeatherCallMissingKey: " + key, null); FlurryAgent.onError("GenCrash", "ConvertFtoCFor:" + city, exception); FlurryAgent.onError(”CacheFail", ”LoadFailed:" + key, null);
108. Have feature flags ready Always have a local feature flag buildTypes { debug { buildConfigField ”boolean", "ENABLE_AWESOME_FEATURE_1", "true" } release { buildConfigField ”boolean", "ENABLE_AWESOME_FEATURE_1", "false" } }
109. Have feature flags ready Always have a local feature flag buildTypes { debug { buildConfigField ”boolean", "ENABLE_AWESOME_FEATURE_1", "true" } release { buildConfigField ”boolean", "ENABLE_AWESOME_FEATURE_1", "false" } }
110. Have feature flags ready And a remote feature flag while a feature stabilizes GET /features?versionCode=3.8.0&flavor=android { "ENABLE_AWESOME_FEATURE_1": false }
111. Have feature flags ready And a remote feature flag while a feature stabilizes GET /features?versionCode=3.8.0&flavor=android { "ENABLE_AWESOME_FEATURE_1": false }
112. Feature flags and experiments Don’t hesitate to duplicate code for experiments if (isFeatureEnabled(FeatureManager.Feature.SEARCH_IMPROVEMENTS)) { searchPresenter = new SearchPresenter(); } else { searchPresenter = new OldSearchPresenter(); }
113. Performance Performance is an important feature Measure continuously – from your CI and the field
114. Performance • Memory usage and leaks • Use leakcanary • Make a habit of looking at MAT regularly • Buttery smooth performance • Keep an eye on Android vitals • Find The road to 60 FPS by Jason Sendros
115. Performance - Startup time • Optimize data management • Prefetch and preload • Try to avoid data joins for your initial layout • Optimize code • Lots of knowledge on the Internet • Optimize your UX • Optimize your UX to enable faster startup • Consider using nimbledroid
116. What next? • Give yourself a couple of quarters • Capture what you want to change in goals • Any change you want needs mindset shift, so get people buy-in • These are guidelines – don’t aim for 100% If you want to help Yahoo Finance get there, come talk to me!
117. Thank you! @vikramb0