$30 off During Our Annual Pro Sale. View Details »

Lessons Learnt on a Lean Startup Journey

Elabor8
November 12, 2013

Lessons Learnt on a Lean Startup Journey

A presentation on using Lean Startup to create Open Universities Australia's Open2Study MOOCs platform. Created by Elabor8 consultants James Pulling and Cecilia Waters and Open Universities Australia Product Manager Takis Diakoumis and Agile Lead Erik van Eekelen.

Elabor8

November 12, 2013
Tweet

More Decks by Elabor8

Other Decks in Business

Transcript

  1. 1 Lessons Learnt on a Lean Start Up Journey Using

    Lean Startup methodology to create Open Universities Australia's Open2Study MOOCs platform Takis Diakoumis, Erik Van Eekelen, Cecilia Waters, James Pulling
  2. 2 Outline 1.  Introductions 2.  What on earth is a

    MOOC? 3.  What is Open2Study? 4.  The Lean Startup methodology 5.  Lean Startup Lessons Learned 6.  Questions
  3. 4 Trailer video removed for size. Video can be found

    here: https://www.youtube.com/watch?v=Bw8HkjGQb3U
  4. 9

  5. 10 The Vision 1.  The first Australian MOOC platform 2. 

    Unlike ALL the other MOOC platforms 3.  Offer a more Social & Active Learning environment 4.  The first to offer a combined University Level Education & Professional Vocational courses 5.  Offer courses that are fit for the purpose for the digital space
  6. 11 Module  1   Module  2   Module  3  

    Module  4   10  topics  (with  quiz)   1  assessment   1  video  
  7. 12 Module  1   Module  2   Module  3  

    Module  4   10  topics  (with  quiz)   1  assessment   1  video  
  8. 13 How it all happened 5th Nov 2012 3 weeks

    later Dec 2012 Mar 21st 2013 •  CEO raised need for MOOC •  Team formed for Business Case Definition •  Product Definition •  Elevator pitch •  Personas •  Key user journeys •  Competitor analysis Feb 2013
  9. 14 How it all happened •  Business case endorsed and

    partial funding approved •  Initial project team created 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  10. 15 How it all happened •  Project kicked off • 

    Streams and teams scaled up •  Use case models •  UX vision and wireframes •  Instructors engaged for production 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  11. 16 How it all happened •  Business case approved • 

    Regression tested •  Platform user tested 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  12. 17 How it all happened Go live! 5th Nov 2012

    3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  13. 18 How is it all going? 8361 Initial registrations 112,370

    Enrolments 96% user satisfaction 9.4% industry average 26% our average 57,731 Current registrations
  14. 20 Definition •  "Lean Startup" is a method for developing

    businesses and products first proposed in 2011 by Eric Ries •  The Lean Startup by Eric Ries •  Derived from Lean Manufacturing •  Identifying which activities are value-creating and which are wasteful •  The application of Lean thinking to the process of innovation
  15. 21 ‘A startup is a human institution designed to deliver

    a new product or service under conditions of extreme uncertainty.’ -- Eric Ries --
  16. 22 Minimum Viable Product The question is not "Can this

    product be built?" Instead, the question is: "Should this product be built?"
  17. 23

  18. 24 2007 2008 The Point -  Social platform for groups

    to solve problems -  Very modest traction -  Customers wanted to save money buying in a group 2010 Groupon -  One deal a day -  Deal doesn’t start until enough people buy the voucher -  Vouchers ideally for local businesses Groupon -  Valued at $1.35 billion in April 2010 Pivot example: Groupon’s first pivot
  19. 26 User testing FAIL FAST Fail often MVP must be

    TRULY MINIMAL MEASURE ALL THE TIME Continually tune EVERYONE is an entrepreneur Actionable metrics Take the time to Learn from measurements Rapid releases, Agile & Lean practices Flat mgmt structures Working in a Lean start up presented many opportunities to Learn… Stop wasting people’s time
  20. 27 Lesson 1: Focus the build •  One of our

    main drivers was ‘Time to Market’ – which is important to any Lean Start Up •  This driver made us start building without full or confirmed requirements •  Was this TOO FAST?!? •  More speed and less haste
  21. 28 At first we only had: -  Business case - 

    UC: Discovered -  Wireframes -  Some visual designs We quickly specified: -  UC: bulleted outlines -  Independent leadership (workshop/ decisions) -  Prioritised to create MVP We then: -  UC’s broken down to User Stories -  Prioritised Backlog created (focused on getting priority requirements to the fine grain level) Lesson 1: Focus the build •  We wanted to keep going fast so we had to focus development •  Stop wasting people’s time and only do the things you have to do
  22. 29 Lesson 2: Make MVP, really MVP •  System and

    feature screen test supporting fail fast, fail often in order to learn and adapt quickly •  Validate or invalidate your assumptions •  First release: -  No classroom -  No way to begin or complete any course -  Just enough to register and enrol users •  We could have gone live with less
  23. 31 LEARN: Missing assessments Want to keep moving MEASURE: Student

    feedback BUILD: Assessment model: Open Thursday - Sunday Lesson 3: Tune as often and as quickly as you can
  24. 32 Extension email had to be sent BUILD 2: Assessment

    model: Open Monday – Sunday LEARN: Improved feedback Spike after extension email MEASURE: Student feedback Completion metrics Lesson 3: Tune as often and as quickly as you can
  25. 33 BUILD 3: Assessment model: Open 1st day of week

    – Close Reminder email LEARN: Keep learning and pivoting MEASURE: Student feedback Completion metrics Lesson 3: Tune as often and as quickly as you can
  26. 34 Lesson 4: Everyone is an entrepreneur Luckily we had

    a number of budding entrepreneurs of our own in the team
  27. 37 Lesson 4: Everyone is an entrepreneur •  Extended enrolment

    period •  Enrolments closed on first day of course – no one saw this as an issue as the next course would start in 4 weeks
  28. 38 Lesson 4: Everyone is an entrepreneur •  Extended enrollment

    period •  Inviting friends to the platform was a key feature •  Extend enrollment period to allow more students to join
  29. 39 Lesson 5: Build, and build the ability to measure

    •  Customer surveys showed students wanted more ways to collaborate •  A new connections feature was built to provide scaffolding to connection to connection features •  We had performed workshops and analysis and were well into the next sprint when it was flagged •  ‘Did we have any user stories written to measure user engagement with our new Connections feature?’
  30. 40 Lesson 5: Build, and build the ability to measure

    •  Connections shipped & we had to measure manually •  Results showed uptake was low •  Build + build ability to Measure in order to Learn •  Lesson learnt – all features must have measure stories •  Stakeholders engaged to learn from these measurements
  31. 41 Lesson 6: Actionable insights over vanity metrics Vanity metric:

    •  Business case KPI of 100K enrolments in 2013 •  Vanity metrics make you look good •  Needed to understand more about our students Actionable Accessible Auditable metrics: •  Required to hold entrepreneurs accountable to real metrics, not vanity metrics
  32. 42 Lesson 6: Actionable insights over vanity metrics •  LMS

    data was manipulated in excel pivot tables •  New project to develop and deploy an open source BI Tool Pentaho, on a read only RDS •  Slice & dice data from cohort to student level •  Actionable insights were easily accessed, eg: for course Y the average mark for module 1 assessment was 15%
  33. 43 Elementary, my dear Watson Our deductions again: 1.  Focus

    the build 2.  Make MVP really MVP 3.  Tune as often and as quickly as you can 4.  Everyone is an entrepreneur 5.  Always build, and build the ability to measure 6.  Actionable insights over vanity metrics