Lessons Learnt on a Lean Startup Journey

356c244bf4c7a1269e6f2e02705438eb?s=47 Elabor8
November 12, 2013

Lessons Learnt on a Lean Startup Journey

A presentation on using Lean Startup to create Open Universities Australia's Open2Study MOOCs platform. Created by Elabor8 consultants James Pulling and Cecilia Waters and Open Universities Australia Product Manager Takis Diakoumis and Agile Lead Erik van Eekelen.

356c244bf4c7a1269e6f2e02705438eb?s=128

Elabor8

November 12, 2013
Tweet

Transcript

  1. 1 Lessons Learnt on a Lean Start Up Journey Using

    Lean Startup methodology to create Open Universities Australia's Open2Study MOOCs platform Takis Diakoumis, Erik Van Eekelen, Cecilia Waters, James Pulling
  2. 2 Outline 1.  Introductions 2.  What on earth is a

    MOOC? 3.  What is Open2Study? 4.  The Lean Startup methodology 5.  Lean Startup Lessons Learned 6.  Questions
  3. 3 1. Introductions

  4. 4 Trailer video removed for size. Video can be found

    here: https://www.youtube.com/watch?v=Bw8HkjGQb3U
  5. 5 2. What on earth is a MOOC?

  6. 6 What is a MOOC?

  7. 7 3. What is Open2Study?

  8. 8 This thing is big!

  9. 9

  10. 10 The Vision 1.  The first Australian MOOC platform 2. 

    Unlike ALL the other MOOC platforms 3.  Offer a more Social & Active Learning environment 4.  The first to offer a combined University Level Education & Professional Vocational courses 5.  Offer courses that are fit for the purpose for the digital space
  11. 11 Module  1   Module  2   Module  3  

    Module  4   10  topics  (with  quiz)   1  assessment   1  video  
  12. 12 Module  1   Module  2   Module  3  

    Module  4   10  topics  (with  quiz)   1  assessment   1  video  
  13. 13 How it all happened 5th Nov 2012 3 weeks

    later Dec 2012 Mar 21st 2013 •  CEO raised need for MOOC •  Team formed for Business Case Definition •  Product Definition •  Elevator pitch •  Personas •  Key user journeys •  Competitor analysis Feb 2013
  14. 14 How it all happened •  Business case endorsed and

    partial funding approved •  Initial project team created 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  15. 15 How it all happened •  Project kicked off • 

    Streams and teams scaled up •  Use case models •  UX vision and wireframes •  Instructors engaged for production 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  16. 16 How it all happened •  Business case approved • 

    Regression tested •  Platform user tested 5th Nov 2012 3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  17. 17 How it all happened Go live! 5th Nov 2012

    3 weeks later Dec 2012 Mar 21st 2013 Feb 2013
  18. 18 How is it all going? 8361 Initial registrations 112,370

    Enrolments 96% user satisfaction 9.4% industry average 26% our average 57,731 Current registrations
  19. 19 4. The Lean Start Up methodology

  20. 20 Definition •  "Lean Startup" is a method for developing

    businesses and products first proposed in 2011 by Eric Ries •  The Lean Startup by Eric Ries •  Derived from Lean Manufacturing •  Identifying which activities are value-creating and which are wasteful •  The application of Lean thinking to the process of innovation
  21. 21 ‘A startup is a human institution designed to deliver

    a new product or service under conditions of extreme uncertainty.’ -- Eric Ries --
  22. 22 Minimum Viable Product The question is not "Can this

    product be built?" Instead, the question is: "Should this product be built?"
  23. 23

  24. 24 2007 2008 The Point -  Social platform for groups

    to solve problems -  Very modest traction -  Customers wanted to save money buying in a group 2010 Groupon -  One deal a day -  Deal doesn’t start until enough people buy the voucher -  Vouchers ideally for local businesses Groupon -  Valued at $1.35 billion in April 2010 Pivot example: Groupon’s first pivot
  25. 25 5. Lean Startup Lessons learned

  26. 26 User testing FAIL FAST Fail often MVP must be

    TRULY MINIMAL MEASURE ALL THE TIME Continually tune EVERYONE is an entrepreneur Actionable metrics Take the time to Learn from measurements Rapid releases, Agile & Lean practices Flat mgmt structures Working in a Lean start up presented many opportunities to Learn… Stop wasting people’s time
  27. 27 Lesson 1: Focus the build •  One of our

    main drivers was ‘Time to Market’ – which is important to any Lean Start Up •  This driver made us start building without full or confirmed requirements •  Was this TOO FAST?!? •  More speed and less haste
  28. 28 At first we only had: -  Business case - 

    UC: Discovered -  Wireframes -  Some visual designs We quickly specified: -  UC: bulleted outlines -  Independent leadership (workshop/ decisions) -  Prioritised to create MVP We then: -  UC’s broken down to User Stories -  Prioritised Backlog created (focused on getting priority requirements to the fine grain level) Lesson 1: Focus the build •  We wanted to keep going fast so we had to focus development •  Stop wasting people’s time and only do the things you have to do
  29. 29 Lesson 2: Make MVP, really MVP •  System and

    feature screen test supporting fail fast, fail often in order to learn and adapt quickly •  Validate or invalidate your assumptions •  First release: -  No classroom -  No way to begin or complete any course -  Just enough to register and enrol users •  We could have gone live with less
  30. 30 Lesson 2: Make MVP, really MVP

  31. 31 LEARN: Missing assessments Want to keep moving MEASURE: Student

    feedback BUILD: Assessment model: Open Thursday - Sunday Lesson 3: Tune as often and as quickly as you can
  32. 32 Extension email had to be sent BUILD 2: Assessment

    model: Open Monday – Sunday LEARN: Improved feedback Spike after extension email MEASURE: Student feedback Completion metrics Lesson 3: Tune as often and as quickly as you can
  33. 33 BUILD 3: Assessment model: Open 1st day of week

    – Close Reminder email LEARN: Keep learning and pivoting MEASURE: Student feedback Completion metrics Lesson 3: Tune as often and as quickly as you can
  34. 34 Lesson 4: Everyone is an entrepreneur Luckily we had

    a number of budding entrepreneurs of our own in the team
  35. 35 Lesson 4: Everyone is an entrepreneur

  36. 36 Lesson 4: Everyone is an entrepreneur

  37. 37 Lesson 4: Everyone is an entrepreneur •  Extended enrolment

    period •  Enrolments closed on first day of course – no one saw this as an issue as the next course would start in 4 weeks
  38. 38 Lesson 4: Everyone is an entrepreneur •  Extended enrollment

    period •  Inviting friends to the platform was a key feature •  Extend enrollment period to allow more students to join
  39. 39 Lesson 5: Build, and build the ability to measure

    •  Customer surveys showed students wanted more ways to collaborate •  A new connections feature was built to provide scaffolding to connection to connection features •  We had performed workshops and analysis and were well into the next sprint when it was flagged •  ‘Did we have any user stories written to measure user engagement with our new Connections feature?’
  40. 40 Lesson 5: Build, and build the ability to measure

    •  Connections shipped & we had to measure manually •  Results showed uptake was low •  Build + build ability to Measure in order to Learn •  Lesson learnt – all features must have measure stories •  Stakeholders engaged to learn from these measurements
  41. 41 Lesson 6: Actionable insights over vanity metrics Vanity metric:

    •  Business case KPI of 100K enrolments in 2013 •  Vanity metrics make you look good •  Needed to understand more about our students Actionable Accessible Auditable metrics: •  Required to hold entrepreneurs accountable to real metrics, not vanity metrics
  42. 42 Lesson 6: Actionable insights over vanity metrics •  LMS

    data was manipulated in excel pivot tables •  New project to develop and deploy an open source BI Tool Pentaho, on a read only RDS •  Slice & dice data from cohort to student level •  Actionable insights were easily accessed, eg: for course Y the average mark for module 1 assessment was 15%
  43. 43 Elementary, my dear Watson Our deductions again: 1.  Focus

    the build 2.  Make MVP really MVP 3.  Tune as often and as quickly as you can 4.  Everyone is an entrepreneur 5.  Always build, and build the ability to measure 6.  Actionable insights over vanity metrics
  44. 44 Open2StudyEduca@on   Open2Study   @open2study   Open2Study   @elabor8

      Elabor8   6. Questions?