Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lessons Learnt on a Lean Startup Journey

Elabor8
November 12, 2013

Lessons Learnt on a Lean Startup Journey

A presentation on using Lean Startup to create Open Universities Australia's Open2Study MOOCs platform. Created by Elabor8 consultants James Pulling and Cecilia Waters and Open Universities Australia Product Manager Takis Diakoumis and Agile Lead Erik van Eekelen.

Elabor8

November 12, 2013
Tweet

More Decks by Elabor8

Other Decks in Business

Transcript

  1. 1
    Lessons Learnt on a Lean
    Start Up Journey
    Using Lean Startup methodology to create Open
    Universities Australia's Open2Study MOOCs platform
    Takis Diakoumis, Erik Van Eekelen, Cecilia Waters, James Pulling

    View full-size slide

  2. 2
    Outline
    1.  Introductions
    2.  What on earth is a MOOC?
    3.  What is Open2Study?
    4.  The Lean Startup methodology
    5.  Lean Startup Lessons Learned
    6.  Questions

    View full-size slide

  3. 3
    1. Introductions

    View full-size slide

  4. 4
    Trailer video removed for size. Video can be found
    here:
    https://www.youtube.com/watch?v=Bw8HkjGQb3U

    View full-size slide

  5. 5
    2. What on earth is a MOOC?

    View full-size slide

  6. 6
    What is a MOOC?

    View full-size slide

  7. 7
    3. What is Open2Study?

    View full-size slide

  8. 8
    This thing is big!

    View full-size slide

  9. 10
    The Vision
    1.  The first Australian MOOC platform
    2.  Unlike ALL the other MOOC platforms
    3.  Offer a more Social & Active Learning environment
    4.  The first to offer a combined University Level
    Education & Professional Vocational courses
    5.  Offer courses that are fit for the purpose for the
    digital space

    View full-size slide

  10. 11
    Module  1   Module  2  
    Module  3   Module  4  
    10  topics  (with  quiz)  
    1  assessment  
    1  video  

    View full-size slide

  11. 12
    Module  1   Module  2  
    Module  3   Module  4  
    10  topics  (with  quiz)  
    1  assessment  
    1  video  

    View full-size slide

  12. 13
    How it all happened
    5th
    Nov
    2012
    3
    weeks
    later
    Dec
    2012
    Mar
    21st
    2013
    •  CEO raised need for MOOC
    •  Team formed for Business Case Definition
    •  Product Definition
    •  Elevator pitch
    •  Personas
    •  Key user journeys
    •  Competitor analysis
    Feb
    2013

    View full-size slide

  13. 14
    How it all happened
    •  Business case endorsed and partial
    funding approved
    •  Initial project team created
    5th
    Nov
    2012
    3
    weeks
    later
    Dec
    2012
    Mar
    21st
    2013
    Feb
    2013

    View full-size slide

  14. 15
    How it all happened
    •  Project kicked off
    •  Streams and teams scaled up
    •  Use case models
    •  UX vision and wireframes
    •  Instructors engaged for production
    5th
    Nov
    2012
    3
    weeks
    later
    Dec
    2012
    Mar
    21st
    2013
    Feb
    2013

    View full-size slide

  15. 16
    How it all happened
    •  Business case approved
    •  Regression tested
    •  Platform user tested
    5th
    Nov
    2012
    3
    weeks
    later
    Dec
    2012
    Mar
    21st
    2013
    Feb
    2013

    View full-size slide

  16. 17
    How it all happened
    Go live!
    5th
    Nov
    2012
    3
    weeks
    later
    Dec
    2012
    Mar
    21st
    2013
    Feb
    2013

    View full-size slide

  17. 18
    How is it all going?
    8361
    Initial
    registrations
    112,370
    Enrolments
    96%
    user satisfaction
    9.4% industry average
    26% our average
    57,731
    Current
    registrations

    View full-size slide

  18. 19
    4. The Lean Start Up methodology

    View full-size slide

  19. 20
    Definition
    •  "Lean Startup" is a method for developing
    businesses and products first proposed in 2011 by
    Eric Ries
    •  The Lean Startup by Eric Ries
    •  Derived from Lean Manufacturing
    •  Identifying which activities are value-creating and
    which are wasteful
    •  The application of Lean thinking to the process of
    innovation

    View full-size slide

  20. 21
    ‘A startup is a human institution designed to
    deliver a new product or service
    under conditions of extreme uncertainty.’
    -- Eric Ries --

    View full-size slide

  21. 22
    Minimum
    Viable
    Product
    The question is not "Can this product
    be built?"
    Instead, the question is:
    "Should
    this product be built?"

    View full-size slide

  22. 24
    2007 2008
    The Point
    -  Social platform for
    groups to solve
    problems
    -  Very modest traction
    -  Customers wanted to
    save money buying in a
    group
    2010
    Groupon
    -  One deal a day
    -  Deal doesn’t start until
    enough people buy the
    voucher
    -  Vouchers ideally for
    local businesses
    Groupon
    -  Valued at $1.35
    billion in April
    2010
    Pivot example: Groupon’s first pivot

    View full-size slide

  23. 25
    5. Lean Startup Lessons learned

    View full-size slide

  24. 26
    User testing
    FAIL FAST
    Fail often
    MVP must
    be TRULY
    MINIMAL
    MEASURE ALL
    THE TIME
    Continually tune
    EVERYONE
    is an
    entrepreneur
    Actionable
    metrics
    Take the time to
    Learn from
    measurements
    Rapid
    releases,
    Agile &
    Lean
    practices
    Flat mgmt
    structures
    Working in a Lean start up presented many
    opportunities to Learn…
    Stop wasting
    people’s time

    View full-size slide

  25. 27
    Lesson 1: Focus the build
    •  One of our main drivers was ‘Time to Market’ –
    which is important to any Lean Start Up
    •  This driver made us start building without full or
    confirmed requirements
    •  Was this TOO FAST?!?
    •  More speed and less haste

    View full-size slide

  26. 28
    At first we only had:
    -  Business case
    -  UC: Discovered
    -  Wireframes
    -  Some visual designs
    We quickly specified:
    -  UC: bulleted outlines
    -  Independent
    leadership (workshop/
    decisions)
    -  Prioritised to create
    MVP
    We then:
    -  UC’s broken down to
    User Stories
    -  Prioritised Backlog
    created (focused on
    getting priority
    requirements to the
    fine grain level)
    Lesson 1: Focus the build
    •  We wanted to keep going fast so we had to focus
    development
    •  Stop wasting people’s time and only do the things
    you have to do

    View full-size slide

  27. 29
    Lesson 2: Make MVP, really MVP
    •  System and feature screen test supporting fail fast,
    fail often in order to learn and adapt quickly
    •  Validate or invalidate your assumptions
    •  First release:
    -  No classroom
    -  No way to begin or complete any course
    -  Just enough to register and enrol users
    •  We could have gone live with less

    View full-size slide

  28. 30
    Lesson 2: Make MVP, really MVP

    View full-size slide

  29. 31
    LEARN:
    Missing assessments
    Want to keep moving
    MEASURE:
    Student feedback
    BUILD:
    Assessment model:
    Open Thursday -
    Sunday
    Lesson 3: Tune as often and as quickly as you can

    View full-size slide

  30. 32
    Extension
    email had
    to be sent
    BUILD 2:
    Assessment model:
    Open Monday – Sunday
    LEARN:
    Improved feedback
    Spike after extension
    email
    MEASURE:
    Student feedback
    Completion metrics
    Lesson 3: Tune as often and as quickly as you can

    View full-size slide

  31. 33
    BUILD 3:
    Assessment model:
    Open 1st day of week –
    Close
    Reminder email
    LEARN:
    Keep learning and
    pivoting
    MEASURE:
    Student feedback
    Completion metrics
    Lesson 3: Tune as often and as quickly as you can

    View full-size slide

  32. 34
    Lesson 4: Everyone is an entrepreneur
    Luckily we had a number of
    budding entrepreneurs of
    our own in the team

    View full-size slide

  33. 35
    Lesson 4: Everyone is an entrepreneur

    View full-size slide

  34. 36
    Lesson 4: Everyone is an entrepreneur

    View full-size slide

  35. 37
    Lesson 4: Everyone is an entrepreneur
    •  Extended enrolment period
    •  Enrolments closed on first day of course – no one
    saw this as an issue as the next course would start in
    4 weeks

    View full-size slide

  36. 38
    Lesson 4: Everyone is an entrepreneur
    •  Extended enrollment period
    •  Inviting friends to the platform was a key feature
    •  Extend enrollment period to allow more students to
    join

    View full-size slide

  37. 39
    Lesson 5: Build, and build the ability to measure
    •  Customer surveys showed students wanted more
    ways to collaborate
    •  A new connections feature was built to provide
    scaffolding to connection to connection features
    •  We had performed workshops and analysis and were
    well into the next sprint when it was flagged
    •  ‘Did we have any user stories written to measure user
    engagement with our new Connections feature?’

    View full-size slide

  38. 40
    Lesson 5: Build, and build the ability to measure
    •  Connections shipped & we had to measure manually
    •  Results showed uptake was low
    •  Build + build ability to Measure in order to Learn
    •  Lesson learnt – all features must have measure
    stories
    •  Stakeholders engaged to learn from these
    measurements

    View full-size slide

  39. 41
    Lesson 6: Actionable insights over vanity metrics
    Vanity metric:
    •  Business case KPI of 100K enrolments in 2013
    •  Vanity metrics make you look good
    •  Needed to understand more about our students
    Actionable Accessible Auditable metrics:
    •  Required to hold entrepreneurs accountable to real
    metrics, not vanity metrics

    View full-size slide

  40. 42
    Lesson 6: Actionable insights over vanity metrics
    •  LMS data was manipulated in excel pivot tables
    •  New project to develop and deploy an open source BI
    Tool Pentaho, on a read only RDS
    •  Slice & dice data from cohort to student level
    •  Actionable insights were easily accessed, eg: for
    course Y the average mark for module 1 assessment
    was 15%

    View full-size slide

  41. 43
    Elementary, my dear Watson
    Our deductions again:
    1.  Focus the build
    2.  Make MVP really MVP
    3.  Tune as often and as quickly as you can
    4.  Everyone is an entrepreneur
    5.  Always build, and build the ability to measure
    6.  Actionable insights over vanity metrics

    View full-size slide

  42. 44
    Open2StudyEduca@on  
    Open2Study  
    @open2study  
    Open2Study  
    @elabor8  
    Elabor8  
    6. Questions?

    View full-size slide