The Next 1100 Haskell Programmers

We report on our experience teaching a Haskell-based functional programming course to over 1100 students for two winter terms. The syllabus was organized around selected material from various sources. Throughout the terms, we emphasized correctness through QuickCheck tests and proofs by induction. The submission architecture was coupled with automatic testing, giving students the possibility to correct mistakes before the deadline. To motivate the students, we complemented the weekly assignments with an informal competition and gave away trophies in a award ceremony.

Joint work by Jasmin Blanchette, Lars Hupel, Tobias Nipkow, Lars Noschinsk, Dmitriy Traytel

Presented at the Haskell Symposium 2014, co-located with ICFP, Gothenburg, Sweden.

Transcript

1. The Next 1100 Haskell Programmers
   Jasmin Blanche e Lars Hupel Tobias Nipkow
   Lars Noschinski Dmitriy Traytel
   Technische Universität München
   September 4th, 2014
   

   View Slide

2. 1 Se ng & Syllabus
   2 Exercises & Compe on
   3 Exam
   

   View Slide

3. Curriculum
   Introduc on to Func onal Programming
   ▶ Compulsory course for 2nd year students
   ▶ Computer Science
   ▶ Informa on Systems
   ▶ Science Teaching
   ▶ Elec ve course for
   ▶ Mathema cs
   ▶ Economics
   ▶ ...
   3 / 22
   

   View Slide

4. Why Haskell?
   Haskell has:
   ▶ A large user community
   ▶ Real-world appeal
   ▶ A variety of textbooks
   ▶ QuickCheck
   ... but also:
   ▶ Lazy evalua on
   4 / 22
   

   View Slide

5. Course Outline
   1. Introduc on to FP
   2. Basic Haskell: QuickCheck,
   guarded equa ons, ...
   3. List comprehensions,
   polymorphism, basic type
   classes, pa ern matching
   4. Proof by structural induc on
   on lists
   5. Higher-order func ons,
   λ-abstrac ons, extensionality
   6. Type classes
   7. Algebraic datatypes and
   structural induc on
   8. I/O: files, web, ...
   9. Modules: module syntax, data
   abstrac on, correctness proofs
   10. Case study: Huffman coding
   11. Lazy evalua on and infinite
   lists
   12. Complexity and op miza on
   13. Case study: parser
   combinators
   5 / 22
   

   View Slide

6. Course Outline
   1. Introduc on to FP
   2. Basic Haskell: QuickCheck,
   guarded equa ons, ...
   3. List comprehensions,
   polymorphism, basic type
   classes, pa ern matching
   4. Proof by structural induc on
   on lists
   5. Higher-order func ons,
   λ-abstrac ons, extensionality
   6. Type classes
   7. Algebraic datatypes and
   structural induc on
   8. I/O: files, web, ...
   9. Modules: module syntax, data
   abstrac on, correctness proofs
   10. Case study: Huffman coding
   11. Lazy evalua on and infinite
   lists
   12. Complexity and op miza on
   13. Case study: parser
   combinators
   5 / 22
   

   View Slide

7. Course Outline
   1. Introduc on to FP
   2. Basic Haskell: QuickCheck,
   guarded equa ons, ...
   3. List comprehensions,
   polymorphism, basic type
   classes, pa ern matching
   4. Proof by structural induc on
   on lists
   5. Higher-order func ons,
   λ-abstrac ons, extensionality
   6. Type classes
   7. Algebraic datatypes and
   structural induc on
   8. I/O: files, web, ...
   9. Modules: module syntax, data
   abstrac on, correctness proofs
   10. Case study: Huffman coding
   11. Lazy evalua on and infinite
   lists
   12. Complexity and op miza on
   13. Case study: parser
   combinators
   5 / 22
   

   View Slide

8. Course Outline
   1. Introduc on to FP
   2. Basic Haskell: QuickCheck,
   guarded equa ons, ...
   3. List comprehensions,
   polymorphism, basic type
   classes, pa ern matching
   4. Proof by structural induc on
   on lists
   5. Higher-order func ons,
   λ-abstrac ons, extensionality
   6. Type classes
   7. Algebraic datatypes and
   structural induc on
   8. I/O: files, web, ...
   9. Modules: module syntax, data
   abstrac on, correctness proofs
   10. Case study: Huffman coding
   11. Lazy evalua on and infinite
   lists
   12. Complexity and op miza on
   13. Case study: parser
   combinators
   Image credit: Randall Munroe
   5 / 22
   

   View Slide

9. 1 Se ng & Syllabus
   2 Exercises & Compe on
   3 Exam
   

   View Slide

10. Weekly Assignments
    ... consist of three parts:
    1. Group exercises
    2. Homework
    3. Compe on problem
    7 / 22
    

    View Slide

11. Tutorial Groups
    ▶ 90 minutes
    ▶ Lead by a student TA
    ▶ ∼20–25 groups
    ▶ Up to 24 students per group
    ▶ Usually ∼10 show up
    8 / 22
    

    View Slide

12. Homework
    ▶ 3–4 problems
    ▶ Time needed to solve problems greatly varies
    ▶ Students can upload their submissions via SSH
    ▶ Cronjob runs a test suite, mails results to students
    ▶ Students can re-upload arbitrarily o en before deadline
    ▶ Graded by student TAs
    9 / 22
    

    View Slide

13. Test Suite
    ▶ Tests hand-wri en by us
    ▶ Checks against a specifica on and a reference implementa on
    ▶ Tests and expected results are secret, failed inputs are reported
    ▶ Problem: excep ons and non-termina on
    ▶ Output of within combinator poten ally confusing for students
    ▶ Excep ons captured with Control.Spoon
    ▶ Future work: automated style checks with HLint?
    10 / 22
    

    View Slide

14. Test Suite
    Lessons Learned
    ▶ Ensure that tests are not vacuous (defaul ng to Unit!)
    ▶ Difficult to explain to students
    ▶ Instan ate polymorphic proper es with small types
    ▶ Write generators by hand, tuned to the problem at hand
    ▶ Tune parameters (maxSize, maxSuccess, ...)
    ▶ Do not trust “Detec ve QuickCheck” alone
    11 / 22
    

    View Slide

15. Image credit:
    Roman Cheplyaka
    

    View Slide

16. Compe on
    1. Some mes a dis nguished homework exercise,
    some mes op onal
    2. Fix a ranking criterion
    ▶ Brevity
    ▶ Efficiency
    ▶ Ar s c value
    3. Rank all student submissions
    4. Comment and publish good solu ons
    5. At the end of the year: Award ceremony
    13 / 22
    

    View Slide

17. Compe on
    1. Some mes a dis nguished homework exercise,
    some mes op onal
    2. Fix a ranking criterion
    ▶ Brevity
    ▶ Efficiency
    ▶ Ar s c value
    3. Rank all student submissions
    4. Comment and publish good solu ons
    5. At the end of the year: Award ceremony
    ... (mostly) run by Jasmin “MC Hammer” Blanche e
    13 / 22
    

    View Slide

18. Compe on
    Mo va on
    The programming contests a ract a substan al frac on of the most
    talented students that we have ... It is our role to discover these
    talents and make them shine. – Piotr Rudnicki
    14 / 22
    

    View Slide

19. Compe on
    Problem from week 2
    Task: Write a func on [Char] -> [[Char]] returning all dis nct
    permuta ons of the input list in reverse alphabe c order.
    Criterion: Token count, then efficiency.
    15 / 22
    

    View Slide

20. Compe on
    Problem from week 2
    Task: Write a func on [Char] -> [[Char]] returning all dis nct
    permuta ons of the input list in reverse alphabe c order.
    Criterion: Token count, then efficiency.
    Winning entry
    perms chars = reverse $ sort
    [c : str |
    c str ] ‘max‘ [””]
    15 / 22
    

    View Slide

21. Compe on
    Problem from weeks 11–12
    Task: Write an AI player for Othello/Reversi
    Criterion: Ranking in a tournament between all contestants
    16 / 22
    

    View Slide

22. Compe on
    Results
    ▶ Many students spent a lot of me on solving the problems
    ▶ ... some even ignored all the other exercises
    ▶ Ranking the solu ons takes a lot of me
    ▶ Vehicle to teach advanced topics outside of the syllabus
    ▶ Fosters library explora on
    ▶ Rewarding for us: recrui ng students
    17 / 22
    

    View Slide

23. Compe on
    Results
    ▶ Many students spent a lot of me on solving the problems
    ▶ ... some even ignored all the other exercises
    ▶ Ranking the solu ons takes a lot of me
    ▶ Vehicle to teach advanced topics outside of the syllabus
    ▶ Fosters library explora on
    ▶ Rewarding for us: recrui ng students
    ▶ CYP – a simple proof checker (Durner)
    ▶ Primi ve corecursion for Isabelle (Panny)
    ▶ Waldmeister for Sledgehammer (Steckermaier)
    ▶ Complexity analysis of a small func onal language (Wimmer)
    ▶ Func onal congruence closure (Franze)
    ▶ ... and >5 future TAs
    17 / 22
    

    View Slide

24. 1 Se ng & Syllabus
    2 Exercises & Compe on
    3 Exam
    

    View Slide

25. Exam Results in 2012
    0 50 100 150 200 250
    0
    10
    20
    30
    40
    50
    Homework points
    Examina on points
    Linear regression
    Passing score
    19 / 22
    

    View Slide

26. Exam Results in 2013
    0 50 100 150 200 250
    0
    10
    20
    30
    40
    Homework points
    Examina on points
    Linear regression
    Passing score
    20 / 22
    

    View Slide

27. Exam
    What works well?
    ▶ Induc on and equa onal reasoning
    ▶ List comprehensions
    ▶ I/O
    What does not work so well?
    ▶ Type inference
    ▶ Higher-order func ons
    ▶ Evalua on order
    21 / 22
    

    View Slide

28. Q & A
    

    View Slide