Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ParaFuzz: Fuzzing Multicore OCaml Programs

ParaFuzz: Fuzzing Multicore OCaml Programs

ParaFuzz is a concurrency testing tool for Multicore OCaml programs that combines property-based testing with grey-box fuzzing applied to parallel programs.

KC Sivaramakrishnan

August 26, 2021
Tweet

More Decks by KC Sivaramakrishnan

Other Decks in Programming

Transcript

  1. • Adds native support for concurrency and parallelism to OCaml

    Multicore OCaml Overlapped execution A B A C B Time
  2. • Adds native support for concurrency and parallelism to OCaml

    Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time
  3. • Adds native support for concurrency and parallelism to OCaml

    Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time Effect Handlers
  4. • Adds native support for concurrency and parallelism to OCaml

    Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time Effect Handlers Domains
  5. • Adds native support for concurrency and parallelism to OCaml

    Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time Effect Handlers Domains using Testing
  6. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination
  7. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination 10
  8. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination happens before 10
  9. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination happens before happens before 10
  10. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination • Logic bugs are more than just detecting data race s ✦ No data races here happens before happens before 10
  11. Testing Parallel Programs • The assertion can fail for a

    particular input and scheduling combination • Logic bugs are more than just detecting data race s ✦ No data races here • Goal — An effective and pragmatic testing technique for the working OCaml programmer. happens before happens before 10
  12. Existing solutions • Testin g ✦ Stress testing — run

    the program over and over again and hope that the assertion is triggere d ✦ Random testing — generate random inputs, and perturb the OS scheduler (somehow) to trigger bugs
  13. Existing solutions • Testin g ✦ Stress testing — run

    the program over and over again and hope that the assertion is triggere d ✦ Random testing — generate random inputs, and perturb the OS scheduler (somehow) to trigger bugs • Model checking — SPIN, TLC model checker s ✦ Strong guarantees, but not practical with limited time budge t ✦ Often works on a model of the program and not directly on the source code
  14. Existing solutions • Testin g ✦ Stress testing — run

    the program over and over again and hope that the assertion is triggere d ✦ Random testing — generate random inputs, and perturb the OS scheduler (somehow) to trigger bugs • Model checking — SPIN, TLC model checker s ✦ Strong guarantees, but not practical with limited time budge t ✦ Often works on a model of the program and not directly on the source code • Formal veri fi catio n ✦ Requires expert knowledge and lots of time and effort
  15. Goal + Approach • Build upon effective pragmatic testing technique

    s ✦ Ignore concurrency for the moment — only input non-determinism
  16. Goal + Approach • Build upon effective pragmatic testing technique

    s ✦ Ignore concurrency for the moment — only input non-determinism • Property-based testin g ✦ Use a generator to generate random inputs to test a functio n ✦ Quick-check
  17. Goal + Approach • Build upon effective pragmatic testing technique

    s ✦ Ignore concurrency for the moment — only input non-determinism • Property-based testin g ✦ Use a generator to generate random inputs to test a functio n ✦ Quick-check • Fuzzin g ✦ Generate random inputs to crash a progra m ✦ AFL — Extremely effective grey-box (coverage-guided) fuzzer
  18. Goal + Approach • Build upon effective pragmatic testing technique

    s ✦ Ignore concurrency for the moment — only input non-determinism • Property-based testin g ✦ Use a generator to generate random inputs to test a functio n ✦ Quick-check • Fuzzin g ✦ Generate random inputs to crash a progra m ✦ AFL — Extremely effective grey-box (coverage-guided) fuzzer • Crowbar = Fuzzing + QuickChec k ✦ Coverage-guided property-fuzzin g ✦ https://github.com/stedolan/crowbar
  19. ParaFuzz • How to control parallel thread scheduling ? ✦

    OS controls thread scheduling in parallel program s ✦ Need to force a buggy schedule Parafuzz = Crowbar (grey-box fuzzing + property-based testing) + Parallelism
  20. Effect handlers • Mock the parallelism API using effect handler

    s • Effect Handler s ✦ Modular and composable basis of non-local control- fl ow mechanism s ✤ Exceptions, async/await, lightweight threads, co-routines, etc . ✦ Structured programming with delimited continuations
  21. Effect Handlers + Fuzzing • Simulate parallel thread scheduler using

    effect handler s ✦ OS thread scheduler → user-level thread schedule r ✦ Retain control over the scheduling decisions
  22. Effect Handlers + Fuzzing • Simulate parallel thread scheduler using

    effect handler s ✦ OS thread scheduler → user-level thread schedule r ✦ Retain control over the scheduling decisions • Fuzzing the schedule r ✦ Yield at every synchronisation point ✤ Synchronisation point — context-switch leads to non-determinis m ✦ Use AFL to pick next thread to run from ready queue
  23. Effect Handlers + Fuzzing • Simulate parallel thread scheduler using

    effect handler s ✦ OS thread scheduler → user-level thread schedule r ✦ Retain control over the scheduling decisions • Fuzzing the schedule r ✦ Yield at every synchronisation point ✤ Synchronisation point — context-switch leads to non-determinis m ✦ Use AFL to pick next thread to run from ready queue • Synchronisation point s ✦ Domain (spawn, join), Atomic (get, set, CAS), Mutex (lock, unlock), Condition variable (wait, notify, broadcast)
  24. ParaFuzz • System layout • Advantage s ✦ Drop-in replacement

    for Multicore OCaml — separate compiler switc h ✦ No false positive s ✦ Deterministic record-and-replay
  25. Future work: Data races • ParaFuzz currently assumes that the

    programs are data-race-free (DRF) ✦ DRF programs in OCaml have SC semantics
  26. Future work: Data races • ParaFuzz currently assumes that the

    programs are data-race-free (DRF) ✦ DRF programs in OCaml have SC semantics • OCaml memory model (PLDI’18) also has a simple operational model for racy program s ✦ Racy reads may return one of a subset of writes performed to a non- atomic location
  27. Future work: Data races • ParaFuzz currently assumes that the

    programs are data-race-free (DRF) ✦ DRF programs in OCaml have SC semantics • OCaml memory model (PLDI’18) also has a simple operational model for racy program s ✦ Racy reads may return one of a subset of writes performed to a non- atomic location • Extend ParaFuzz to racy program s ✦ Use AFL to pick the value that a read should retur n ✦ Force a yield at non-atomic reads and writes
  28. Future work: Data races • ParaFuzz currently assumes that the

    programs are data-race-free (DRF) ✦ DRF programs in OCaml have SC semantics • OCaml memory model (PLDI’18) also has a simple operational model for racy program s ✦ Racy reads may return one of a subset of writes performed to a non- atomic location • Extend ParaFuzz to racy program s ✦ Use AFL to pick the value that a read should retur n ✦ Force a yield at non-atomic reads and writes • Can we make it fast enough for pragmatic use?
  29. Summary • ParaFuzz ✦ property-based fuzzing tool for Parallel OCaml

    program s ✦ Easy to use — drop-in replacement for Multicore OCaml program s ✦ Effective and ef fi cient at fi nding concurrency bug s • Future work — Detecting bugs under data races https://github.com/ocaml-multicore/parafuzz