Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JUGNsk Meetup #11. Александр Филатов: "Java PathFinder: летим на Марс без багов и дедлоков."

0605d4091b5f5ab4da0cb1e921aa3f57?s=47 jugnsk
October 29, 2019

JUGNsk Meetup #11. Александр Филатов: "Java PathFinder: летим на Марс без багов и дедлоков."

В современном мире разработчикам сложно избежать темы многопоточности. Необходимо, чтобы приложения работали быстро, с минимальной задержкой, задействовали все доступные вычислительные ядра (не забудьте про масштабируемость!), использовали минимум оперативной памяти и, конечно же, делали именно то, что нужно. К сожалению, даже продвинутые методы тестирования многопоточных программ дают слишком ненадежный результат -- всё зависает обязательно на продакшн сервере, только под большой нагрузкой и наверняка перед самым Вашим отъездом в отпуск.

В докладе мы зададимся вопросом: как проверить, что ваше Java приложение работает при любом, даже самом неожиданном сценарии конкурентного выполнения -- не зависает, не приводит к некорректным результатам и не бросает неожиданные исключения? Оказывается, не обязательно быть специалистом по верификации многопоточных алгоритмов, чтобы использовать продвинутые инструменты анализа паралелльных систем и успешно находить ошибки в коде. В ходе выступления на примерах будет показано, что используя Java PathFinder (https://github.com/javapathfinder) — специальную JVM, разработанную в NASA - разработчик может очень быстро находить довольно нетривиальные ошибки, проявляющиеся только при конкурентном исполнении.

0605d4091b5f5ab4da0cb1e921aa3f57?s=128

jugnsk

October 29, 2019
Tweet

Transcript

  1. Java Path Finder: летим на Марс без багов и дедлоков

    1 Александр Филатов, Huawei
  2. О себе 2 2014 - 2019: разработчик в Excelsior JET

    (JVM с AOT компиляцией) Специализация: сборка мусора, алгоритмы синхронизации 2019 - сейчас: разработчик в Huawei Novosibirsk Research Center Специализация: Java concurrency
  3. Concurrency 3 Parallelism

  4. Concurrency 4 Parallelism

  5. Concurrency Threads Mutable shared memory 5

  6. Concurrency Threads Mutable shared memory data races 6 problems

  7. Concurrency Threads Mutable shared memory Mutexes Semaphores Monitors Latches Barriers

    data races 7 problems Concurrent data structures
  8. Concurrency Threads Mutable shared memory Mutexes Semaphores Monitors Latches Barriers

    data races deadlocks livelocks priority inversion ABA problem memory model constraints 8 problems Concurrent data structures
  9. Concurrency Threads Mutable shared memory Mutexes Semaphores Monitors Latches Barriers

    data races deadlocks livelocks priority inversion ABA problem memory model constraints 9 problems Concurrent data structures Алексей Шипилев: Close Encounters of The Java Memory Model Kind https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/ https://www.youtube.com/watch?v=C6b_dFtujKo
  10. Concurrency Threads Mutable shared memory Mutexes Semaphores Monitors Latches Barriers

    data races deadlocks livelocks priority inversion ABA problem memory model constraints 10 problems Concurrent data structures СЛОЖНА :(
  11. Concurrency Threads Mutable shared memory Mutexes Semaphores Monitors Latches Barriers

    data races deadlocks livelocks priority inversion ABA problem memory model constraints 11 problems Concurrent data structures Плохо тестируется Плохо отлаживается
  12. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). 12

  13. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } 13
  14. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } 14
  15. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } 15
  16. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 16
  17. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 17
  18. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 18
  19. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 19
  20. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 20
  21. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 21
  22. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 22
  23. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 23 W1: x = 0, y = 0;
  24. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 24 W1: x = 0, y = 0;
  25. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 25 W1: x = 0, y = 0;
  26. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 26 W1: x = 0, y = 0;
  27. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 27 W1: x = 0, y = 0;
  28. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } 28 W1: x = 0, y = 0; W2: x = 1, y = 1;
  29. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 29
  30. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 30
  31. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 31
  32. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 32
  33. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 33
  34. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 34
  35. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 35
  36. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 36
  37. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 37 W2: x = 0, y = 0;
  38. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 38 W2: x = 0, y = 0;
  39. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 39 W2: x = 0, y = 0; W1: x = 0, y = 0;
  40. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 40 W2: x = 0, y = 0; W1: x = 0, y = 0;
  41. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 1, y = 1; 41 W2: x = 0, y = 0; W1: x = 0, y = 0; W1: x = 0, y = 0; W2: x = 0, y = 0;
  42. Многопоточность: недетерминизм Много допустимых сценариев исполнения (a.k.a. execution trace). private

    static class Worker1 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W1: x = " + copy_x + ", y = " + copy_y + "; "); } } private static class Worker2 extends Thread { public void run() { int copy_x = x; int copy_y = y; x++; y++; log("W2: x = " + copy_x + ", y = " + copy_y + "; "); } } private static volatile int x = 0; private static volatile int y = 0; public static void main(String[] arg) throws Exception { Worker1 worker1 = new Worker1(); Worker2 worker2 = new Worker2(); worker1.start(); worker2.start(); worker1.join(); worker2.join(); } W1: x = 0, y = 0; W2: x = 0, y = 0; W1: x = 0, y = 0; W2: x = 0, y = 1; W1: x = 0, y = 0; W2: x = 1, y = 0; W1: x = 0, y = 0; W2: x = 1, y = 1; W1: x = 0, y = 1; W2: x = 0, y = 0; W1: x = 1, y = 0; W2: x = 0, y = 0; W1: x = 1, y = 1; W2: x = 0, y = 0; 42
  43. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    43 Thread 2 int copy_x = x; x++;
  44. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Thread 2 int copy_x = x; x++; Start 44
  45. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 45 T1: int copy_x = x; Thread 2 int copy_x = x; x++;
  46. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 46 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++;
  47. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 47 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++;
  48. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 48 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++;
  49. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 49 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++;
  50. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 50 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++;
  51. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 51 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: x++;
  52. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 52 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: x++;
  53. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 53 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: x++; T1: int copy_x = x;
  54. Перебор достижимых состояний Thread 1 int copy_x = x; x++;

    Start 54 T1: int copy_x = x; T1: int copy_x = x; T2: int copy_x = x; Thread 2 int copy_x = x; x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T1: int copy_x = x; T2: int copy_x = x; T1: x++; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: int copy_x = x; T2: int copy_x = x; T2: x++; T1: x++; T1: int copy_x = x;
  55. Перебор достижимых состояний Строим дерево допустимых сценариев выполнения 55

  56. Перебор достижимых состояний Строим дерево допустимых сценариев выполнения Каждый лист

    дерева содержит допустимую трассу 56
  57. Перебор достижимых состояний Строим дерево допустимых сценариев выполнения Каждый лист

    дерева содержит допустимую трассу Для конкретной трассы легко проверить необходимые свойства: assert-ы исключения deadlock-и 57
  58. Перебор достижимых состояний Строим дерево допустимых сценариев выполнения Каждый лист

    дерева содержит допустимую трассу Для конкретной трассы легко проверить необходимые свойства: assert-ы исключения deadlock-и Если во всех листьях все хорошо => программа принципиально не может сломаться 58
  59. Перебор достижимых состояний Строим дерево допустимых сценариев выполнения Каждый лист

    дерева содержит допустимую трассу Для конкретной трассы легко проверить необходимые свойства: assert-ы исключения deadlock-и Если во всех листьях все хорошо => программа принципиально не может сломаться 59
  60. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться А сколько их? 60
  61. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться А сколько их? 61
  62. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться А сколько их? Очень много. 62
  63. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться 63
  64. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться Horror story of the day: Java Memory Model is not sequentially consistent 64
  65. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться Horror story of the day: Java Memory Model is not sequentially consistent JLS 17.4.3: “If we were to use sequential consistency as our memory model, many of the compiler and processor optimizations that we have discussed would be illegal.” 65
  66. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться 66
  67. Перебор достижимых состояний Если во всех листьях все хорошо =>

    программа принципиально не может сломаться Если существует трасса с ошибочным поведением => программа некорректна. И мы знаем как воспроизвести ошибку. 67
  68. Перебор достижимых состояний Если существует трасса с ошибочным поведением =>

    программа некорректна. И мы знаем как воспроизвести ошибку. Тестирование 2.0 68
  69. Проверка моделей (software model checking) Allen Emerson, E.; Clarke, Edmund

    M. (1980) "Characterizing correctness properties of parallel programs using fixpoints" Turing Award 2007: Clarke, Emerson, Sifakis “for their seminal work founding and developing the field of model checking” Известные инструменты: SPIN (verification of the kernel’s sleep and wakeup primitives in Plan 9) TLA+ http://research.microsoft.com/en-us/um/people/lamport/tla/formal-methods-amazon.pdf 69
  70. Проверка моделей (software model checking) Преимущества: Анализ = обход графа

    70
  71. Проверка моделей (software model checking) Преимущества: Анализ = обход графа

    Способны построить контрпример 71
  72. Проверка моделей (software model checking) Преимущества: Анализ = обход графа

    Способны построить контрпример Мощнее железо 一 быстрее анализ 72
  73. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    73
  74. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    “Птичьи языки” для написания моделей 74
  75. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    “Птичьи языки” для написания моделей 75 https://github.com/elastic/elasticsearch-formal-models/blob/master/ZenWithTerms/tla/ZenWithTerms.tla
  76. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    “Птичьи языки” для написания моделей “Академичность” 76
  77. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    “Птичьи языки” для написания моделей “Академичность” 77 In logic, linear temporal logic or linear-time temporal logic[1][2] (LTL) is a modal temporal logic with modalities referring to time. In LTL, one can encode formulae about the future of paths, e.g., a condition will eventually be true, a condition will be true until another fact becomes true, etc. https://en.wikipedia.org/wiki/Linear_temporal_logic
  78. Проверка моделей (software model checking) Недостатки: Комбинаторный взрыв числа состояний

    “Птичьи языки” для написания моделей “Академичность” 78 www.patricktrentin.com/teaching/fm2018/lesson04/lesson04.pdf
  79. Java Path Finder “Extensible model checking framework for Java bytecode

    programs” Разработан в НАСА (Ames Research Center) Apache-2.0 license: https://github.com/javapathfinder/jpf-core По сути 一 исследовательская JVM, написанная на Java 79
  80. Multiple producer multiple consumer intrusive queue public abstract class Queue

    { public static class Node { public Queue.Node next; } public abstract void enqueue(Node element); // null if empty public abstract Node dequeue(); } 80
  81. Baseline implementation public class BaselineQueue extends Queue { private final

    LinkedList<Queue.Node> impl = new LinkedList<>(); @Override public synchronized void enqueue(Queue.Node element) { impl.add(element); } @Override public synchronized Queue.Node dequeue() { return impl.poll(); } } 81
  82. Baseline implementation public class BaselineQueue extends Queue { private final

    LinkedList<Queue.Node> impl = new LinkedList<>(); @Override public synchronized void enqueue(Queue.Node element) { impl.add(element); } @Override public synchronized Queue.Node dequeue() { return impl.poll(); } } Это было просто :) Прокачка памяти Плохая масштабируемость 82
  83. next next next next next Head Tail 83

  84. next next next next next Head Tail enqueue() 84

  85. next next next next next Head Tail enqueue() 85

  86. next next next next next Head Tail enqueue() next 86

  87. next next next next next Head Tail enqueue() next 87

  88. next next next next next Head Tail next 88

  89. next next next next next Head Tail next dequeue() 89

  90. next next next next next Head Tail next dequeue() 90

  91. next next next next Head Tail next dequeue() 91

  92. next next next next Head Tail next dequeue() 92

  93. next next next next Head Tail next dequeue() 93

  94. next next next next Head Tail next dequeue() 94

  95. next next next next Head Tail next 95

  96. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); 96
  97. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); 97
  98. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 98
  99. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 99
  100. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 100
  101. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 101
  102. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 102
  103. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail NULL 103
  104. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 104
  105. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 105
  106. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 106
  107. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 107
  108. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 108
  109. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail 109
  110. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail next 110
  111. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail next 111
  112. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail next 112
  113. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } Head Tail next 113
  114. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } 114
  115. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 115
  116. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 116
  117. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 117
  118. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 118
  119. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 119
  120. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 120
  121. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail next 121
  122. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 122
  123. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 123
  124. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 124
  125. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 125
  126. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 126
  127. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail 127
  128. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail NULL 128
  129. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail NULL 129
  130. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } Head Tail NULL 130
  131. Separate locks private Node head = null, tail = null;

    private final Object HEAD_LOCK = new Object(); private final Object TAIL_LOCK = new Object(); public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) { if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; } } public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; } } 131
  132. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 132
  133. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 133
  134. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 134
  135. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 135
  136. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 136
  137. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 137
  138. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 138
  139. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 139
  140. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 140 next
  141. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail BLOCKED! 141 next
  142. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 142 next
  143. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 143 next
  144. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 144 next
  145. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail next 145
  146. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 146
  147. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 147
  148. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail 148
  149. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Видишь суслика? Есть ли баги? 149
  150. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Ага, какой-то там анализатор будет их героически находить. Видишь суслика? Есть ли баги? 150
  151. Запускаем JPF ====================Test.jpf==================== target=Test target.args= classpath=. sourcepath=. report.console.property_violation = output,trace

    151
  152. Запускаем JPF /home/JPF/jpf-core/bin/jpf Test.jpf 152

  153. Запускаем JPF /home/JPF/jpf-core/bin/jpf Test.jpf JavaPathfinder core system v8.0 (rev 32)

    - (C) 2005-2014 United States Government. All rights reserved. ================================= system under test Test.main() ================================= search started: 10/24/19 10:43 PM ================================= trace #1 --------------------------------- transition #0 thread: 0 gov.nasa.jpf.vm.choice.ThreadChoiceFromSet {id:"ROOT" ,1/1,isCascaded:false} [3157 insn w/o sources] Test.java:64 : private static final Random rand = new Random(123); [2 insn w/o sources] Test.java:64 : private static final Random rand = new Random(123); Test.java:1 : import java.util.Random; [1 insn w/o sources] Test.java:67 : Queue q = QueueProvider.getQueue(); 153
  154. Запускаем JPF /home/JPF/jpf-core/bin/jpf Test.jpf JavaPathfinder core system v8.0 (rev 32)

    - (C) 2005-2014 United States Government. All rights reserved. ================================= system under test Test.main() ================================= search started: 10/24/19 10:43 PM ================================= trace #1 --------------------------------- transition #0 thread: 0 gov.nasa.jpf.vm.choice.ThreadChoiceFromSet {id:"ROOT" ,1/1,isCascaded:false} [3157 insn w/o sources] Test.java:64 : private static final Random rand = new Random(123); [2 insn w/o sources] Test.java:64 : private static final Random rand = new Random(123); Test.java:1 : import java.util.Random; [1 insn w/o sources] Test.java:67 : Queue q = QueueProvider.getQueue(); 270 строк: все действия каждого потока 154
  155. Запускаем JPF: результат ====================================================== results error #1: gov.nasa.jpf.vm.NotDeadlockedProperty "deadlock encountered:

    thread java.lang.Thread:{i..." ====================================================== statistics elapsed time: 00:00:00 states: new=603,visited=447,backtracked=1017,end=3 search: maxDepth=54,constraints=0 choice generators: thread=602 (signal=0,lock=108,sharedRef=349,threadApi=91,reschedule=54), data=0 heap: new=409,released=295,maxLive=390,gcCycles=971 instructions: 20933 max memory: 77MB loaded code: classes=69,methods=1506 155
  156. Запускаем JPF: результат ====================================================== results error #1: gov.nasa.jpf.vm.NotDeadlockedProperty "deadlock encountered:

    thread java.lang.Thread:{i..." ====================================================== statistics elapsed time: 00:00:00 states: new=603,visited=447,backtracked=1017,end=3 search: maxDepth=54,constraints=0 choice generators: thread=602 (signal=0,lock=108,sharedRef=349,threadApi=91,reschedule=54), data=0 heap: new=409,released=295,maxLive=390,gcCycles=971 instructions: 20933 max memory: 77MB loaded code: classes=69,methods=1506 156
  157. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 157
  158. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 158
  159. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 159
  160. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 160
  161. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 161
  162. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL 162
  163. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL BLOCKED! 163
  164. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL BLOCKED! 164
  165. public Node dequeue() { synchronized (TAIL_LOCK) { synchronized (HEAD_LOCK) {

    if (this.tail == this.head) { Node result = this.head; head = tail = null; return result; } } Node result = this.tail; this.tail = result.next; return result; public void enqueue(Node element) { synchronized (HEAD_LOCK) { Node curHead = this.head; if (curHead == null) { synchronized (TAIL_LOCK) { head = tail = element; return; } } curHead.next = element; element.next = null; this.head = element; Head Tail NULL BLOCKED! BLOCKED! 165
  166. Попытка №2: минимальная область блокировки private Node getHead() { synchronized

    (HEAD_LOCK) { return head; } } private void setHead(Node newHead) { synchronized (HEAD_LOCK) { head = newHead; } } private Node getTail() { synchronized (TAIL_LOCK) { return tail; } } private void setTail(Node newTail) { synchronized (TAIL_LOCK) { tail = newTail; } } 166
  167. Попытка №2: минимальная область блокировки public void enqueue(Node element) {

    Node curHead = getHead(); if (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node tailCopy = getTail(); Node headCopy = getHead(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 167
  168. Запускаем JPF /home/JPF/jpf-core/bin/jpf Test.jpf ====================================================== results error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.NullPointerException:

    referencing field ..." ====================================================== statistics elapsed time: 00:00:00 states: new=126,visited=70,backtracked=162,end=2 search: maxDepth=44,constraints=0 choice generators: thread=125 (signal=0,lock=42,sharedRef=45,threadApi=26,reschedule=12), data=0 heap: new=419,released=103,maxLive=390,gcCycles=183 instructions: 8910 max memory: 61MB loaded code: classes=73,methods=1540 320 строк: все действия каждого потока 168
  169. Запускаем JPF ====================Test.jpf==================== target=Test target.args= classpath=. sourcepath=. search.class = gov.nasa.jpf.search.heuristic.BFSHeuristic

    report.console.property_violation = output,trace 169
  170. Запускаем JPF /home/JPF/jpf-core/bin/jpf Test.jpf ====================================================== results error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.NullPointerException:

    referencing field ..." ====================================================== statistics elapsed time: 00:00:00 states: new=351,visited=305,backtracked=655,end=0 search: maxDepth=16,constraints=0 choice generators: thread=281 (signal=0,lock=155,sharedRef=123,threadApi=3,reschedule=0), data=0 heap: new=591,released=44,maxLive=389,gcCycles=586 instructions: 14320 max memory: 61MB loaded code: classes=73,methods=1540 180 строк (было 320) 170
  171. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 171 Head Tail NULL
  172. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 172 Head Tail NULL
  173. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 173 Head Tail NULL
  174. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 174 Head Tail NULL
  175. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 175 Head Tail NULL
  176. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 176 Head Tail NULL
  177. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 177 Head Tail NULL
  178. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 178 Head Tail NULL
  179. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 179 Head Tail NULL
  180. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 180 Head Tail NULL
  181. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 181 Head Tail NULL
  182. public void enqueue(Node element) { Node curHead = getHead(); if

    (curHead == null) { setHead(element); setTail(element); return; } curHead.next = element; element.next = null; setHead(element); } public Node dequeue() { Node headCopy = getHead(); Node tailCopy = getTail(); if (tailCopy == headCopy) { setHead(null); setTail(null); return headCopy; } setTail(tailCopy.next); return tailCopy; } 182 Head Tail NULL NullPointerException
  183. Что-нибудь серьезное? 183

  184. MPSC queue http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue 184 https://www.boost.org/doc/libs/1_63_0/boost/fiber/detail/context_mpsc_queue.hpp

  185. MPSC queue http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue 185 Автор: Dmitry Vyukov (aka Dmitiy V'jukov,

    dvyukov, Дмитрий Вьюков) Intel Black Belt Software Developer in Parallel Programming Winner of Intel Threading Challenge 2010 Invented a dozen of novel synchronization algorithms (queues, mutexes, registers, lifetime management, etc) Developed Relacy Race Detector - a tool for precise verification of synchronization algorithms for relaxed memory models https://www.boost.org/doc/libs/1_63_0/boost/fiber/detail/context_mpsc_queue.hpp
  186. MPSC queue http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue 186

  187. Vjukov queue 187 public static class Node { public volatile

    Queue.Node next; } struct mpscq_node_t { mpscq_node_t* volatile next; };
  188. Vjukov queue 188 public static class Node { public volatile

    Queue.Node next; } struct mpscq_node_t { mpscq_node_t* volatile next; }; struct mpscq_t { mpscq_node_t* volatile head; mpscq_node_t* tail; mpscq_node_t stub; }; public class VjukovQueue extends Queue { volatile Node head; Node tail; Node stub;
  189. Vjukov queue 189 VjukovQueue() { stub = new Node(); head

    = stub; tail = stub; stub.next = null; } void mpscq_create(mpscq_t* self) { self->head = &self->stub; self->tail = &self->stub; self->stub.next = 0; }
  190. 190 mpscq_node_t* mpscq_pop(mpscq_t* self) { mpscq_node_t* tail = self->tail; mpscq_node_t*

    next = tail->next; if (tail == &self->stub) { if (0 == next) return 0; self->tail = next; tail = next; next = next->next; } if (next) { self->tail = next; return tail; } public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; }
  191. 191 mpscq_node_t* head = self->head; if (tail != head) return

    0; mpscq_push(self, &self->stub); next = tail->next; if (next) { self->tail = next; return tail; } return 0; } Node head = this.head; if (tail != head) return null; enqueue(stub); next = tail.next; if (next != null) { this.tail = next; return tail; } return null; } dequeue
  192. Vjukov queue 192 void mpscq_push(mpscq_t* self, mpscq_node_t* n) { n->next

    = 0; mpscq_node_t* prev = XCHG(&self->head, n); prev->next = n; }
  193. Vjukov queue 193 void mpscq_push(mpscq_t* self, mpscq_node_t* n) { n->next

    = 0; mpscq_node_t* prev = XCHG(&self->head, n); prev->next = n; } public void enqueue(Node element) { element.next = null; Node prev = XCHG(element); prev.next = element; }
  194. Vjukov queue 194 void mpscq_push(mpscq_t* self, mpscq_node_t* n) { n->next

    = 0; mpscq_node_t* prev = XCHG(&self->head, n); prev->next = n; } public void enqueue(Node element) { element.next = null; Node prev = XCHG(element); prev.next = element; }
  195. Vjukov queue 195 public void enqueue(Node element) { element.next =

    null; Node prev = XCHG(element); prev.next = element; } private Node XCHG(Node n) { synchronized (this) { Node old = head; head = n; return old; } }
  196. Vjukov queue 196 public void enqueue(Node element) { element.next =

    null; Node prev = XCHG(element); prev.next = element; } private Node XCHG(Node n) { synchronized (this) { Node old = head; head = n; return old; } } private AtomicReference<Node> head = new AtomicReference<>(); private Node XCHG(Node n) { return head.getAndSet(n); }
  197. Data integrity problem 197 А что если dequeue() вернет один

    и тот же объект разным потокам? Как отследить, что такая ситуация не возникает?
  198. Data integrity problem 198 А что если dequeue() вернет один

    и тот же объект разным потокам? Как отследить, что такая ситуация не возникает? static class CheckedNode extends Queue.Node { volatile int x = 0; void beforeRelease() { // before enqueue assert x == 0; x = 1; assert x == 1; } void afterAcquire() { // after dequeue assert x == 1; x = 0; assert x == 0; } }
  199. Data integrity problem 199 static void startInNewThread(Runnable r) { new

    Thread(r).start(); } static void task(Queue q) { CheckedNode n = new CheckedNode(); n.beforeRelease(); q.enqueue(n); CheckedNode v = q.dequeue(); if (v != null) v.afterAcquire(); } static void test() { Queue q = new VjukovQueue(); startInNewThread(() -> task(q)); startInNewThread(() -> task(q)); }
  200. Data integrity problem 200 ====================================================== results error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.AssertionError

    at Test$CheckedNode.afte..." ====================================================== statistics elapsed time: 00:00:00 states: new=720,visited=637,backtracked=1356,end=35 search: maxDepth=19,constraints=0 choice generators: thread=623 (signal=0,lock=90,sharedRef=457,threadApi=2,reschedule=74), data=0 heap: new=802,released=1890,maxLive=395,gcCycles=1259 instructions: 25342 max memory: 77MB loaded code: classes=72,methods=1553
  201. Data integrity problem 201 ====================================================== results error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.AssertionError

    at Test$CheckedNode.afte..." ====================================================== statistics elapsed time: 00:00:00 states: new=720,visited=637,backtracked=1356,end=35 search: maxDepth=19,constraints=0 choice generators: thread=623 (signal=0,lock=90,sharedRef=457,threadApi=2,reschedule=74), data=0 heap: new=802,released=1890,maxLive=395,gcCycles=1259 instructions: 25342 max memory: 77MB loaded code: classes=72,methods=1553
  202. 202 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  203. 203 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  204. 204 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  205. 205 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  206. 206 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  207. 207 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  208. 208 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  209. 209 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  210. 210 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  211. 211 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  212. 212 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  213. 213 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  214. 214 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  215. 215 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next
  216. 216 public Node dequeue() { Node tail = this.tail; Node

    next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… public Node dequeue() { Node tail = this.tail; Node next = tail.next; if (tail == this.stub) { if (null == next) return null; this.tail = next; tail = next; next = next.next; } if (next != null) { this.tail = next; return tail; } ……………………………… Head Tail next Почему?
  217. 217 Почему?

  218. 218 Почему?

  219. 219 Почему? Multiple Producer Single Consumer Правильная структура данных использовалась

    в неправильном контексте
  220. 220 Почему? Multiple Producer Single Consumer Правильная структура данных использовалась

    в неправильном многопоточном контексте
  221. Найденные ошибки 221 Deadlock Нарушение внутренних свойств (NullPointerException) Логическая ошибка

    (assert) -- использование неподходящей структуры данных
  222. Найденные ошибки 222 Deadlock Нарушение внутренних свойств (NullPointerException) Логическая ошибка

    (assert) -- использование неподходящей структуры данных Требуются правильные тесты и правильные проверки.
  223. Найденные ошибки 223 Deadlock Нарушение внутренних свойств (NullPointerException) Логическая ошибка

    (assert) -- использование неподходящей структуры данных Требуются хорошие многопоточные тесты и правильные проверки.
  224. Найденные ошибки 224 Deadlock Нарушение внутренних свойств (NullPointerException) Логическая ошибка

    (assert) -- использование неподходящей структуры данных Требуются хорошие многопоточные тесты и достаточный набор assert-ов.
  225. Найденные ошибки 225 Deadlock Нарушение внутренних свойств (NullPointerException) Логическая ошибка

    (assert) -- использование неподходящей структуры данных Требуются хорошие многопоточные тесты и достаточный набор assert-ов.
  226. KSUH: real case 226 “A Fair Fast Scalable Reader-Writer Lock

    “, 1993 Orran Krieger, Michael Stumm, Ron Unrau, Jonathan Hanna In Proc. of the International Conference on Parallel Processing Реализация на C. Верифицирован с помощью SPIN (модель на C-подобном языке Promela)
  227. KSUH: real case 227 “A Fair Fast Scalable Reader-Writer Lock

    “, 1993 Orran Krieger, Michael Stumm, Ron Unrau, Jonathan Hanna In Proc. of the International Conference on Parallel Processing Реализация на C. Верифицирован с помощью SPIN (модель на C-подобном языке Promela) Разные языки для реализации и для моделирования.
  228. KSUH: real case 228 “Using Hardware Transactional Memory to Correct

    and Simplify and Readers-writer Lock Algorithm”, 2013 Dave Dice, Yossi Lev, Yujie Liu, Victor Luchangco, Mark Moir Прошло 20 лет. Нашли ошибку в реализации KSUH, связанную с управлением памятью. Отлаживались на реальной программе.
  229. KSUH: real case 229 “Using Hardware Transactional Memory to Correct

    and Simplify and Readers-writer Lock Algorithm”, 2013 Dave Dice, Yossi Lev, Yujie Liu, Victor Luchangco, Mark Moir Прошло 20 лет. Нашли ошибку в реализации KSUH, связанную с управлением памятью. Отлаживались на реальной программе. Java Path Finder: Весь класс 250 строк. Тест 90 строк. Анализ нашел проблему за 7 минут. Понадобилось 700Mb RAM.
  230. KSUH: real case 230 “Using Hardware Transactional Memory to Correct

    and Simplify and Readers-writer Lock Algorithm”, 2013 Dave Dice, Yossi Lev, Yujie Liu, Victor Luchangco, Mark Moir Прошло 20 лет. Нашли ошибку в реализации KSUH, связанную с управлением памятью. Отлаживались на реальной программе. Java Path Finder: Весь класс 250 строк. Тест 90 строк. Анализ нашел проблему за 7 минут. Понадобилось 700Mb RAM. Люди с учеными степенями
  231. KSUH: real case 231 “Using Hardware Transactional Memory to Correct

    and Simplify and Readers-writer Lock Algorithm”, 2013 Dave Dice, Yossi Lev, Yujie Liu, Victor Luchangco, Mark Moir Прошло 20 лет. Нашли ошибку в реализации KSUH, связанную с управлением памятью. Отлаживались на реальной программе. Java Path Finder: Весь класс 250 строк. Тест 90 строк. Анализ нашел проблему за 7 минут. Понадобилось 700Mb RAM. Люди с учеными степенями Роботы
  232. Ограничения JPF Масштабируемость 10 KLOC 3-4 одновременно работающих потока 3-6

    часов на один тест 10-20GB RAM 232
  233. Ограничения JPF Масштабируемость 10 KLOC 3-4 одновременно работающих потока 3-6

    часов на один тест 10-20GB RAM Не гарантирует корректность Нужно писать assert-ы Нужно писать тесты Тонкости JMM 233
  234. Ограничения JPF Масштабируемость 10 KLOC 3-4 одновременно работающих потока 3-6

    часов на один тест 10-20GB RAM Не гарантирует корректность Нужно писать assert-ы Нужно писать тесты Тонкости JMM Safe harbor Перед принятием решения проконсультируйтесь со специалистами. 234
  235. Спасибо за внимание! 235