Shifting Gears With Gearman

Shifting Gears With Gearman

Everyone starts with a simple one machine setup, running PHP, MySQL and Apache(or Nginx). Sooner or later some tasks will start taking too long and your application will start to slow down or fail, or your infractstructure will not be enough.
This talk will show you how you can optimize and increase the performance of your application by distributing tasks and what problems you will encounter, and how to possibly mitigate them

F2d82b268a7cbccc9809c939428df64f?s=128

Vranac Srdjan

April 27, 2014
Tweet

Transcript

  1. SHIFTING GEARS WITH GEARMAN Srdjan Vranac // code4hire.com // @vranac

    0
  2. JUST SCRAPE IT! 30.000.000 PRODUCTS / 12 HOURS = 2.500.000

    PRODUCTS / 3600 SECONDS = 695 PRODUCTS 600 NODES = 1.1 SEC. PER PRODUCT
  3. JUST RESIZE IT!

  4. ASYNCHRONOUS PROCESSING

  5. MESSAGING AND QUEUES Process 4 Process 1 Process 3 Process

    2 COMMUNICATION BETWEEN PROCESSESS
  6. GEARMAN MANAGER

  7. GEARMAN Gearman Job Server Worker 1 Worker 2 Worker 3

    Worker ... Web Server
  8. NO RUNNING ORDER OR RELIABILITY

  9. DATA Web Server Db Gearman Job Server Worker 1 Worker

    2 Worker 3 Worker ...
  10. LOCATION DE-NORMALIZATION

  11. CAP THEOREM Availability Partition Tolerance Consistency CA AP CP

  12. EVENTUAL CONSISTENCY

  13. ATOMIC OPERATIONS

  14. WHAT IS GOING ON?

  15. DEBBUGGING DISTRIBUTED SYSTEMS IS HARD!

  16. MONITORING

  17. COUNT EVERYTHING

  18. UPTIME, VMSTAT, NETSTAT, IOSTAT, MPSTAT, TCPDUMP, GDB, VALGRIND, ULIMIT, SYSCTL

    ETC...
  19. THE END ?

  20. LINKS , , Gearman Supervisor StatsD Measure Anything, Measure Everything