Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shifting Gears With Gearman

Shifting Gears With Gearman

Everyone starts with a simple one machine setup, running PHP, MySQL and Apache(or Nginx). Sooner or later some tasks will start taking too long and your application will start to slow down or fail, or your infractstructure will not be enough.
This talk will show you how you can optimize and increase the performance of your application by distributing tasks and what problems you will encounter, and how to possibly mitigate them

Vranac Srdjan

April 27, 2014
Tweet

More Decks by Vranac Srdjan

Other Decks in Programming

Transcript

  1. SHIFTING
    GEARS WITH
    GEARMAN
    Srdjan Vranac // code4hire.com // @vranac
    0

    View Slide

  2. JUST SCRAPE IT!
    30.000.000 PRODUCTS
    / 12 HOURS = 2.500.000 PRODUCTS
    / 3600 SECONDS = 695 PRODUCTS
    600 NODES = 1.1 SEC. PER PRODUCT

    View Slide

  3. JUST RESIZE IT!

    View Slide

  4. ASYNCHRONOUS
    PROCESSING

    View Slide

  5. MESSAGING AND QUEUES
    Process 4
    Process 1 Process 3
    Process 2
    COMMUNICATION BETWEEN
    PROCESSESS

    View Slide

  6. GEARMAN
    MANAGER

    View Slide

  7. GEARMAN
    Gearman Job
    Server
    Worker 1
    Worker 2
    Worker 3
    Worker ...
    Web Server

    View Slide

  8. NO RUNNING ORDER OR
    RELIABILITY

    View Slide

  9. DATA
    Web Server
    Db
    Gearman Job
    Server
    Worker 1
    Worker 2
    Worker 3
    Worker ...

    View Slide

  10. LOCATION
    DE-NORMALIZATION

    View Slide

  11. CAP THEOREM
    Availability
    Partition
    Tolerance
    Consistency
    CA
    AP
    CP

    View Slide

  12. EVENTUAL CONSISTENCY

    View Slide

  13. ATOMIC OPERATIONS

    View Slide

  14. WHAT IS GOING ON?

    View Slide

  15. DEBBUGGING DISTRIBUTED
    SYSTEMS IS HARD!

    View Slide

  16. MONITORING

    View Slide

  17. COUNT EVERYTHING

    View Slide

  18. UPTIME, VMSTAT, NETSTAT,
    IOSTAT, MPSTAT, TCPDUMP,
    GDB, VALGRIND, ULIMIT,
    SYSCTL ETC...

    View Slide

  19. THE END ?

    View Slide

  20. LINKS
    , ,
    Gearman Supervisor StatsD
    Measure Anything, Measure Everything

    View Slide