$30 off During Our Annual Pro Sale. View Details »

Shifting Gears With Gearman

Shifting Gears With Gearman

Everyone starts with a simple one-machine setup, running PHP(or whatever else you fancy), MySQL/PostgreSql and Apache/Nginx. Sooner or later that will not be enough, and you will be faced with some important architecture decisions. This talk will show you how you can optimize and increase the performance of your application by distributing tasks in an simple and affordable way.

Vranac Srdjan

October 04, 2014
Tweet

More Decks by Vranac Srdjan

Other Decks in Technology

Transcript

  1. Shifting Gears With
    Gearman
    Srdjan Vranac // code4hire.com // @vranac

    View Slide

  2. business owner,
    developer
    , consultant, mercenary, writing
    terrible code that performs exceptionally, wrangling ele
    PHP
    ants
    and
    Python
    s, obsessed with
    process automation
    , interested in
    continuous integration and delivery, clean code, testing, best
    practices and distributed systems

    View Slide

  3. Just Scrape It!
    30.000.000 Products
    / 12 Hours = 2.500.000 Products
    / 3600 Seconds = 695 Products
    600 nodes = 1.1 Sec. per product

    View Slide

  4. Just Resize It!

    View Slide

  5. Asynchronous Processing

    View Slide

  6. Messaging And Queues
    Process 4
    Process 1 Process 3
    Process 2
    Communication between Processess

    View Slide

  7. Gearman
    Manager

    View Slide

  8. Gearman
    Gearman Job
    Server
    Worker 1
    Worker 2
    Worker 3
    Worker ...
    Web Server

    View Slide

  9. No Running Order Or Reliability

    View Slide

  10. Data
    Web Server
    Db
    Gearman Job
    Server
    Worker 1
    Worker 2
    Worker 3
    Worker ...

    View Slide

  11. Location

    View Slide

  12. De-Normalization

    View Slide

  13. CAP Theorem
    Availability
    Partition
    Tolerance
    Consistency
    CA
    AP
    CP

    View Slide

  14. Eventual Consistency

    View Slide

  15. Atomic Operations

    View Slide

  16. What Is Going On?

    View Slide

  17. Debbugging Distributed Systems is Hard!

    View Slide

  18. Monitoring

    View Slide

  19. Count Everything

    View Slide

  20. Uptime, Vmstat, Netstat, Iostat, Mpstat,
    Tcpdump, Gdb, Valgrind, Ulimit, Sysctl
    etc...

    View Slide

  21. THE END ?

    View Slide

  22. Links
    , ,
    Gearman Supervisor StatsD
    Measure Anything, Measure Everything

    View Slide