Shifting Gears With Gearman

Shifting Gears With Gearman

Everyone starts with a simple one-machine setup, running PHP(or whatever else you fancy), MySQL/PostgreSql and Apache/Nginx. Sooner or later that will not be enough, and you will be faced with some important architecture decisions. This talk will show you how you can optimize and increase the performance of your application by distributing tasks in an simple and affordable way.

F2d82b268a7cbccc9809c939428df64f?s=128

Vranac Srdjan

October 04, 2014
Tweet

Transcript

  1. Shifting Gears With Gearman Srdjan Vranac // code4hire.com // @vranac

  2. business owner, developer , consultant, mercenary, writing terrible code that

    performs exceptionally, wrangling ele PHP ants and Python s, obsessed with process automation , interested in continuous integration and delivery, clean code, testing, best practices and distributed systems
  3. Just Scrape It! 30.000.000 Products / 12 Hours = 2.500.000

    Products / 3600 Seconds = 695 Products 600 nodes = 1.1 Sec. per product
  4. Just Resize It!

  5. Asynchronous Processing

  6. Messaging And Queues Process 4 Process 1 Process 3 Process

    2 Communication between Processess
  7. Gearman Manager

  8. Gearman Gearman Job Server Worker 1 Worker 2 Worker 3

    Worker ... Web Server
  9. No Running Order Or Reliability

  10. Data Web Server Db Gearman Job Server Worker 1 Worker

    2 Worker 3 Worker ...
  11. Location

  12. De-Normalization

  13. CAP Theorem Availability Partition Tolerance Consistency CA AP CP

  14. Eventual Consistency

  15. Atomic Operations

  16. What Is Going On?

  17. Debbugging Distributed Systems is Hard!

  18. Monitoring

  19. Count Everything

  20. Uptime, Vmstat, Netstat, Iostat, Mpstat, Tcpdump, Gdb, Valgrind, Ulimit, Sysctl

    etc...
  21. THE END ?

  22. Links , , Gearman Supervisor StatsD Measure Anything, Measure Everything