Bumping our Stack to PHP 5.5

A bit of a dev story, followed by a bit more of an ops story.

Here's what happened when we decided it was about time to get around to upgrading our PHP 5.3 app to something a bit more current.

Transcript

1. View Slide

2. Glen Mailer
   

   View Slide

3. A short

   audience survey
   

   View Slide

4. Set the Scene

   (as of September 2013)
   

   View Slide

5. The Codebase
   

   View Slide

6. First commit April 2010

   60k commits since
   According to git
   

   View Slide

7. 9300 PHP files

   690k lines of code
   According to cloc
   

   View Slide

8. 10 releases a week

   35 active committers

   10 concurrent streams

   1 deployable
   

   View Slide

9. The Platform
   

   View Slide

10. 8 Test Environments

    1 Staging Environment

    2 Live Environments
    

    View Slide

11. 74 Live PHP boxes
    

    View Slide

12. 1300 req/s peak

    500k uniques/week
    

    View Slide

13. Why Upgrade?
    

    View Slide

14. Using basically the same
    PHP version as the first
    commit
    with security patches
    

    View Slide

15. PHP 5.3 is dead
    

    View Slide

16. Bug fixes

    Performance
    

    View Slide

17. New Features!
    short array syntax
    traits
    $this in closures
    yield
    finally
    

    View Slide

18. View Slide

19. Bug #64827

    GC causes segfaults
    

    View Slide

20. The Plan
    

    View Slide

21. Upgrade a dev box
    Support 5.3 and 5.5
    Pass all tests
    Upgrade 1 Test env
    Upgrade CI
    Upgrade Test & Staging
    Incrementally Upgrade Live
    

    View Slide

22. Vagrant

    Chef

    Unit Tests

    Acceptance Tests
    

    View Slide

23. Hack hack hack...
    

    View Slide

24. 3 days later
    

    View Slide

25. What changed?
    

    View Slide

26. APC

    becomes

    OPcache and APCu
    

    View Slide

27. Check for new function, fall back to old one
    if (function_exists('opcache_reset')) {
    opcache_reset();
    } else {
    apc_clear_cache();
    }
    

    View Slide

28. “The PHP api will retain
    compatibility with APC, as
    will common configuration
    options, providing a drop
    in replacement.”
    APCu
    

    View Slide

29. Not a drop in replacement
    

    View Slide

30. APCu 4.0.1

    APCu 4.0.2

    master is closer still
    

    View Slide

31. No more

    mysql_
    

    View Slide

32. It will not be missed
    

    View Slide

33. preg_replace /e

    becomes

    preg_replace_callback
    

    View Slide

34. (string) array()

    Now raises a warning
    Still returns “Array”
    

    View Slide

35. "Type-checked" function
    function inc($int) {
    if (!is_int($int)) {
    throw new InvalidArgumentException(
    'Expected int, got: ' . $int
    );
    }
    return $int + 1;
    }
    

    View Slide

36. Unit test for type-checked function
    /**
    * @dataprovider provideIncInvalid
    */
    function testIncRejectsInvalid($a) {
    try {
    inc($a);
    $this->fail('Expected exception');
    } catch (InvalidArgumentException $ex) {}
    }
    function provideIncInvalid() {
    return array(
    null, true, "string", array()
    );
    }
    

    View Slide

37. Simplest fix
    throw new InvalidArgumentException(
    'Expected int, got: ' .
    json_encode($int)
    );
    

    View Slide

38. $a = "string”;

    $a['index'];

    Now raises a warning
    

    View Slide

39. // protected $a;

    ++$this->a

    Now raises a warning
    

    View Slide

40. Progress
    

    View Slide

41. Upgrade a dev box

    Support 5.3 and 5.5

    Pass all tests

    Upgrade 1 Test env

    Upgrade CI

    Upgrade Test & Staging

    Incrementally Upgrade Live
    

    View Slide

42. Upgrade a dev box

    Support 5.3 and 5.5

    Pass all tests

    Upgrade 1 Test env

    Upgrade CI

    Upgrade Test & Staging

    Incrementally Upgrade Live
    

    View Slide

43. PHPNW 2013
    

    View Slide

44. The Rollout
    

    View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. APCu Failures

    APCu #49: Segfaults

    APCu #19: deadlock
    

    View Slide

50. Forwards

    or

    Backwards
    

    View Slide

51. View Slide

52. Segfaults - 100 point Moving Average
    7.2/s
    6.0/s
    4.8/s
    3.6/s
    2.4/s
    1.2/s
    Oct Nov Dec Jan
    

    View Slide

53. View Slide

54. Shared Memory?

    I'm out.
    

    View Slide

55. Can we

    leverage OPCache?
    

    View Slide

56. The Barwell Cache
    

    View Slide

57. function barwell_store($key, $value) {
    $path = BARWELL_BASE . '/' . md5($key);
    file_put_contents(
    $path,
    'var_export($value, 1)
    );
    opcache_invalidate($path);
    }
    function barwell_fetch($key) {
    return include(BARWELL_BASE .
    '/' . md5($key));
    }
    

    View Slide

58. function barwell_store_obj($key, $value) {
    $path = BARWELL_BASE . '/' . md5($key);
    file_put_contents(
    $path,
    'serialize($value) . '")'
    );
    opcache_invalidate($path);
    }
    

    View Slide

59. Are you mad?
    

    View Slide

60. View Slide

61. View Slide

62. Probably.
    

    View Slide

63. Take a moment.

    Stop.

    Think.
    

    View Slide

64. What is the problem
    we're trying to solve?
    

    View Slide

65. Fast k/v access

    for hot paths
    

    View Slide

66. How fast do we really
    need to be?
    

    View Slide

67. Move almost everything
    over to memcached
    

    View Slide

68. Autoload Map

    Environment Config

    Weird Zend Reflection
    

    View Slide

69. function simple_store($key, $value) {
    file_put_contents(
    path($key), serialize($value));
    }
    function simple_fetch($key) {
    return unserialize(
    file_get_contents(path($key)));
    }
    

    View Slide

70. The

    Aftermath
    

    View Slide

71. Performance
    

    View Slide

72. Average Response Time in ms
    

    View Slide

73. Average Memory Use in MB
    

    View Slide

74. Average Response Time in ms

    on one API endpoint with
    minimal external calling
    

    View Slide

75. New Features
    

    View Slide

76. 17 Traits

    +

    4 in tests
    

    View Slide

77. 2 uses of

    finally
    

    View Slide

78. 37 uses of

    yield
    

    View Slide

79. New array syntax?
    

    View Slide

80. A few snowflake boxes
    are still on 5.3
    

    View Slide

81. Have bumped minor
    version a couple of
    times since
    

    View Slide

82. Began upgrading

    to 5.6 this week
    

    View Slide

83. Takeaways
    

    View Slide

84. Use configuration
    management tools
    

    View Slide

85. Frequently upgrade
    things in small
    increments
    

    View Slide

86. Shared memory is
    inherently complicated
    

    View Slide

87. Monitor absolutely
    everything
    

    View Slide

88. When you think you're
    doing something crazy

    

    You might be right
    

    View Slide

89. Understand why you're
    doing something
    

    View Slide

90. Fin.
    

    View Slide

91. @glenathan
    
    https://joind.in/11801
    
    

    View Slide