$30 off During Our Annual Pro Sale. View Details »

Hitting the Turbo Button: Upgrading to PHP 7 at Etsy

Will Gallego
September 17, 2016

Hitting the Turbo Button: Upgrading to PHP 7 at Etsy

PHP 7 offers fantastic resources to developers, trims memory consumption substantially, and can significantly reduce the CPU usage your software requires–often with few changes to your code. For those looking to migrate to PHP 7 or unsure of best practices to incorporate into your upgrade, you’ll learn about a battle tested design for deployment at scale. In this talk, I’ll discuss how Etsy designed a core upgrade to the site and how that might be applicable to companies of all sizes. Discussions will include how we chose between Hack/HHVM and PHP 7 and some of the tools built along the way, such as Phan, an open source static analyzer. This will also include an in depth review of how we deployed the upgrade, detailing some of the bumps in the process. Attendees should come away with a strong foundation of how the next phase of PHP can drastically improve performance and add powerful new features to aid in development.

Will Gallego

September 17, 2016
Tweet

More Decks by Will Gallego

Other Decks in Programming

Transcript

  1. 1

    Hitting the Turbo Button:
    Upgrading to PHP 7 at
    Etsy
    @wcgallego

    View Slide

  2. 2
    Hey, I’m Will!
    Ops Engineer @ Etsy
    @wcgallego

    View Slide

  3. 3
    @wcgallego

    View Slide

  4. 4

    Roadmap
    •Brief history of PHP 5 point release upgrades at
    Etsy
    •Our use of HHVM
    •Deciding between Hack/HHVM vs. PHP 7
    •Deployment of PHP 7
    •Wins, pitfalls, and features in PHP 7
    @wcgallego

    View Slide

  5. 5
    #VUNPTUJNQPSUBOUMZ
    *XBOUZPVUPCFTVDDFTTGVMBU
    VQHSBEJOH1)1
    @wcgallego

    View Slide

  6. 6
    PHP @ Etsy Timeline
    @wcgallego
    Prehistory 2012 2013 2014 2015 …
    2012
    …to 5.3
    PHP 5.2
    2013
    …to 5.4
    2014
    …to 5.5
    2015
    …5.6?

    View Slide

  7. Reworking our
    API Architecture
    @wcgallego

    View Slide

  8. 8
    @wcgallego
    Webs
    Browser
    DBs
    Previous Approach

    View Slide

  9. 9
    @wcgallego
    Webs
    Browser
    Internal API
    (multicurl)
    API Node API Node API Node
    API First Approach
    DBs

    View Slide

  10. 10
    @wcgallego
    Rising API Requests

    View Slide

  11. HHVM to the rescue!
    @wcgallego

    View Slide

  12. 12
    @wcgallego

    View Slide

  13. 13
    @wcgallego

    View Slide

  14. 14
    @wcgallego

    View Slide

  15. 15
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    Early PHP 7 Testing/Benchmarks

    View Slide

  16. 16
    https://wiki.php.net/rfc/php6
    “Version 6 is generally associated with failure in
    the world of dynamic language”
    @wcgallego
    Wait…what happened to PHP 6?

    View Slide

  17. 17
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    Porting Etsy Extensions to PHP 7

    View Slide

  18. 18
    https://bugs.php.net/bug.php?id=72396
    @wcgallego
    https://php-lxr.adamharvey.name/source/
    http://lxr.php.net/

    View Slide

  19. 19

    1)1WT)BDL))7.

    8IJDIEPXFDIPPTF
    @wcgallego

    View Slide

  20. 20
    http://talks.php.net/velocity15#/wpbench
    http://hhvm.com/blog/9293/lockdown-results-and-hhvm-performance
    https://kinsta.com/blog/the-definitive-php-7-final-version-hhvm-
    benchmark/
    @wcgallego

    View Slide

  21. 21
    Hack/HHVM
    @wcgallego
    Cons
    •More experiential knowledge with

    PHP/mod_php than Hack/HHVM/
    fastcgi
    •Debugging multithreaded extensions
    •Different environment than
    Facebook: multi-curl in API at Etsy
    vs. async at Facebook
    •HHVM deploy shorter, but Hack
    everywhere a multiyear project
    Pros
    •Async (cooperative multitasking)
    •Generics
    •Collections (Vectors, Maps, Sets, Pairs)
    •Tuples, Enums, Shapes
    •compile time features like void return

    View Slide

  22. 22
    PHP 7
    @wcgallego
    Cons
    •Need to write our own static analyzer
    •Core not yet prod ready
    •Extensions hadn’t seen wide internal
    testing like core
    •Some type features Hack has not in
    PHP
    Pros
    •All of our editors supported PHP 7 - vim,
    emacs, sublime text, phpstorm, etc.
    •Unsure direction Facebook takes Hack
    •Maintain current understanding of how
    our site runs
    •Shorter migration
    •Rasmus

    View Slide

  23. 23
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    Decision to deploy PHP 7

    View Slide

  24. 24
    4
    2
    1 3
    Staging
    Fix Tests 100% Dev
    Compatible with
    PHP 5 and PHP
    7 code, still
    running PHP 5
    Test PHP 7
    running in safe
    environment
    Prevent
    introduction of
    PHP 7 code to
    PHP 5 boxes
    PHP 7 Deployment Plan
    Production
    Slow introduction
    to pools, careful
    eye on perf and
    errors
    @wcgallego

    View Slide

  25. 25
    PHPUnit

    Upgrade before you start!
    @wcgallego

    View Slide

  26. 26
    PHPUnit
    Tests: 17815
    Assertions: 61346
    Errors: 66
    Failures: 120
    https://github.com/krakjoe/uopz
    @wcgallego

    View Slide

  27. 27
    Phan
    https://github.com/etsy/phan
    @wcgallego

    View Slide

  28. 28

    Tee-ing Requests
    @wcgallego

    View Slide

  29. 29
    https://github.com/buger/gor
    @wcgallego

    View Slide

  30. 30
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    Single web node running PHP 7

    View Slide

  31. 31
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    PHP 7 deployed to all web nodes
    NPOUITTJODFCFHJOOJOHFYUFOTJPOTQPSUJOH
    NPOUITJOUFHSBUJOHJOUP&UTZ
    NPOUITBGUFSSFMFBTF

    View Slide

  32. 32
    ))7.1)1
    @wcgallego

    View Slide

  33. 33

    Other Snowflakes
    @wcgallego

    View Slide

  34. 34
    PHP @ Etsy Timeline
    @wcgallego
    Spring

    2015
    Summer

    2015
    Fall

    2015
    January

    2016
    February

    2016
    Summer

    2016
    Deployed to Dev VMs
    PHP 7 everywhere!

    View Slide

  35. 35
    Perf wins!
    @wcgallego

    View Slide

  36. .FNPSZXJOT
    36
    @wcgallego

    View Slide

  37. 4ZTUFN$16XJOT
    37
    @wcgallego

    View Slide

  38. 6TFS$16XJOT
    38
    @wcgallego

    View Slide

  39. )PNFQBHFEFMJWFSZXJOT
    39
    @wcgallego

    View Slide

  40. -JTUJOHEFMJWFSZXJOT
    40
    @wcgallego

    View Slide

  41. 'FXFSIPTUT
    41
    @wcgallego

    View Slide

  42. 42

    $BVUJPO
    @wcgallego

    View Slide

  43. $PEFQJUGBMMT6OJGPSN7BS1BSTJOH
    43
    @wcgallego
    $class->$foo['bar']();
    In PHP 5 that is interpreted as:
    $class->($foo['bar'])()
    while in PHP 7 it is:
    ($class->$foo)['bar']()
    It is easy enough to fix by adding curlies:
    $class->{$foo[‘bar']}()

    View Slide

  44. $PEFQJUGBMMT6OJGPSN7BS1BSTJOH
    44
    @wcgallego
    $class->$foo['bar']();
    In PHP 5 that is interpreted as:
    $class->($foo['bar'])()
    while in PHP 7 it is:
    ($class->$foo)['bar']()
    It is easy enough to fix by adding curlies:
    $class->{$foo[‘bar']}()

    View Slide

  45. $PEFQJUGBMMT6OJGPSN7BS1BSTJOH
    45
    @wcgallego
    $class->$foo['bar']();
    In PHP 5 that is interpreted as:
    $class->($foo['bar'])()
    while in PHP 7 it is:
    ($class->$foo)['bar']()
    It is easy enough to fix by adding curlies:
    $class->{$foo[‘bar']}()

    View Slide

  46. $PEFQJUGBMMT6OJGPSN7BS1BSTJOH
    46
    @wcgallego
    $class->$foo['bar']();
    In PHP 5 that is interpreted as:
    $class->($foo['bar'])()
    while in PHP 7 it is:
    ($class->$foo)['bar']()
    It is easy enough to fix by adding curlies:
    $class->{$foo[‘bar']}()

    View Slide

  47. $PEFQJUGBMMT"TTPDJBUJWF"SSBZ4PSUJOH
    $foo = [
    "a" => 1,
    "b" => 1
    ];
    asort($foo);
    print_r($foo);
    47
    @wcgallego
    PHP 5:
    Array
    (
    [b] => 1
    [a] => 1
    )
    PHP 7:
    Array
    (
    [a] => 1
    [b] => 1
    )

    View Slide

  48. $PEFQJUGBMMT"TTPDJBUJWF"SSBZ4PSUJOH
    $foo = [
    "a" => 1,
    "b" => 1
    ];
    asort($foo);
    print_r($foo);
    48
    @wcgallego
    PHP 5:
    Array
    (
    [b] => 1
    [a] => 1
    )
    PHP 7:
    Array
    (
    [a] => 1
    [b] => 1
    )

    View Slide

  49. $PEFQJUGBMMT"TTPDJBUJWF"SSBZ4PSUJOH
    $foo = [
    "a" => 1,
    "b" => 1
    ];
    asort($foo);
    print_r($foo);
    49
    @wcgallego
    PHP 5:
    Array
    (
    [b] => 1
    [a] => 1
    )
    PHP 7:
    Array
    (
    [a] => 1
    [b] => 1
    )

    View Slide

  50. $PEFQJUGBMMT$BUDIBCMF&SSPST
    50
    @wcgallego
    // PHP 5 era code that will break.
    function handler(Exception $e) { ... }
    set_exception_handler('handler');
    // PHP 5 and 7 compatible.
    function handler($e) { ... }
    // PHP 7 only.
    function handler(Throwable $e) { ... }

    View Slide

  51. $PEFQJUGBMMT3FNPWFE%FQSFDBUFE'VODUJPOT
    51
    @wcgallego
    mysql_* -> mysqli_*
    For example:

    mysql_connect() should now be
    mysqli_connect() or used PDO class
    Legacy extension: https://github.com/php/pecl-database-mysql

    View Slide

  52. 52
    0UIFSJODPNQBUJCMFDIBOHFT
    http://php.net/manual/en/migration70.incompatible.php
    http://php.net/manual/en/migration70.deprecated.php
    @wcgallego

    View Slide

  53. 53
    &YUFOTJPOT
    @wcgallego
    • gearman
    • msgpack
    • oauth
    • memcached
    • gmagick
    •…a dozen more!

    View Slide

  54. 54
    SOAP ’n Certs
    PHP 7 is stricter about cert comparisons
    @wcgallego

    View Slide

  55. 55
    SOAP ’n Certs
    1. Disable WSDL cache

    (http://lornajane.net/posts/2015/soapfault-when-switching-php-versions)
    2. Update your certs!
    3. use FQDN instead of IPs
    4.* Could also turn off peer verification (but probably not advisable)
    @wcgallego

    View Slide

  56. 56
    /6."
    @wcgallego

    View Slide

  57. 57
    On NUMA
    @wcgallego

    View Slide

  58. 58
    But enough of the doom and gloom...
    @wcgallego

    View Slide

  59. /FXGFBUVSFTUPDIFDLPVU
    • Scalar Type Declarations
    • Return Type Declarations
    • NULL coalescing operator
    • phpdbg
    • spaceship operator
    • const arrays
    59
    @wcgallego

    View Slide

  60. 60
    .PSFGFBUVSFT
    http://php.net/manual/en/migration70.new-features.php
    @wcgallego

    View Slide

  61. 'VUVSFUXFBLTBU&UTZ
    • File backed opcache - cli speed up, avoid
    thundering herd on graceful restarts
    • Better compilation! Latest GCC, ICC, etc.
    • FDO (Feedback Directed Optimization) as
    part of GCC > 5
    • Tuning to avoid filling up opcache as often
    61
    @wcgallego

    View Slide

  62. 1BSUJOHUIPVHIUT
    • Plan first, be ready to be flexible
    • Static analyzers/Tests/Teeing are wonderful
    • Have a back up plan
    • Don’t be afraid to contribute back to PHP
    • Get ready for a huge bucket of win
    62
    @wcgallego

    View Slide

  63. 63
    5IBOLT
    @wcgallego
    https://joind.in/talk/41de3

    View Slide

  64. 64
    <>IUUQTXXXGMJDLSDPNQIPUPTECSFLLF
    <>JNBHFIUUQXXXCVTJOFTTJOTJEFSDPNJOTJEFFUTZTOFXQFSLGJMMFEPGGJDF
    <>IUUQTXXXGMJDLSDPNQIPUPTKBOJUPST
    <>IUUQTXXXGMJDLSDPNQIPUPT!/
    <>IUUQTFOXJLJQFEJBPSHXJLJ1FSM
    <>IUUQTXXXGMJDLSDPNQIPUPTDPOCPO
    <>IUUQTXXXGMJDLSDPNQIPUPT!/
    <>IUUQTXXXGMJDLSDPNQIPUPT!/
    <>IUUQTXXXGMJDLSDPNQIPUPTFDPMJ
    @wcgallego

    View Slide