$30 off During Our Annual Pro Sale. View Details »

Dynamic certificate internals with ngx_mruby #nagoyark03

Dynamic certificate internals with ngx_mruby #nagoyark03

Talk for Nagoya Ruby Kaigi 03 (http://regional.rubykaigi.org/nagoya03/) by OKUMURA Takahiro.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[Links in the slides]
mruby: https://github.com/mruby/mruby

200万 Webサイトを支える ロリポップ!と mruby:
https://speakerdeck.com/harasou/200mo-websaitowozhi-eru-roripotupu-to-mruby

haconiwa: https://github.com/haconiwa/haconiwa

ngx_mruby の Nginx::Var クラスの実装を理解する〜変数取得編
http://blog.hifumi.info/2016/11/07/ngx_mruby-nginx-var-using-method-missing/

How to build ngx_mruby:
https://github.com/matsumotory/ngx_mruby/wiki/Install
https://github.com/hsbt/ngx_mruby-package-builder
https://hub.docker.com/r/hfmgarden/ngx_mruby
https://github.com/giraffi/docker-nginx-mruby-base

100行あったmod_rewirteを ngx_mrubyで書き換えた話:
https://speakerdeck.com/buty4649/100xing-atutamod-rewirtewo-ngx-mrubydeshu-kihuan-etahua

goope: https://goope.jp

Too many domains and certificates:
https://gist.github.com/hfm/4a045a429f9303c90eac7c348d1a424a

SSL_CTX_set_cert_cb(): https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_cert_cb.html

Announce that ngx_mruby supports dynamic certificates:
https://twitter.com/matsumotory/status/685341115814289408

ngx_mruby に mruby_ssl_handshake_handler を実装した:
http://blog.hifumi.info/2016/10/03/ngx_mruby-mruby_ssl_handshake_handler/

mruby のテスト用に MySQL 環境を自動で構築する mruby-test-mysqld を書いた
:
http://blog.hifumi.info/2016/09/06/mruby-test-mysqld/

How to test code with mruby:
http://www.slideshare.net/hsbt/20150525-testing-casualtalks

Okumura Takahiro

February 13, 2017
Tweet

More Decks by Okumura Takahiro

Other Decks in Programming

Transcript

  1. Dynamic certificate internals with
    ngx_mruby
    OKUMURA Takahiro / GMO Pepabo, Inc.
    2017.2.11 Nagoya Ruby Kaigi 03

    View Slide

  2. Senior web operation engineer at
    GMO Pepabo, Inc.
    A maintainer of ngx_mruby.
    http://blog.hifumi.info
    hfm
    hfm
    OKUMURA

    Takahiro (okkun)

    View Slide

  3. Today’s Agenda
    Introduce mruby/ngx_mruby
    How to build ngx_mruby
    How hosting too many domains & certificates
    Dynamic certificate with ngx_mruby

    View Slide

  4. IUUQTTQFBLFSEFDLDPNIGNHNPIPTDPO
    EXTENDED

    View Slide

  5. mruby

    View Slide

  6. mruby
    •lightweight and embeddable implementation
    of the Ruby language
    •https://github.com/mruby/mruby

    View Slide

  7. Pepabo mruby
    IUUQTTQFBLFSEFDLDPNIBSBTPVNPXFCTBJUPXP[IJFSVSPSJQPUVQVUPNSVCZ

    View Slide

  8. Pepabo mruby

    View Slide

  9. ngx_mruby

    View Slide

  10. ngx_mruby
    •A fast and memory-efficient web server
    extension mechanism using mruby for nginx
    •Created by @matsumotory
    •An alternative to mod_mruby

    View Slide

  11. ngx_mruby sample code
    location /hello_world {
    mruby_content_handler_code ‘
    Nginx.rputs "Hello ngx_mruby world!"
    ';
    }
    $ curl http://127.0.0.1/hello_world
    Hello ngx_mruby world!

    View Slide

  12. ngx_mruby sample code
    v = Nginx::Var.new
    Nginx.rputs "Date: #{v.date_local}"
    #=> Date: Friday, 14-Oct-2016 02:20:00 JST
    # Using ngx_http_geoip_module
    Nginx.rputs "Country: #{v.geoip_country_name}"
    #=> Country: Japan

    View Slide

  13. ngx_mruby sample code
    v = Nginx::Var.new
    Nginx.rputs "Date: #{v.date_local}"
    #=> Date: Friday, 14-Oct-2016 02:20:00 JST
    # Using ngx_http_geoip_module
    Nginx.rputs "Country: #{v.geoip_country_name}"
    #=> Country: Japan
    Do you want to know Nginx::Var internal?
    => ngx_mruby ͷ Nginx::Var Ϋϥεͷ࣮૷Λཧղ͢Δʙม਺औಘฤ
    http://blog.hifumi.info/2016/11/07/ngx_mruby-nginx-var-using-method-missing/

    View Slide

  14. How to build ngx_mruby
    •build.sh
    •hsbt/ngx_mruby-package-builder
    •Dockerfile
    •nginx-build

    View Slide

  15. build.sh
    •Generate binary
    •https://github.com/matsumotory/ngx_mruby/wiki/Install

    View Slide

  16. hsbt/ngx_mruby-package-builder
    •Providing RPM and DEB packages
    •CentOS 6, 7
    •Ubuntu 14.04, 16.04
    •https://github.com/hsbt/ngx_mruby-package-builder

    View Slide

  17. Dockerfile
    •https://hub.docker.com/r/hfmgarden/ngx_mruby
    •Alpine Linux
    •Ubuntu 16.04

    View Slide

  18. nginx-build
    •Created by @cubicdaiya
    •Use case
    •https://github.com/giraffi/docker-nginx-mruby-base

    View Slide

  19. We use ngx_mruby in production.
    IUUQTTQFBLFSEFDLDPNCVUZYJOHBUVUBNPESFXJSUFXPOHYNSVCZEFTIVLJIVBOFUBIVB

    View Slide

  20. Preliminary remarks
    mruby is good
    ngx_mruby is good
    We use ngx_mruby in production

    View Slide

  21. How hosting
    too many
    domains and certificates
    Certificate
    Certificate
    Certificate

    View Slide

  22. Website builder “Goope”

    View Slide

  23. Custom domains with SSL certificates

    View Slide

  24. Two problems
    •IP address per domain
    •Web server load too many certificates

    View Slide

  25. Two problems
    •IP address per domain
    ➡ TLS SNI extension
    •Web server load too many certificates

    View Slide

  26. TLS SNI extension
    TLS
    SNI
    sample-a.com
    Certificate IP Address
    Certificate IP Address
    Certificate IP Address
    sample-b.com
    sample-c.com
    IP Address
    Certificate
    Certificate
    Certificate
    sample-a.com
    sample-b.com
    sample-c.com

    View Slide

  27. Two problems
    •IP address per domain
    ✓ TLS SNI extension
    •Web server load too many certificates

    View Slide

  28. Two problems
    •IP address per domain
    ✓ TLS SNI extension
    •Web server load too many certificates

    View Slide

  29. Too many domains and certificates
    https://a.jp
    application
    N
    https://b.jp
    Certificate
    Certificate
    c’mon

    View Slide

  30. Too many domains and certificates
    https://a.jp
    application
    N
    https://b.jp https://z.jp

    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate

    oh… Certificate
    Certificate
    Certificate

    View Slide

  31. Too many domains and certificates
    https://a.jp
    application
    N
    https://b.jp https://z.jp https://aa.jp
    … …
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate

    Oops…

    Certificate
    Certificate
    Certificate

    View Slide

  32. Too many domains and certificates
    https://a.jp
    application
    N
    https://b.jp https://z.jp https://aa.jp
    … …
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate

    x 10,000
    # ps axfo rss,cmd
    RSS CMD
    2872 bash
    256168 nginx (master)
    258384 \_ nginx (worker)
    Certificate
    Certificate
    Certificate

    View Slide

  33. Too many domains and certificates
    https://a.jp
    application
    N
    https://b.jp https://z.jp https://aa.jp
    … …
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate
    Certificate

    x 10,000
    # ps axfo rss,cmd
    RSS CMD
    2824 bash
    512808 nginx (master)
    511396 \_ nginx (worker)
    SIGHUP
    !?
    Certificate
    Certificate
    Certificate
    Certificate

    reproduce: https://gist.github.com/hfm/4a045a429f9303c90eac7c348d1a424a

    View Slide

  34. Too many domains and certificates
    •Heavy load
    •Large amounts of memory

    View Slide

  35. Two problems
    •IP address per domain
    ✓ TLS SNI extension
    •Web server load too many certificates

    View Slide

  36. Two problems
    •IP address per domain
    ✓ TLS SNI extension
    •Web server load too many certificates
    ➡ Dynamic certificates

    View Slide

  37. Dynamic certificate
    with
    ngx_mruby

    View Slide

  38. SSL_CTX_set_cert_cb()
    •available in OpenSSL 1.0.2 and later
    •handle certificate callback function
    •It’s called before certificate will be used by
    client/server
    •https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_cert_cb.html

    View Slide

  39. SSL_CTX_set_cert_cb()
    Client Server
    ClientHello
    (server_name)
    ServerHello
    Certificate
    ɾ
    ɾ
    ɾ
    awesome_ssl_cert_handler

    View Slide

  40. IUUQTUXJUUFSDPNNBUTVNPUPSZTUBUVT IUUQTHJUIVCDPNNBUTVNPUPSZOHY@NSVCZQVMM
    ngx_mruby supports it

    View Slide

  41. mruby_ssl_handshake_handler_code
    # simple pattern (inline)
    mruby_ssl_handshake_handler_code '
    ssl = Nginx::SSL.new
    ssl.certificate = "/path/to/#{ssl.servername}.crt"
    ssl.certificate_key = "/path/to/#{ssl.servername}.key"
    ’;

    View Slide

  42. mruby_ssl_handshake_handler
    # simple pattern (file)
    mruby_ssl_handshake_handler /path/to/handler.rb cache;
    # /path/to/handler.rb
    ssl = Nginx::SSL.new
    ssl.certificate = "/path/to/#{ssl.servername}.crt"
    ssl.certificate_key = "/path/to/#{ssl.servername}.key"

    View Slide

  43. mruby_ssl_handshake_handler
    # simple pattern (file)
    mruby_ssl_handshake_handler /path/to/handler.rb cache;
    # /path/to/handler.rb
    ssl = Nginx::SSL.new
    ssl.certificate = "/path/to/#{ssl.servername}.crt"
    ssl.certificate_key = "/path/to/#{ssl.servername}.key"
    => ngx_mruby ʹ mruby_ssl_handshake_handler Λ࣮૷ͨ͠
    http://blog.hifumi.info/2016/10/03/ngx_mruby-mruby_ssl_handshake_handler/

    View Slide

  44. Two problems
    •IP address per domain
    ✓ TLS SNI extention
    •Web server load too many certificates
    ✓ Dynamic certificates with ngx_mruby

    View Slide

  45. The service architecture
    for
    dynamic certificate

    View Slide

  46. Dynamic certificates architecture
    client
    application
    cache
    db
    1. sent server_name
    2. fetch crt/key pair from cache or db
    3. proxy
    2-a. set cache if missing
    direct pattern

    View Slide

  47. Dynamic certificates architecture
    client
    application
    cache
    db
    1. sent server_name
    2. fetch crt/key pair
    3. proxy
    internal api
    api pattern

    View Slide

  48. worker handlers
    mruby_init_worker /path/to/mruby_init_worker.rb cache;
    mruby_exit_worker /path/to/mruby_exit_worker.rb cache;

    View Slide

  49. mruby_init_worker.rb
    conf = Ini.load_file('config.ini')
    redis = Redis.new(conf['redis']['server'], 6379)
    Userdata.new(“redis_#{Process.pid}").redis_conn = redis
    mysql = MySQL::Database.new(conf[‘mysql’][‘host’],
    conf[‘mysql’][‘user’], conf[‘mysql’][‘pass’],
    conf[‘mysql’]['db'])
    Userdata.new(“mysql_#{Process.pid}").mysql_conn = mysql
    mruby_init_worker /path/to/mruby_init_worker.rb cache;
    mruby_exit_worker /path/to/mruby_exit_worker.rb cache;

    View Slide

  50. Connecting with nginx workers
    nginx master
    worker
    worker
    redis
    mysql

    View Slide

  51. mruby_exit_worker.rb
    # Close all connections
    redis = Userdata.new("redis_#{Process.pid}").redis_conn
    redis.close unless redis.nil?
    mysql = Userdata.new("mysql_#{Process.pid}").mysql_conn
    mysql.close unless mysql.nil?
    mruby_init_worker /path/to/mruby_init_worker.rb cache;
    mruby_exit_worker /path/to/mruby_exit_worker.rb cache;

    View Slide

  52. Disconnecting with nginx workers
    nginx master
    worker
    worker
    redis
    mysql

    View Slide

  53. Reconnecting with nginx workers
    nginx master
    worker
    worker
    redis
    mysql

    View Slide

  54. Reconnecting with nginx workers
    nginx master
    worker
    worker
    redis
    mysql
    MySQL server has gone away
    ×
    ×

    View Slide

  55. Reconnecting with nginx workers
    nginx master
    worker
    worker
    redis
    mysql
    Reconnection
    def mysql_reconnect
    ...
    mysql = MySQL::Database.new(host, user, pass, db)
    Userdata.new("mysql_#{Process.pid}").mysql_conn = mysql
    end

    View Slide

  56. Testing

    View Slide

  57. hfm/mruby-test-mysqld
    •mruby porting from Test::mysqld of perl
    •mysqld runner for tests with mruby
    http://blog.hifumi.info/2016/09/06/mruby-test-mysqld/

    View Slide

  58. Testing sample (1/2)
    if Object.const_defined?(:MTest)
    class DynCrtTest < MTest::Unit::TestCase
    def setup
    @mysqld = TestMysqld.new(database: 'foo')
    @mysql = MySQL::Database.new '127.0.0.1', 'root',
    '', 'foo', 3306
    end
    def teardown
    @mysql.close
    @mysqld.stop
    end
    ...

    View Slide

  59. Testing sample (2/2)
    if Object.const_defined?(:MTest)
    ...
    def test_dyn_crt
    ...
    end
    end
    MTest::Unit.new.run
    else
    ...
    end

    View Slide

  60. IUUQXXXTMJEFTIBSFOFUITCUUFTUJOHDBTVBMUBMLT
    TFFBMTP

    View Slide

  61. Summary
    We use ngx_mruby in production
    ngx_mruby supports dynamic certificates
    using SNI and OpenSSL 1.0.2

    View Slide