Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dynamic certificate internals with ngx_mruby #nagoyark03

Dynamic certificate internals with ngx_mruby #nagoyark03

Talk for Nagoya Ruby Kaigi 03 (http://regional.rubykaigi.org/nagoya03/) by OKUMURA Takahiro.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[Links in the slides]
mruby: https://github.com/mruby/mruby

200万 Webサイトを支える ロリポップ!と mruby:
https://speakerdeck.com/harasou/200mo-websaitowozhi-eru-roripotupu-to-mruby

haconiwa: https://github.com/haconiwa/haconiwa

ngx_mruby の Nginx::Var クラスの実装を理解する〜変数取得編
http://blog.hifumi.info/2016/11/07/ngx_mruby-nginx-var-using-method-missing/

How to build ngx_mruby:
https://github.com/matsumotory/ngx_mruby/wiki/Install
https://github.com/hsbt/ngx_mruby-package-builder
https://hub.docker.com/r/hfmgarden/ngx_mruby
https://github.com/giraffi/docker-nginx-mruby-base

100行あったmod_rewirteを ngx_mrubyで書き換えた話:
https://speakerdeck.com/buty4649/100xing-atutamod-rewirtewo-ngx-mrubydeshu-kihuan-etahua

goope: https://goope.jp

Too many domains and certificates:
https://gist.github.com/hfm/4a045a429f9303c90eac7c348d1a424a

SSL_CTX_set_cert_cb(): https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_cert_cb.html

Announce that ngx_mruby supports dynamic certificates:
https://twitter.com/matsumotory/status/685341115814289408

ngx_mruby に mruby_ssl_handshake_handler を実装した:
http://blog.hifumi.info/2016/10/03/ngx_mruby-mruby_ssl_handshake_handler/

mruby のテスト用に MySQL 環境を自動で構築する mruby-test-mysqld を書いた
:
http://blog.hifumi.info/2016/09/06/mruby-test-mysqld/

How to test code with mruby:
http://www.slideshare.net/hsbt/20150525-testing-casualtalks

47e8318347fb8cd6a2c39bc6cf6e333e?s=128

Okumura Takahiro

February 13, 2017
Tweet

More Decks by Okumura Takahiro

Other Decks in Programming

Transcript

  1. Dynamic certificate internals with ngx_mruby OKUMURA Takahiro / GMO Pepabo,

    Inc. 2017.2.11 Nagoya Ruby Kaigi 03
  2. Senior web operation engineer at GMO Pepabo, Inc. A maintainer

    of ngx_mruby. http://blog.hifumi.info hfm hfm OKUMURA
 Takahiro (okkun)
  3. Today’s Agenda Introduce mruby/ngx_mruby How to build ngx_mruby How hosting

    too many domains & certificates Dynamic certificate with ngx_mruby
  4. IUUQTTQFBLFSEFDLDPNIGNHNPIPTDPO EXTENDED

  5. mruby

  6. mruby •lightweight and embeddable implementation of the Ruby language •https://github.com/mruby/mruby

  7. Pepabo mruby IUUQTTQFBLFSEFDLDPNIBSBTPVNPXFCTBJUPXP[IJFSVSPSJQPUVQVUPNSVCZ

  8. Pepabo mruby

  9. ngx_mruby

  10. ngx_mruby •A fast and memory-efficient web server extension mechanism using

    mruby for nginx •Created by @matsumotory •An alternative to mod_mruby
  11. ngx_mruby sample code location /hello_world { mruby_content_handler_code ‘ Nginx.rputs "Hello

    ngx_mruby world!" '; } $ curl http://127.0.0.1/hello_world Hello ngx_mruby world!
  12. ngx_mruby sample code v = Nginx::Var.new Nginx.rputs "Date: #{v.date_local}" #=>

    Date: Friday, 14-Oct-2016 02:20:00 JST # Using ngx_http_geoip_module Nginx.rputs "Country: #{v.geoip_country_name}" #=> Country: Japan
  13. ngx_mruby sample code v = Nginx::Var.new Nginx.rputs "Date: #{v.date_local}" #=>

    Date: Friday, 14-Oct-2016 02:20:00 JST # Using ngx_http_geoip_module Nginx.rputs "Country: #{v.geoip_country_name}" #=> Country: Japan Do you want to know Nginx::Var internal? => ngx_mruby ͷ Nginx::Var Ϋϥεͷ࣮૷Λཧղ͢Δʙม਺औಘฤ http://blog.hifumi.info/2016/11/07/ngx_mruby-nginx-var-using-method-missing/
  14. How to build ngx_mruby •build.sh •hsbt/ngx_mruby-package-builder •Dockerfile •nginx-build

  15. build.sh •Generate binary •https://github.com/matsumotory/ngx_mruby/wiki/Install

  16. hsbt/ngx_mruby-package-builder •Providing RPM and DEB packages •CentOS 6, 7 •Ubuntu

    14.04, 16.04 •https://github.com/hsbt/ngx_mruby-package-builder
  17. Dockerfile •https://hub.docker.com/r/hfmgarden/ngx_mruby •Alpine Linux •Ubuntu 16.04

  18. nginx-build •Created by @cubicdaiya •Use case •https://github.com/giraffi/docker-nginx-mruby-base

  19. We use ngx_mruby in production. IUUQTTQFBLFSEFDLDPNCVUZYJOHBUVUBNPESFXJSUFXPOHYNSVCZEFTIVLJIVBOFUBIVB

  20. Preliminary remarks mruby is good ngx_mruby is good We use

    ngx_mruby in production
  21. How hosting too many domains and certificates Certificate Certificate Certificate

  22. Website builder “Goope”

  23. Custom domains with SSL certificates

  24. Two problems •IP address per domain •Web server load too

    many certificates
  25. Two problems •IP address per domain ➡ TLS SNI extension

    •Web server load too many certificates
  26. TLS SNI extension TLS SNI sample-a.com Certificate IP Address Certificate

    IP Address Certificate IP Address sample-b.com sample-c.com IP Address Certificate Certificate Certificate sample-a.com sample-b.com sample-c.com
  27. Two problems •IP address per domain ✓ TLS SNI extension

    •Web server load too many certificates
  28. Two problems •IP address per domain ✓ TLS SNI extension

    •Web server load too many certificates
  29. Too many domains and certificates https://a.jp application N https://b.jp Certificate

    Certificate c’mon
  30. Too many domains and certificates https://a.jp application N https://b.jp https://z.jp

    … Certificate Certificate Certificate Certificate Certificate Certificate oh… Certificate Certificate Certificate
  31. Too many domains and certificates https://a.jp application N https://b.jp https://z.jp

    https://aa.jp … … Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Oops… Certificate Certificate Certificate
  32. Too many domains and certificates https://a.jp application N https://b.jp https://z.jp

    https://aa.jp … … Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate x 10,000 # ps axfo rss,cmd RSS CMD 2872 bash 256168 nginx (master) 258384 \_ nginx (worker) Certificate Certificate Certificate
  33. Too many domains and certificates https://a.jp application N https://b.jp https://z.jp

    https://aa.jp … … Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate Certificate x 10,000 # ps axfo rss,cmd RSS CMD 2824 bash 512808 nginx (master) 511396 \_ nginx (worker) SIGHUP !? Certificate Certificate Certificate Certificate reproduce: https://gist.github.com/hfm/4a045a429f9303c90eac7c348d1a424a
  34. Too many domains and certificates •Heavy load •Large amounts of

    memory
  35. Two problems •IP address per domain ✓ TLS SNI extension

    •Web server load too many certificates
  36. Two problems •IP address per domain ✓ TLS SNI extension

    •Web server load too many certificates ➡ Dynamic certificates
  37. Dynamic certificate with ngx_mruby

  38. SSL_CTX_set_cert_cb() •available in OpenSSL 1.0.2 and later •handle certificate callback

    function •It’s called before certificate will be used by client/server •https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_cert_cb.html
  39. SSL_CTX_set_cert_cb() Client Server ClientHello (server_name) ServerHello Certificate ɾ ɾ ɾ

    awesome_ssl_cert_handler
  40. IUUQTUXJUUFSDPNNBUTVNPUPSZTUBUVT IUUQTHJUIVCDPNNBUTVNPUPSZOHY@NSVCZQVMM ngx_mruby supports it

  41. mruby_ssl_handshake_handler_code # simple pattern (inline) mruby_ssl_handshake_handler_code ' ssl = Nginx::SSL.new

    ssl.certificate = "/path/to/#{ssl.servername}.crt" ssl.certificate_key = "/path/to/#{ssl.servername}.key" ’;
  42. mruby_ssl_handshake_handler # simple pattern (file) mruby_ssl_handshake_handler /path/to/handler.rb cache; # /path/to/handler.rb

    ssl = Nginx::SSL.new ssl.certificate = "/path/to/#{ssl.servername}.crt" ssl.certificate_key = "/path/to/#{ssl.servername}.key"
  43. mruby_ssl_handshake_handler # simple pattern (file) mruby_ssl_handshake_handler /path/to/handler.rb cache; # /path/to/handler.rb

    ssl = Nginx::SSL.new ssl.certificate = "/path/to/#{ssl.servername}.crt" ssl.certificate_key = "/path/to/#{ssl.servername}.key" => ngx_mruby ʹ mruby_ssl_handshake_handler Λ࣮૷ͨ͠ http://blog.hifumi.info/2016/10/03/ngx_mruby-mruby_ssl_handshake_handler/
  44. Two problems •IP address per domain ✓ TLS SNI extention

    •Web server load too many certificates ✓ Dynamic certificates with ngx_mruby
  45. The service architecture for dynamic certificate

  46. Dynamic certificates architecture client application cache db 1. sent server_name

    2. fetch crt/key pair from cache or db 3. proxy 2-a. set cache if missing direct pattern
  47. Dynamic certificates architecture client application cache db 1. sent server_name

    2. fetch crt/key pair 3. proxy internal api api pattern
  48. worker handlers mruby_init_worker /path/to/mruby_init_worker.rb cache; mruby_exit_worker /path/to/mruby_exit_worker.rb cache;

  49. mruby_init_worker.rb conf = Ini.load_file('config.ini') redis = Redis.new(conf['redis']['server'], 6379) Userdata.new(“redis_#{Process.pid}").redis_conn =

    redis mysql = MySQL::Database.new(conf[‘mysql’][‘host’], conf[‘mysql’][‘user’], conf[‘mysql’][‘pass’], conf[‘mysql’]['db']) Userdata.new(“mysql_#{Process.pid}").mysql_conn = mysql mruby_init_worker /path/to/mruby_init_worker.rb cache; mruby_exit_worker /path/to/mruby_exit_worker.rb cache;
  50. Connecting with nginx workers nginx master worker worker redis mysql

  51. mruby_exit_worker.rb # Close all connections redis = Userdata.new("redis_#{Process.pid}").redis_conn redis.close unless

    redis.nil? mysql = Userdata.new("mysql_#{Process.pid}").mysql_conn mysql.close unless mysql.nil? mruby_init_worker /path/to/mruby_init_worker.rb cache; mruby_exit_worker /path/to/mruby_exit_worker.rb cache;
  52. Disconnecting with nginx workers nginx master worker worker redis mysql

  53. Reconnecting with nginx workers nginx master worker worker redis mysql

  54. Reconnecting with nginx workers nginx master worker worker redis mysql

    MySQL server has gone away × ×
  55. Reconnecting with nginx workers nginx master worker worker redis mysql

    Reconnection def mysql_reconnect ... mysql = MySQL::Database.new(host, user, pass, db) Userdata.new("mysql_#{Process.pid}").mysql_conn = mysql end
  56. Testing

  57. hfm/mruby-test-mysqld •mruby porting from Test::mysqld of perl •mysqld runner for

    tests with mruby http://blog.hifumi.info/2016/09/06/mruby-test-mysqld/
  58. Testing sample (1/2) if Object.const_defined?(:MTest) class DynCrtTest < MTest::Unit::TestCase def

    setup @mysqld = TestMysqld.new(database: 'foo') @mysql = MySQL::Database.new '127.0.0.1', 'root', '', 'foo', 3306 end def teardown @mysql.close @mysqld.stop end ...
  59. Testing sample (2/2) if Object.const_defined?(:MTest) ... def test_dyn_crt ... end

    end MTest::Unit.new.run else ... end
  60. IUUQXXXTMJEFTIBSFOFUITCUUFTUJOHDBTVBMUBMLT TFFBMTP

  61. Summary We use ngx_mruby in production ngx_mruby supports dynamic certificates

    using SNI and OpenSSL 1.0.2