Pro Yearly is on sale from $80 to $50! »

Wi-Fi x Scala: Implementing Captive Portal in Scala and deploy into #ScalaMatsuri

1f745ff900e1be51aedae18cae76593c?s=47 Kurochan
June 29, 2019

Wi-Fi x Scala: Implementing Captive Portal in Scala and deploy into #ScalaMatsuri

Scala Matsuri 2019 Unconference

1f745ff900e1be51aedae18cae76593c?s=128

Kurochan

June 29, 2019
Tweet

Transcript

  1. Wi-Fi x Scala:
 Implementing a Captive Portal in Scala and

    deploy into #ScalaMatsuri Yuta Kurosaki AdTech Studio / CyberAgent, Inc.
  2. Did you see this popup? ! 会場でこのポップアップを⾒た⼈?

  3. Yuta Kurosaki • Backend engineer at CyberAgent, Inc. • Scala

    / AWS +years • AdTech • developing DSP (Demand Side Platform) • over K reqs / sec (bid request) • ⽉間数千億リクエストをさばく技術 • http://bit.ly/architecture_night @kuro_m @kurochan
  4. Contents • Captive Portal x Scala • What is Captive

    Portal? • Implementation, Deploy, Operation • Wi-Fi in ScalaMatsuri • Design, Preparation, Deploy, Operation • Troubles ScalaでCaptive Portalを実装した話と、ScalaMatsuriでのWi-Fiについて話します
  5. Captive Portal x Scala

  6. What is Captive Portal?

  7. Captive Portal • Authentication / Authorization System of Free Wi-Fi

    • Often appear popup when you connect Free Wi-Fi 街中のFree Wi-Fiに接続した時にポップアップを⾒たことがあるかもしれません
  8. Captive Portal Detection • Implementation of Captive Portal Detection is

    a few different between iOS/macOS and Android, Windows, Linux, etc... • Example: • When you connect Free Wi-Fi, your device's OS try to access 
 http://check.example.com implicitly. • If your device received contents as excepted, this network can connect to the Internet . • If your device received HTTP Found, network redirects to the Captive Portal. OSによって細かい違いがありますが、Wi-Fi接続時に意図しないHTTPリダイレクトを検知し、
 ポップアップを表⽰するのがCaptive Portal Detectionです
  9. Why did I implement a Captive Portal? • This is

    ScalaMatsuri. I just wantd to use Scala for Wi-Fi. ScalaMatsuriなのでWi-FiにもScalaを組み込みたかったので作ってみました
  10. Implementation

  11. How to Implement a Captive Portal? • For authorized users:

    allow all packets. • For not authorized users: redirect all HTTP traffic to Captive Portal. • I decited to use OpenFlow + Scala for dynamic packet processing. 動的なパケット処理をする必要があるのでOpenFlowとScalaを選択
  12. OpenFlow • One of Software Defined Network Technology. • Control

    Plane and Data Plane model. • Data Plane is either software or hardware. • Major Control Plane implementation • Java: Floodlight, Ruby: Trema, Python: Ryu OpenFlowはSoftware Defined Networkの技術のひとつで、
 ソフトウェアスイッチだけでなく、ハードウェアのスイッチも制御できます https://qiita.com/ttsubo/items/ addd c d adfcf
  13. Implementing an OpenFlow Controller • Used Floodlight for only serialize

    / deserialize OpenFlow Message packet. • Other OpenFlow functions are implemented from scratch by Scala. • TCP connection and streaming is handled by netty. Scala Application OpenFlow Switch OpenFlow Message OpenFlowメッセージの処理の部分にFloodlightを、その他の部分はScalaとnettyを使って実装
  14. Performance Test • Simple Controller: when packet-in, then packet-out. •

    C . xlarge • Mbps • This test is all packet processed through Scala App. • In Actually most packet processing is
 offloaded by writing FlowRule to
 OpenFlow switch. • This performance is enough. Scala Application OpenFlow Message The Internet 全パケットをScalaで処理して200Mbpsでしたが、
 実際はフローがキャッシュされるのでパフォーマンスは⼗分です
  15. Architecture • OpenFlow Gateway: netty • serialize / deserialize OpenFlowMessages.

    • HTTP Server: Akka HTTP • provide CaptivePortal Web and authorize. • Captive Portal Controller: Akka Streams • modify user packet and manage OpenFlow rule. nettyとAkka HTTPとAkka Streamsを使って実装
  16. • serialize / deserialize
 OpenFlowMessages. OpenFlow Gateway OpenFlow GatewayはOpenFlow Switchとの通信と


    メッセージのシリアライズ/デシリアライズを担当
  17. HTTP Server • Redirect all HTTP traffic to Captive Portal.

    • Render Captive Portal Web. • Write into Redis if user is authorized. HTTP ServerはCaptive Portalの画⾯と認可するユーザーの情報をRedisに書き込みます
  18. Captive Portal Controller • Authorized user: • allow all packets.

    • Unauthorized user: • Modify packet destination to "Fake DNS"
 If packet is DNS query. • "Fake DNS" always return
 IP address of Captive Portal • This is why all HTTP traffic of
 unauthorized users directed to Captive Portal. • Akka Streams is suitable for this case because packet processing is streaming. 認可されていないユーザーのDNSの通信は全て"Fake DNS"宛に書き換えることで
 HTTPの通信を強制的にリダイレクトし、Captive Portalにつながるようにします
  19. Case: Authorized user 認証済みユーザーの通信が来た場合 Captive Portal Server The Internet User

    OpenFlow Switch
  20. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch
  21. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In
  22. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Authorized!
  23. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Authorized! Packet Out
  24. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Authorized! Packet Out Flow Cached
  25. Case: Authorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Authorized! Packet Out Flow Cached
  26. Case: Authorized user (second packet) Captive Portal Server The Internet

    User OpenFlow Switch Flow Cached
  27. Case: Authorized user (second packet) Captive Portal Server The Internet

    User OpenFlow Switch Flow Cached
  28. Case: Authorized user (second packet) Captive Portal Server The Internet

    User OpenFlow Switch Flow Cached
  29. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow

    Switch
  30. Case: Unauthorized user Captive Portal Server The Internet User DNS

    packet OpenFlow Switch
  31. Case: Unauthorized user Captive Portal Server The Internet User Packet

    In OpenFlow Switch
  32. Case: Unauthorized user Captive Portal Server The Internet User Packet

    In Unauthorized! OpenFlow Switch
  33. Case: Unauthorized user Captive Portal Server The Internet User Packet

    In Unauthorized! Modify destination to Fake DNS server OpenFlow Switch
  34. Case: Unauthorized user Captive Portal Server The Internet User Packet

    In Unauthorized! Packet Out Modify destination to Fake DNS server OpenFlow Switch
  35. Case: Unauthorized user Captive Portal Server The Internet User Packet

    In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server OpenFlow Switch
  36. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server Fake DNS response
  37. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow

    Switch Packet In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server Fake DNS response Redirect to Captive Portal
  38. Deployment

  39. Deploy into DataCenter • Linux Virtual Machine: Core GB (It

    was too match resource) • The reason why we deployed into datacenter is explain later. 後述の理由により今回はデータセンタのサーバーにデプロイしました
  40. Operation

  41. There was NO Operation! • It Works! Almost... • Known

    Issues • Failed to detect Captive Portal only Samsung Galaxy Series • Workaround: Open some http site (e.g. http://neverssl.com ) instead of Captive Portal Detection. • Registered users!! 特定の端末でどうしても動かなかったのですが、ほぼうまく動作しました!
  42. WiFi

  43. Scala Matsuri Wi-Fi Team • Engineers at CyberAgent, Inc.

  44. Design

  45. Goals • Connected everywhere • It's natural for someone to

    want to Comfortable Internet. • Technical challenge • Not just a work, use as real user traffic lab environment or playground. ただの業務にしたくなかったのと、せっかくなので⾊々なチャレンジをしました
  46. Architecture • Internet connectivity was 
 provided by Home NOC

    Operator's Group
 (AS ) • Scala Matsuri venue and Data Center is
 connected by flet's NGN Home NOC Operator's Groupのデータセンタと接続し、
 ⾼速なインターネット接続を提供して頂きました The Internet AS59105 Scala Matsuri venue
  47. Scala Matsuri Venue • Wi-Fi Access Points x • Cisco

    Aironet x • Cisco Meraki MR x • PoE Switching Hub x • Router x • NEC IX • Server x • Cable • LAN cable: about meters in total ! • Optical fiber cable: meters 会場ではたくさんの機材を設置しました
  48. Preparation

  49. Site survey at May 5⽉の下⾒

  50. Lab environment at our office • We were not expert

    of Wi-Fi. Wi-Fiの運⽤はしたことがなかったので社内に検証環境を作りました
  51. Cabling Plan • There are few time to setup Wi-Fi

    on Day . • To setup quickly, we made a cabling map. 当⽇の構築に使える時間が⾮常に限られてるため、事前にしっかり準備しました
  52. "Wireless is wired" ... • Required total length of LAN

    cable is about meters. • We made cables by hand. 無線LANは有線で繋がっているのでケーブル制作がけっこうたいへんでした…
  53. Deploy

  54. Deploy Wi-Fi APs • I was walked around over k

    steps and over kcals burned 初⽇の消費カロリー
  55. Entrance Hall( F)

  56. Foyer( F)

  57. Conference Hall ( F, F)

  58. Media Hall ( F)

  59. Conference Room ( F)

  60. Operation

  61. Prepare for accidents • Monitoring traffic, Wi-Fi APs, amount of

    clients, etc... • Datadog • ping, DHCP, DNS • Zabbix + Grafana • SNMP 障害に備えて、DatadogとZabbixとGrafanaで監視環境を構築
  62. Traffic • unique clients connected • yesterday: download GB, upload

    GB 2⽇間で828端末がネットワークに接続
  63. Wi-Fi Clients

  64. Troubles

  65. Day

  66. Day • Day used only meeting room of F, so

    it was OK maybe... 初⽇は4Fしか使⽤しなかったので来場者に影響はなかったはず…
  67. Day • Day used only meeting room of F, so

    it was OK maybe... Fixed up quickly 急いで修復⼯事をしました
  68. Morning of Day 2⽇⽬の朝

  69. Morning of Day • 2⽇⽬の朝

  70. Morning of Day • 2⽇⽬の朝

  71. Morning of Day • Cable was dropped out ... どうしてこうなった

  72. Conclusion • Implementing a Captive Portal in Scala. • Now

    open source ! • https://github.com/kurochan/scaptive-portal • Succeeded to provide comfortable Wi-Fi
  73. Thank you for enjoying #ScalaMatsuri and the Internet !