Wi-Fi x Scala: Implementing Captive Portal in Scala and deploy into #ScalaMatsuri

Scala Matsuri 2019 Unconference

Transcript

1. Wi-Fi x Scala:
 Implementing a Captive Portal in Scala and deploy into #ScalaMatsuri Yuta Kurosaki AdTech Studio / CyberAgent, Inc.
2. Did you see this popup? ! 会場でこのポップアップを⾒た⼈？
3. Yuta Kurosaki • Backend engineer at CyberAgent, Inc. • Scala / AWS +years • AdTech • developing DSP (Demand Side Platform) • over K reqs / sec (bid request) • ⽉間数千億リクエストをさばく技術 • http://bit.ly/architecture_night @kuro_m @kurochan
4. Contents • Captive Portal x Scala • What is Captive Portal? • Implementation, Deploy, Operation • Wi-Fi in ScalaMatsuri • Design, Preparation, Deploy, Operation • Troubles ScalaでCaptive Portalを実装した話と、ScalaMatsuriでのWi-Fiについて話します
5. Captive Portal x Scala
6. What is Captive Portal?
7. Captive Portal • Authentication / Authorization System of Free Wi-Fi • Often appear popup when you connect Free Wi-Fi 街中のFree Wi-Fiに接続した時にポップアップを⾒たことがあるかもしれません
8. Captive Portal Detection • Implementation of Captive Portal Detection is a few different between iOS/macOS and Android, Windows, Linux, etc... • Example: • When you connect Free Wi-Fi, your device's OS try to access 
 http://check.example.com implicitly. • If your device received contents as excepted, this network can connect to the Internet . • If your device received HTTP Found, network redirects to the Captive Portal. OSによって細かい違いがありますが、Wi-Fi接続時に意図しないHTTPリダイレクトを検知し、
 ポップアップを表⽰するのがCaptive Portal Detectionです
9. Why did I implement a Captive Portal? • This is ScalaMatsuri. I just wantd to use Scala for Wi-Fi. ScalaMatsuriなのでWi-FiにもScalaを組み込みたかったので作ってみました
10. Implementation
11. How to Implement a Captive Portal? • For authorized users: allow all packets. • For not authorized users: redirect all HTTP traffic to Captive Portal. • I decited to use OpenFlow + Scala for dynamic packet processing. 動的なパケット処理をする必要があるのでOpenFlowとScalaを選択
12. OpenFlow • One of Software Defined Network Technology. • Control Plane and Data Plane model. • Data Plane is either software or hardware. • Major Control Plane implementation • Java: Floodlight, Ruby: Trema, Python: Ryu OpenFlowはSoftware Defined Networkの技術のひとつで、
 ソフトウェアスイッチだけでなく、ハードウェアのスイッチも制御できます https://qiita.com/ttsubo/items/ addd c d adfcf
13. Implementing an OpenFlow Controller • Used Floodlight for only serialize / deserialize OpenFlow Message packet. • Other OpenFlow functions are implemented from scratch by Scala. • TCP connection and streaming is handled by netty. Scala Application OpenFlow Switch OpenFlow Message OpenFlowメッセージの処理の部分にFloodlightを、その他の部分はScalaとnettyを使って実装
14. Performance Test • Simple Controller: when packet-in, then packet-out. • C . xlarge • Mbps • This test is all packet processed through Scala App. • In Actually most packet processing is
 offloaded by writing FlowRule to
 OpenFlow switch. • This performance is enough. Scala Application OpenFlow Message The Internet 全パケットをScalaで処理して200Mbpsでしたが、
 実際はフローがキャッシュされるのでパフォーマンスは⼗分です
15. Architecture • OpenFlow Gateway: netty • serialize / deserialize OpenFlowMessages. • HTTP Server: Akka HTTP • provide CaptivePortal Web and authorize. • Captive Portal Controller: Akka Streams • modify user packet and manage OpenFlow rule. nettyとAkka HTTPとAkka Streamsを使って実装
16. • serialize / deserialize
 OpenFlowMessages. OpenFlow Gateway OpenFlow GatewayはOpenFlow Switchとの通信と
 メッセージのシリアライズ/デシリアライズを担当
17. HTTP Server • Redirect all HTTP traffic to Captive Portal. • Render Captive Portal Web. • Write into Redis if user is authorized. HTTP ServerはCaptive Portalの画⾯と認可するユーザーの情報をRedisに書き込みます
18. Captive Portal Controller • Authorized user: • allow all packets. • Unauthorized user: • Modify packet destination to "Fake DNS"
 If packet is DNS query. • "Fake DNS" always return
 IP address of Captive Portal • This is why all HTTP traffic of
 unauthorized users directed to Captive Portal. • Akka Streams is suitable for this case because packet processing is streaming. 認可されていないユーザーのDNSの通信は全て"Fake DNS"宛に書き換えることで
 HTTPの通信を強制的にリダイレクトし、Captive Portalにつながるようにします
19. Case: Authorized user 認証済みユーザーの通信が来た場合 Captive Portal Server The Internet User OpenFlow Switch
20. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch
21. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch Packet In
22. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Authorized!
23. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Authorized! Packet Out
24. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Authorized! Packet Out Flow Cached
25. Case: Authorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Authorized! Packet Out Flow Cached
26. Case: Authorized user (second packet) Captive Portal Server The Internet User OpenFlow Switch Flow Cached
27. Case: Authorized user (second packet) Captive Portal Server The Internet User OpenFlow Switch Flow Cached
28. Case: Authorized user (second packet) Captive Portal Server The Internet User OpenFlow Switch Flow Cached
29. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow Switch
30. Case: Unauthorized user Captive Portal Server The Internet User DNS packet OpenFlow Switch
31. Case: Unauthorized user Captive Portal Server The Internet User Packet In OpenFlow Switch
32. Case: Unauthorized user Captive Portal Server The Internet User Packet In Unauthorized! OpenFlow Switch
33. Case: Unauthorized user Captive Portal Server The Internet User Packet In Unauthorized! Modify destination to Fake DNS server OpenFlow Switch
34. Case: Unauthorized user Captive Portal Server The Internet User Packet In Unauthorized! Packet Out Modify destination to Fake DNS server OpenFlow Switch
35. Case: Unauthorized user Captive Portal Server The Internet User Packet In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server OpenFlow Switch
36. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server Fake DNS response
37. Case: Unauthorized user Captive Portal Server The Internet User OpenFlow Switch Packet In Unauthorized! Modify destination to Fake DNS server Packet Out Fake DNS Server Fake DNS response Redirect to Captive Portal
38. Deployment
39. Deploy into DataCenter • Linux Virtual Machine: Core GB (It was too match resource) • The reason why we deployed into datacenter is explain later. 後述の理由により今回はデータセンタのサーバーにデプロイしました
40. Operation
41. There was NO Operation! • It Works! Almost... • Known Issues • Failed to detect Captive Portal only Samsung Galaxy Series • Workaround: Open some http site (e.g. http://neverssl.com ) instead of Captive Portal Detection. • Registered users!! 特定の端末でどうしても動かなかったのですが、ほぼうまく動作しました！
42. WiFi
43. Scala Matsuri Wi-Fi Team • Engineers at CyberAgent, Inc.
44. Design
45. Goals • Connected everywhere • It's natural for someone to want to Comfortable Internet. • Technical challenge • Not just a work, use as real user traffic lab environment or playground. ただの業務にしたくなかったのと、せっかくなので⾊々なチャレンジをしました
46. Architecture • Internet connectivity was 
 provided by Home NOC Operator's Group
 (AS ) • Scala Matsuri venue and Data Center is
 connected by flet's NGN Home NOC Operator's Groupのデータセンタと接続し、
 ⾼速なインターネット接続を提供して頂きました The Internet AS59105 Scala Matsuri venue
47. Scala Matsuri Venue • Wi-Fi Access Points x • Cisco Aironet x • Cisco Meraki MR x • PoE Switching Hub x • Router x • NEC IX • Server x • Cable • LAN cable: about meters in total ! • Optical fiber cable: meters 会場ではたくさんの機材を設置しました
48. Preparation
49. Site survey at May 5⽉の下⾒
50. Lab environment at our office • We were not expert of Wi-Fi. Wi-Fiの運⽤はしたことがなかったので社内に検証環境を作りました
51. Cabling Plan • There are few time to setup Wi-Fi on Day . • To setup quickly, we made a cabling map. 当⽇の構築に使える時間が⾮常に限られてるため、事前にしっかり準備しました
52. "Wireless is wired" ... • Required total length of LAN cable is about meters. • We made cables by hand. 無線LANは有線で繋がっているのでケーブル制作がけっこうたいへんでした…
53. Deploy
54. Deploy Wi-Fi APs • I was walked around over k steps and over kcals burned 初⽇の消費カロリー
55. Entrance Hall( F)
56. Foyer( F)
57. Conference Hall ( F, F)
58. Media Hall ( F)
59. Conference Room ( F)
60. Operation
61. Prepare for accidents • Monitoring traffic, Wi-Fi APs, amount of clients, etc... • Datadog • ping, DHCP, DNS • Zabbix + Grafana • SNMP 障害に備えて、DatadogとZabbixとGrafanaで監視環境を構築
62. Traffic • unique clients connected • yesterday: download GB, upload GB 2⽇間で828端末がネットワークに接続
63. Wi-Fi Clients
64. Troubles
65. Day
66. Day • Day used only meeting room of F, so it was OK maybe... 初⽇は4Fしか使⽤しなかったので来場者に影響はなかったはず…
67. Day • Day used only meeting room of F, so it was OK maybe... Fixed up quickly 急いで修復⼯事をしました
68. Morning of Day 2⽇⽬の朝
69. Morning of Day • 2⽇⽬の朝
70. Morning of Day • 2⽇⽬の朝
71. Morning of Day • Cable was dropped out ... どうしてこうなった
72. Conclusion • Implementing a Captive Portal in Scala. • Now open source ! • https://github.com/kurochan/scaptive-portal • Succeeded to provide comfortable Wi-Fi
73. Thank you for enjoying #ScalaMatsuri and the Internet !