Slide 1
Slide 1 text
Wi-Fi x Scala:
Implementing a Captive Portal in Scala
and deploy into #ScalaMatsuri
Yuta Kurosaki
AdTech Studio / CyberAgent, Inc.
Slide 2
Slide 2 text
Did you see this popup? !
会場でこのポップアップを⾒た⼈？
Slide 3
Slide 3 text
Yuta Kurosaki
• Backend engineer at CyberAgent, Inc.
• Scala / AWS +years
• AdTech
• developing DSP (Demand Side Platform)
• over K reqs / sec (bid request)
• ⽉間数千億リクエストをさばく技術
• http://bit.ly/architecture_night
@kuro_m @kurochan
Slide 4
Slide 4 text
Contents
• Captive Portal x Scala
• What is Captive Portal?
• Implementation, Deploy, Operation
• Wi-Fi in ScalaMatsuri
• Design, Preparation, Deploy, Operation
• Troubles
ScalaでCaptive Portalを実装した話と、ScalaMatsuriでのWi-Fiについて話します
Slide 5
Slide 5 text
Captive Portal x Scala
Slide 6
Slide 6 text
What is Captive Portal?
Slide 7
Slide 7 text
Captive Portal
• Authentication / Authorization System of Free Wi-Fi
• Often appear popup when you connect Free Wi-Fi
街中のFree Wi-Fiに接続した時にポップアップを⾒たことがあるかもしれません
Slide 8
Slide 8 text
Captive Portal Detection
• Implementation of Captive Portal Detection is a few different between
iOS/macOS and Android, Windows, Linux, etc...
• Example:
• When you connect Free Wi-Fi, your device's OS try to access
http://check.example.com implicitly.
• If your device received contents as excepted, this network can connect to the Internet .
• If your device received HTTP Found, network redirects to the Captive Portal.
OSによって細かい違いがありますが、Wi-Fi接続時に意図しないHTTPリダイレクトを検知し、
ポップアップを表⽰するのがCaptive Portal Detectionです
Slide 9
Slide 9 text
Why did I implement a Captive Portal?
• This is ScalaMatsuri. I just wantd to use Scala for Wi-Fi.
ScalaMatsuriなのでWi-FiにもScalaを組み込みたかったので作ってみました
Slide 10
Slide 10 text
Implementation
Slide 11
Slide 11 text
How to Implement a Captive Portal?
• For authorized users: allow all packets.
• For not authorized users: redirect all HTTP traffic to Captive Portal.
• I decited to use OpenFlow + Scala for dynamic packet processing.
動的なパケット処理をする必要があるのでOpenFlowとScalaを選択
Slide 12
Slide 12 text
OpenFlow
• One of Software Defined Network Technology.
• Control Plane and Data Plane model.
• Data Plane is either software or hardware.
• Major Control Plane implementation
• Java: Floodlight, Ruby: Trema, Python: Ryu
OpenFlowはSoftware Defined Networkの技術のひとつで、
ソフトウェアスイッチだけでなく、ハードウェアのスイッチも制御できます
https://qiita.com/ttsubo/items/ addd c d adfcf
Slide 13
Slide 13 text
Implementing an OpenFlow Controller
• Used Floodlight for only serialize / deserialize OpenFlow Message packet.
• Other OpenFlow functions are implemented from scratch by Scala.
• TCP connection and streaming is handled by netty.
Scala Application OpenFlow Switch
OpenFlow Message
OpenFlowメッセージの処理の部分にFloodlightを、その他の部分はScalaとnettyを使って実装
Slide 14
Slide 14 text
Performance Test
• Simple Controller: when packet-in, then packet-out.
• C . xlarge
• Mbps
• This test is all packet processed through Scala App.
• In Actually most packet processing is
offloaded by writing FlowRule to
OpenFlow switch.
• This performance is enough. Scala Application
OpenFlow Message
The Internet
全パケットをScalaで処理して200Mbpsでしたが、
実際はフローがキャッシュされるのでパフォーマンスは⼗分です
Slide 15
Slide 15 text
Architecture
• OpenFlow Gateway: netty
• serialize / deserialize OpenFlowMessages.
• HTTP Server: Akka HTTP
• provide CaptivePortal Web and authorize.
• Captive Portal Controller: Akka Streams
• modify user packet and manage OpenFlow rule.
nettyとAkka HTTPとAkka Streamsを使って実装
Slide 16
Slide 16 text
• serialize / deserialize
OpenFlowMessages.
OpenFlow Gateway
OpenFlow GatewayはOpenFlow Switchとの通信と
メッセージのシリアライズ/デシリアライズを担当
Slide 17
Slide 17 text
HTTP Server
• Redirect all HTTP traffic to Captive Portal.
• Render Captive Portal Web.
• Write into Redis if user is authorized.
HTTP ServerはCaptive Portalの画⾯と認可するユーザーの情報をRedisに書き込みます
Slide 18
Slide 18 text
Captive Portal Controller
• Authorized user:
• allow all packets.
• Unauthorized user:
• Modify packet destination to "Fake DNS"
If packet is DNS query.
• "Fake DNS" always return
IP address of Captive Portal
• This is why all HTTP traffic of
unauthorized users directed to Captive Portal.
• Akka Streams is suitable for this case because packet processing is streaming.
認可されていないユーザーのDNSの通信は全て"Fake DNS"宛に書き換えることで
HTTPの通信を強制的にリダイレクトし、Captive Portalにつながるようにします
Slide 19
Slide 19 text
Case: Authorized user
認証済みユーザーの通信が来た場合
Captive Portal Server
The Internet
User
OpenFlow Switch
Slide 20
Slide 20 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Slide 21
Slide 21 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Slide 22
Slide 22 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Authorized!
Slide 23
Slide 23 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Authorized!
Packet Out
Slide 24
Slide 24 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Authorized!
Packet Out
Flow Cached
Slide 25
Slide 25 text
Case: Authorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Authorized!
Packet Out
Flow Cached
Slide 26
Slide 26 text
Case: Authorized user (second packet)
Captive Portal Server
The Internet
User
OpenFlow Switch
Flow Cached
Slide 27
Slide 27 text
Case: Authorized user (second packet)
Captive Portal Server
The Internet
User
OpenFlow Switch
Flow Cached
Slide 28
Slide 28 text
Case: Authorized user (second packet)
Captive Portal Server
The Internet
User
OpenFlow Switch
Flow Cached
Slide 29
Slide 29 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Slide 30
Slide 30 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
DNS packet
OpenFlow Switch
Slide 31
Slide 31 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
Packet In
OpenFlow Switch
Slide 32
Slide 32 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
Packet In
Unauthorized!
OpenFlow Switch
Slide 33
Slide 33 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
Packet In
Unauthorized!
Modify destination
to Fake DNS server
OpenFlow Switch
Slide 34
Slide 34 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
Packet In
Unauthorized!
Packet Out
Modify destination
to Fake DNS server
OpenFlow Switch
Slide 35
Slide 35 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
Packet In
Unauthorized!
Modify destination
to Fake DNS server
Packet Out
Fake DNS Server
OpenFlow Switch
Slide 36
Slide 36 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Unauthorized!
Modify destination
to Fake DNS server
Packet Out
Fake DNS Server
Fake DNS response
Slide 37
Slide 37 text
Case: Unauthorized user
Captive Portal Server
The Internet
User
OpenFlow Switch
Packet In
Unauthorized!
Modify destination
to Fake DNS server
Packet Out
Fake DNS Server
Fake DNS response
Redirect to Captive Portal
Slide 38
Slide 38 text
Deployment
Slide 39
Slide 39 text
Deploy into DataCenter
• Linux Virtual Machine: Core GB (It was too match resource)
• The reason why we deployed into datacenter is explain later.
後述の理由により今回はデータセンタのサーバーにデプロイしました
Slide 40
Slide 40 text
Operation
Slide 41
Slide 41 text
There was NO Operation!
• It Works! Almost...
• Known Issues
• Failed to detect Captive Portal only Samsung Galaxy Series
• Workaround: Open some http site (e.g. http://neverssl.com ) instead of Captive Portal Detection.
• Registered users!!
特定の端末でどうしても動かなかったのですが、ほぼうまく動作しました！
Slide 42
Slide 42 text
WiFi
Slide 43
Slide 43 text
Scala Matsuri Wi-Fi Team
• Engineers at CyberAgent, Inc.
Slide 44
Slide 44 text
Design
Slide 45
Slide 45 text
Goals
• Connected everywhere
• It's natural for someone to want to Comfortable Internet.
• Technical challenge
• Not just a work, use as real user traffic lab environment or playground.
ただの業務にしたくなかったのと、せっかくなので⾊々なチャレンジをしました
Slide 46
Slide 46 text
Architecture
• Internet connectivity was
provided by Home NOC Operator's Group
(AS )
• Scala Matsuri venue and Data Center is
connected by flet's NGN
Home NOC Operator's Groupのデータセンタと接続し、
⾼速なインターネット接続を提供して頂きました
The Internet
AS59105
Scala Matsuri venue
Slide 47
Slide 47 text
Scala Matsuri Venue
• Wi-Fi Access Points x
• Cisco Aironet x
• Cisco Meraki MR x
• PoE Switching Hub x
• Router x
• NEC IX
• Server x
• Cable
• LAN cable: about meters in total !
• Optical fiber cable: meters
会場ではたくさんの機材を設置しました
Slide 48
Slide 48 text
Preparation
Slide 49
Slide 49 text
Site survey at May
5⽉の下⾒
Slide 50
Slide 50 text
Lab environment at our office
• We were not expert of Wi-Fi.
Wi-Fiの運⽤はしたことがなかったので社内に検証環境を作りました
Slide 51
Slide 51 text
Cabling Plan
• There are few time to setup Wi-Fi on Day .
• To setup quickly, we made a cabling map.
当⽇の構築に使える時間が⾮常に限られてるため、事前にしっかり準備しました
Slide 52
Slide 52 text
"Wireless is wired" ...
• Required total length of LAN cable is about meters.
• We made cables by hand.
無線LANは有線で繋がっているのでケーブル制作がけっこうたいへんでした…
Slide 53
Slide 53 text
Deploy
Slide 54
Slide 54 text
Deploy Wi-Fi APs
• I was walked around over k steps and over kcals burned
初⽇の消費カロリー
Slide 55
Slide 55 text
Entrance Hall( F)
Slide 56
Slide 56 text
Foyer( F)
Slide 57
Slide 57 text
Conference Hall ( F, F)
Slide 58
Slide 58 text
Media Hall ( F)
Slide 59
Slide 59 text
Conference Room ( F)
Slide 60
Slide 60 text
Operation
Slide 61
Slide 61 text
Prepare for accidents
• Monitoring traffic, Wi-Fi APs, amount of clients, etc...
• Datadog
• ping, DHCP, DNS
• Zabbix + Grafana
• SNMP
障害に備えて、DatadogとZabbixとGrafanaで監視環境を構築
Slide 62
Slide 62 text
Traffic
• unique clients connected
• yesterday: download GB, upload GB
2⽇間で828端末がネットワークに接続
Slide 63
Slide 63 text
Wi-Fi Clients
Slide 64
Slide 64 text
Troubles
Slide 65
Slide 65 text
Day
Slide 66
Slide 66 text
Day
• Day used only meeting room of F, so it was OK maybe...
初⽇は4Fしか使⽤しなかったので来場者に影響はなかったはず…
Slide 67
Slide 67 text
Day
• Day used only meeting room of F, so it was OK maybe...
Fixed up quickly
急いで修復⼯事をしました
Slide 68
Slide 68 text
Morning of Day
2⽇⽬の朝
Slide 69
Slide 69 text
Morning of Day
•
2⽇⽬の朝
Slide 70
Slide 70 text
Morning of Day
•
2⽇⽬の朝
Slide 71
Slide 71 text
Morning of Day
• Cable was dropped out ...
どうしてこうなった
Slide 72
Slide 72 text
Conclusion
• Implementing a Captive Portal in Scala.
• Now open source !
• https://github.com/kurochan/scaptive-portal
• Succeeded to provide comfortable Wi-Fi
Slide 73
Slide 73 text
Thank you for enjoying
#ScalaMatsuri and the Internet !