Wi-Fi x Scala: Implementing Captive Portal in Scala and deploy into #ScalaMatsuri

Scala Matsuri 2019 Unconference

Transcript

1. Wi-Fi x Scala:

   Implementing a Captive Portal in Scala
   and deploy into #ScalaMatsuri
   Yuta Kurosaki
   AdTech Studio / CyberAgent, Inc.
   

   View Slide

2. Did you see this popup? !
   会場でこのポップアップを⾒た⼈？
   

   View Slide

3. Yuta Kurosaki
   • Backend engineer at CyberAgent, Inc.
   • Scala / AWS +years
   • AdTech
   • developing DSP (Demand Side Platform)
   • over K reqs / sec (bid request)
   • ⽉間数千億リクエストをさばく技術
   • http://bit.ly/architecture_night
   @kuro_m @kurochan
   

   View Slide

4. Contents
   • Captive Portal x Scala
   • What is Captive Portal?
   • Implementation, Deploy, Operation
   • Wi-Fi in ScalaMatsuri
   • Design, Preparation, Deploy, Operation
   • Troubles
   ScalaでCaptive Portalを実装した話と、ScalaMatsuriでのWi-Fiについて話します
   

   View Slide

5. Captive Portal x Scala
   

   View Slide

6. What is Captive Portal?
   

   View Slide

7. Captive Portal
   • Authentication / Authorization System of Free Wi-Fi
   • Often appear popup when you connect Free Wi-Fi
   街中のFree Wi-Fiに接続した時にポップアップを⾒たことがあるかもしれません
   

   View Slide

8. Captive Portal Detection
   • Implementation of Captive Portal Detection is a few different between
   iOS/macOS and Android, Windows, Linux, etc...
   • Example:
   • When you connect Free Wi-Fi, your device's OS try to access 

   http://check.example.com implicitly.
   • If your device received contents as excepted, this network can connect to the Internet .
   • If your device received HTTP Found, network redirects to the Captive Portal.
   OSによって細かい違いがありますが、Wi-Fi接続時に意図しないHTTPリダイレクトを検知し、

   ポップアップを表⽰するのがCaptive Portal Detectionです
   

   View Slide

9. Why did I implement a Captive Portal?
   • This is ScalaMatsuri. I just wantd to use Scala for Wi-Fi.
   ScalaMatsuriなのでWi-FiにもScalaを組み込みたかったので作ってみました
   

   View Slide

10. Implementation
    

    View Slide

11. How to Implement a Captive Portal?
    • For authorized users: allow all packets.
    • For not authorized users: redirect all HTTP traffic to Captive Portal.
    • I decited to use OpenFlow + Scala for dynamic packet processing.
    動的なパケット処理をする必要があるのでOpenFlowとScalaを選択
    

    View Slide

12. OpenFlow
    • One of Software Defined Network Technology.
    • Control Plane and Data Plane model.
    • Data Plane is either software or hardware.
    • Major Control Plane implementation
    • Java: Floodlight, Ruby: Trema, Python: Ryu
    OpenFlowはSoftware Defined Networkの技術のひとつで、

    ソフトウェアスイッチだけでなく、ハードウェアのスイッチも制御できます
    https://qiita.com/ttsubo/items/ addd c d adfcf
    

    View Slide

13. Implementing an OpenFlow Controller
    • Used Floodlight for only serialize / deserialize OpenFlow Message packet.
    • Other OpenFlow functions are implemented from scratch by Scala.
    • TCP connection and streaming is handled by netty.
    Scala Application OpenFlow Switch
    OpenFlow Message
    OpenFlowメッセージの処理の部分にFloodlightを、その他の部分はScalaとnettyを使って実装
    

    View Slide

14. Performance Test
    • Simple Controller: when packet-in, then packet-out.
    • C . xlarge
    • Mbps
    • This test is all packet processed through Scala App.
    • In Actually most packet processing is

    offloaded by writing FlowRule to

    OpenFlow switch.
    • This performance is enough. Scala Application
    OpenFlow Message
    The Internet
    全パケットをScalaで処理して200Mbpsでしたが、

    実際はフローがキャッシュされるのでパフォーマンスは⼗分です
    

    View Slide

15. Architecture
    • OpenFlow Gateway: netty
    • serialize / deserialize OpenFlowMessages.
    • HTTP Server: Akka HTTP
    • provide CaptivePortal Web and authorize.
    • Captive Portal Controller: Akka Streams
    • modify user packet and manage OpenFlow rule.
    nettyとAkka HTTPとAkka Streamsを使って実装
    

    View Slide

16. • serialize / deserialize

    OpenFlowMessages.
    OpenFlow Gateway
    OpenFlow GatewayはOpenFlow Switchとの通信と

    メッセージのシリアライズ/デシリアライズを担当
    

    View Slide

17. HTTP Server
    • Redirect all HTTP traffic to Captive Portal.
    • Render Captive Portal Web.
    • Write into Redis if user is authorized.
    HTTP ServerはCaptive Portalの画⾯と認可するユーザーの情報をRedisに書き込みます
    

    View Slide

18. Captive Portal Controller
    • Authorized user:
    • allow all packets.
    • Unauthorized user:
    • Modify packet destination to "Fake DNS"

    If packet is DNS query.
    • "Fake DNS" always return

    IP address of Captive Portal
    • This is why all HTTP traffic of

    unauthorized users directed to Captive Portal.
    • Akka Streams is suitable for this case because packet processing is streaming.
    認可されていないユーザーのDNSの通信は全て"Fake DNS"宛に書き換えることで

    HTTPの通信を強制的にリダイレクトし、Captive Portalにつながるようにします
    

    View Slide

19. Case: Authorized user
    認証済みユーザーの通信が来た場合
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    

    View Slide

20. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    

    View Slide

21. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    

    View Slide

22. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Authorized!
    

    View Slide

23. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Authorized!
    Packet Out
    

    View Slide

24. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Authorized!
    Packet Out
    Flow Cached
    

    View Slide

25. Case: Authorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Authorized!
    Packet Out
    Flow Cached
    

    View Slide

26. Case: Authorized user (second packet)
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Flow Cached
    

    View Slide

27. Case: Authorized user (second packet)
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Flow Cached
    

    View Slide

28. Case: Authorized user (second packet)
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Flow Cached
    

    View Slide

29. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    

    View Slide

30. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    DNS packet
    OpenFlow Switch
    

    View Slide

31. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    Packet In
    OpenFlow Switch
    

    View Slide

32. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    Packet In
    Unauthorized!
    OpenFlow Switch
    

    View Slide

33. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    Packet In
    Unauthorized!
    Modify destination
    to Fake DNS server
    OpenFlow Switch
    

    View Slide

34. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    Packet In
    Unauthorized!
    Packet Out
    Modify destination
    to Fake DNS server
    OpenFlow Switch
    

    View Slide

35. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    Packet In
    Unauthorized!
    Modify destination
    to Fake DNS server
    Packet Out
    Fake DNS Server
    OpenFlow Switch
    

    View Slide

36. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Unauthorized!
    Modify destination
    to Fake DNS server
    Packet Out
    Fake DNS Server
    Fake DNS response
    

    View Slide

37. Case: Unauthorized user
    Captive Portal Server
    The Internet
    User
    OpenFlow Switch
    Packet In
    Unauthorized!
    Modify destination
    to Fake DNS server
    Packet Out
    Fake DNS Server
    Fake DNS response
    Redirect to Captive Portal
    

    View Slide

38. Deployment
    

    View Slide

39. Deploy into DataCenter
    • Linux Virtual Machine: Core GB (It was too match resource)
    • The reason why we deployed into datacenter is explain later.
    後述の理由により今回はデータセンタのサーバーにデプロイしました
    

    View Slide

40. Operation
    

    View Slide

41. There was NO Operation!
    • It Works! Almost...
    • Known Issues
    • Failed to detect Captive Portal only Samsung Galaxy Series
    • Workaround: Open some http site (e.g. http://neverssl.com ) instead of Captive Portal Detection.
    • Registered users!!
    特定の端末でどうしても動かなかったのですが、ほぼうまく動作しました！
    

    View Slide

42. WiFi
    

    View Slide

43. Scala Matsuri Wi-Fi Team
    • Engineers at CyberAgent, Inc.
    

    View Slide

44. Design
    

    View Slide

45. Goals
    • Connected everywhere
    • It's natural for someone to want to Comfortable Internet.
    • Technical challenge
    • Not just a work, use as real user traffic lab environment or playground.
    ただの業務にしたくなかったのと、せっかくなので⾊々なチャレンジをしました
    

    View Slide

46. Architecture
    • Internet connectivity was 

    provided by Home NOC Operator's Group

    (AS )
    • Scala Matsuri venue and Data Center is

    connected by flet's NGN
    Home NOC Operator's Groupのデータセンタと接続し、

    ⾼速なインターネット接続を提供して頂きました
    The Internet
    AS59105
    Scala Matsuri venue
    

    View Slide

47. Scala Matsuri Venue
    • Wi-Fi Access Points x
    • Cisco Aironet x
    • Cisco Meraki MR x
    • PoE Switching Hub x
    • Router x
    • NEC IX
    • Server x
    • Cable
    • LAN cable: about meters in total !
    • Optical fiber cable: meters
    会場ではたくさんの機材を設置しました
    

    View Slide

48. Preparation
    

    View Slide

49. Site survey at May
    5⽉の下⾒
    

    View Slide

50. Lab environment at our office
    • We were not expert of Wi-Fi.
    Wi-Fiの運⽤はしたことがなかったので社内に検証環境を作りました
    

    View Slide

51. Cabling Plan
    • There are few time to setup Wi-Fi on Day .
    • To setup quickly, we made a cabling map.
    当⽇の構築に使える時間が⾮常に限られてるため、事前にしっかり準備しました
    

    View Slide

52. "Wireless is wired" ...
    • Required total length of LAN cable is about meters.
    • We made cables by hand.
    無線LANは有線で繋がっているのでケーブル制作がけっこうたいへんでした…
    

    View Slide

53. Deploy
    

    View Slide

54. Deploy Wi-Fi APs
    • I was walked around over k steps and over kcals burned
    初⽇の消費カロリー
    

    View Slide

55. Entrance Hall( F)
    

    View Slide

56. Foyer( F)
    

    View Slide

57. Conference Hall ( F, F)
    

    View Slide

58. Media Hall ( F)
    

    View Slide

59. Conference Room ( F)
    

    View Slide

60. Operation
    

    View Slide

61. Prepare for accidents
    • Monitoring traffic, Wi-Fi APs, amount of clients, etc...
    • Datadog
    • ping, DHCP, DNS
    • Zabbix + Grafana
    • SNMP
    障害に備えて、DatadogとZabbixとGrafanaで監視環境を構築
    

    View Slide

62. Traffic
    • unique clients connected
    • yesterday: download GB, upload GB
    2⽇間で828端末がネットワークに接続
    

    View Slide

63. Wi-Fi Clients
    

    View Slide

64. Troubles
    

    View Slide

65. Day
    

    View Slide

66. Day
    • Day used only meeting room of F, so it was OK maybe...
    
    
    
    
    
    
    初⽇は4Fしか使⽤しなかったので来場者に影響はなかったはず…
    

    View Slide

67. Day
    • Day used only meeting room of F, so it was OK maybe...
    
    
    
    
    
    
    Fixed up quickly
    急いで修復⼯事をしました
    

    View Slide

68. Morning of Day
    2⽇⽬の朝
    

    View Slide

69. Morning of Day
    •
    
    2⽇⽬の朝
    

    View Slide

70. Morning of Day
    •
    
    2⽇⽬の朝
    
    

    View Slide

71. Morning of Day
    • Cable was dropped out ...
    どうしてこうなった
    

    View Slide

72. Conclusion
    • Implementing a Captive Portal in Scala.
    • Now open source !
    • https://github.com/kurochan/scaptive-portal
    • Succeeded to provide comfortable Wi-Fi
    

    View Slide

73. Thank you for enjoying
    #ScalaMatsuri and the Internet !
    

    View Slide