Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gorush: A push notification server written in Go

Bo-Yi Wu
October 28, 2017

Gorush: A push notification server written in Go

* Why I create the Gorush Project?
* Detail implementation in Golang
* Testing and deploy Go project.
* Run Gorush in Kubernetes.

Bo-Yi Wu

October 28, 2017
Tweet

More Decks by Bo-Yi Wu

Other Decks in Technology

Transcript

  1. A push notification server written in Go
    Gorush
    1
    Bo-Yi Wu


    Mopcon @ 2017.10.28


    View full-size slide

  2. ●Why I create the Gorush Project?


    ●Detail implementation in Golang


    ●Testing and deploy Go project.


    ●Run Gorush in Kubernetes.
    2
    Outline

    View full-size slide

  3. ● Mediatek Engineer


    ● DevOps


    ● Golang, PHP, Node.js ..


    ● Some open source


    ● Gitea


    ● Gin


    ● Drone


    ● appleboy @GitHub


    ● appleboy @twitter


    ● appleboy @slideshare


    ● appleboy46 @facebook
    About Me
    3

    View full-size slide

  4. ● drone/drone


    ● go-gitea/gitea


    ● gin-gonic/gin


    ● appleboy/gorush
    Go Open Source in GitHub
    4

    View full-size slide

  5. ● drone/drone


    ● go-gitea/gitea


    ● gin-gonic/gin


    ● appleboy/gorush
    Go Open Source in GitHub
    5

    View full-size slide

  6. https://mcs.mediatek.com/


    6

    View full-size slide

  7. Mediatek


    Cloud
    Sandbox
    Trigger
    Push
    Email
    hook
    MQTT
    TCP
    HTTP
    Mediatek Cloud Sandbox
    7

    View full-size slide

  8. Mediatek


    Cloud
    Sandbox
    Trigger
    Push
    Email
    hook
    MQTT
    TCP
    HTTP
    Mediatek Cloud Sandbox
    8
    Forcus on Push notification

    View full-size slide

  9. https://github.com/appleboy/gorush
    Gorush
    9

    View full-size slide

  10. A general push notification server


    for smartphone application
    Gorush
    10

    View full-size slide

  11. ●Written in Go


    ●Push requests to APNs and FCM


    ●Simple HTTP APIs (JSON)


    ●RPC Protocol (gRPC)
    Gorush
    11

    View full-size slide

  12. Overview
    Client
    FCM
    APN
    s
    Gorush
    POST /push
    JSON Body
    HTTPS
    APNs


    Protocol
    12

    View full-size slide

  13. ●System Requirement


    ●Good Performance


    ●High Concurrency


    ●Simple net/http Package


    ●Easy to Learn
    Why I Choose Go
    13

    View full-size slide

  14. https://github.com/golang/go/wiki/FromXToGo
    Why we switch

    From XX to Go
    14

    View full-size slide

  15. https://github.com/golangtw/jobs
    Who use Go in Taiwan
    15

    View full-size slide

  16. 16
    Push System Architecture

    View full-size slide

  17. Before Seven years ago
    FCM
    APNs
    API Server
    API Server
    API Server
    PUSH
    17

    View full-size slide

  18. Before Two Years ago
    FCM
    APNs
    Worker
    Worker
    Worker
    PUSH
    Queue
    API
    API
    API
    de-queue
    Asynchronous Notification
    AWS
    en-queue
    18

    View full-size slide

  19. Nginx


    Traefik
    Now…
    FCM
    APNs
    Gorush
    Gorush
    Gorush
    PUSH
    API
    API
    API
    Http


    Post /push


    Gorush = Queue + Worker
    19

    View full-size slide

  20. How to implement simple notification service
    Gorush Architecture
    20

    View full-size slide

  21. How to send notification
    Client
    FCM
    APN
    s
    Gorush
    POST /push
    JSON Body
    HTTPS
    APNs


    Protocol
    21

    View full-size slide

  22. ●Http Server


    ●gRPC Server


    ●Push API


    ●Worker
    22
    Gorush Component

    View full-size slide

  23. net/http + Gin Route+ Facebook Grace
    23

    View full-size slide

  24. Push API using Gin Framework
    24

    View full-size slide

  25. Initial Worker Queue
    25

    View full-size slide

  26. API Request and Response
    26

    View full-size slide

  27. Send notification to Queue
    27

    View full-size slide

  28. Start Worker
    28

    View full-size slide

  29. Gorush Major Feature
    29

    View full-size slide

  30. ●GET /api/stat/go


    ●GET /api/stat/app


    ●GET /sys/stats


    ●GET /metrics


    ●POST /api/push


    ●GET /healthz
    API List
    30

    View full-size slide

  31. GET /api/stat/go
    31

    View full-size slide

  32. GET /api/stat/app
    32

    View full-size slide

  33. GET /sys/stats
    33

    View full-size slide

  34. GET /metrics (Prometheus)
    34

    View full-size slide

  35. GET /metrics (Prometheus + Grafana)
    $ for i in {1..99999}; do bat -b.N=1000 -b.C=100 POST local:8088/api/push
    35

    View full-size slide

  36. GET /metrics (Prometheus + Grafana)
    36

    View full-size slide

  37. https://github.com/appleboy/gorush/#post-apipush
    POST /api/push
    37

    View full-size slide

  38. Push API Support Sync Mode
    Async vs. Sync Push
    38

    View full-size slide

  39. Notification
    Notification
    Notification
    Notification
    worker
    worker
    worker
    Client
    Notification
    worker
    WaitGroup
    39

    View full-size slide

  40. Notification
    Log
    Wait response
    Send to Work
    Queue
    40

    View full-size slide

  41. $ gorush
    Start Gorush Server
    42

    View full-size slide

  42. $ gorush –c config.yml
    Custom config file
    43

    View full-size slide

  43. $ GORUSH_CORE_PORT=8089 gorush
    Global Variable
    44

    View full-size slide

  44. ● /etc/gorush/config.yml


    ● $HOME/.gorush/config.yml


    ● . (Current Folder)
    45
    Auto Load Config file

    View full-size slide

  45. Server Confg
    Route
    Config
    46

    View full-size slide

  46. Server Config
    Route
    Config
    47

    View full-size slide

  47. 48
    Go Support

    Auto TLS
    Integrate Let's Encrypt

    View full-size slide

  48. Autocert package in Go
    49

    View full-size slide

  49. Write one line code to support

    renew Let’s Encrypt automatically
    https://goo.gl/pfeC5Q
    50

    View full-size slide

  50. Android Config
    iOS Config
    Log Config
    51

    View full-size slide

  51. Send Android Notification
    API Key From Request
    52

    View full-size slide

  52. iOS Support .p12, .pem or .p8
    53
    Supports new Apple Token Based Authentication (JWT)

    View full-size slide

  53. Send iOS Notification
    54

    View full-size slide

  54. Success or fail records
    55

    View full-size slide

  55. memory, blotdb, buntdb, leveldb, redis
    Storage
    56

    View full-size slide

  56. Init Interface
    58

    View full-size slide

  57. package flg
    Send Notification

    From CLI
    59

    View full-size slide

  58. Flag in Go
    60

    View full-size slide

  59. Flag doesn’t support

    Global variable
    Docker Container
    61

    View full-size slide

  60. Configuration Management
    62

    View full-size slide

  61. $ gorush -android –m="message" -k="API Key" –t="token"
    Sned Android
    63

    View full-size slide

  62. $ gorush -ios -m="message" -i="certificate path"


    -t="token" -topic="topic"
    Send iOS
    64

    View full-size slide

  63. write once run anywhere
    Cross Platform
    65

    View full-size slide

  64. https://github.com/mitchellh/gox
    Simple Go Cross Compilation
    66

    View full-size slide

  65. https://github.com/facebookgo/grace
    Graceful restart & zero downtime
    68

    View full-size slide

  66. How to resolve this problem?
    Facebook Grace

    doesn’t Support Windows
    69

    View full-size slide

  67. // +build windows


    https://golang.org/pkg/go/build/
    Go build Flag
    70

    View full-size slide

  68. Build Binary into

    Docker Container
    71

    View full-size slide

  69. Docker Multi-Stage Build
    72

    View full-size slide


  70. Build small image using docker multi-stage
    $ docker build -t appleboy/gorush .


    https://goo.gl/zQbhmu
    73

    View full-size slide

  71. Auto testing and deploy in GO
    74

    View full-size slide

  72. Continuous Delivery system


    built on container technology
    What is Drone?
    76

    View full-size slide

  73. How to setup

    deploy process?
    77
    Using .drone.yml file

    View full-size slide

  74. Git Test Release Docker Notify
    How to build Go binary in Drone
    78

    View full-size slide

  75. ● Testing Code


    ● Analytic Code


    ● Code Quality


    ● Build Binary


    ● Deploy Binary


    ● Build Docker Image


    ● Deploy Docker Container


    ● Send Notification
    Pipeline Process in Go
    79

    View full-size slide

  76. Git Test Release Docker Notify
    How to build Go binary in Drone
    80

    View full-size slide

  77. First Step: Git Clone
    81

    View full-size slide

  78. Git Test Release Docker Notify
    How to build Go binary in Drone
    82

    View full-size slide

  79. Choose Image
    Testing Process
    83

    View full-size slide

  80. Code Quality in Go (golint)
    84

    View full-size slide

  81. Check comment typos (Misspell)
    85

    View full-size slide

  82. Check code format in Go (gofmt)
    86

    View full-size slide

  83. Check vendor folder (govendor)
    87

    View full-size slide

  84. Testing in Go
    $ coverage all


    https://github.com/appleboy/golang-testing
    88

    View full-size slide

  85. Git Test Release Docker Notify
    How to build Go binary in Drone
    89

    View full-size slide

  86. Choose Image
    Step Name
    Run on for Git tag event
    90

    View full-size slide

  87. Build binary for Cross Platform (gox)
    91

    View full-size slide

  88. Deploy to GitHub Release Page
    92

    View full-size slide

  89. Git Test Release Docker Notify
    How to build Go binary in Drone
    93

    View full-size slide

  90. Drone Envars
    Drone Secrets
    Plugin Image
    94

    View full-size slide

  91. Git Test Release Docker Notify
    How to build Go binary in Drone
    95

    View full-size slide

  92. Plugin Image
    Push Notification From FB Bot
    96

    View full-size slide

  93. Happy Coding 97

    View full-size slide

  94. Run Gorush in Kubernets
    98

    View full-size slide

  95. 99
    Container Health Check

    View full-size slide

  96. 100
    Global Configuration Map

    View full-size slide

  97. $ kubectl create -f k8s


    $ kubectl get services


    $ minikube service frontend
    Three Steps
    102

    View full-size slide

  98. Interesting In Drone
    http://bit.ly/devops-drone
    103

    View full-size slide

  99. Thanks All coming.
    Any Problem?
    104

    View full-size slide