Slackboard - Slack proxy server in Go
Tatsuhiko Kubo@cubicdaiya
Shibuya.go#2 2016/03/22
Slide 2
Slide 2 text
@cubicdaiya / Tatsuhiko Kubo
Principal Engineer, SRE @ Mercari, Inc.
ngx_small_light, ngx_dynamic_upstream,
nginx-build, slackboard, cachectl, gaurun, widebullet, etc…
Slide 3
Slide 3 text
nginxຊΛࣥච͠·ͨ͠ʢ2016/01/16ൃചʣ
Slide 4
Slide 4 text
Slackboard
Slide 5
Slide 5 text
Slackboard
• Slack proxy server in Go
• https://github.com/cubicdaiya/slackboard
• Provides 3 commands
• slackboard - proxy server daemon for Slack
• slackboard-cli - client for slackboard
• slackboard-log - client like cronlog for slackboard
Slide 6
Slide 6 text
Install Slackboard
go get -u github.com/cubicdaiya/slackboard/…
Slide 7
Slide 7 text
Start Slackboard
slackboard -c slackboard.toml
Slide 8
Slide 8 text
Configuration for slackboard
[core]
# listen port
port = “29800”
# URL for Incoming Webhooks
slack_url = “https://hooks.slack.com/services/…”
[log]
# access log path or redirector
access_log = “stdout”
# error log path or redirector
error_log = “stderr”
# log level
level = “error”
[ui]
# document root for Web UI
root = “/var/www/slackboard/ui”
Slide 9
Slide 9 text
slackboard-cli
$ echo mercari | \
slackboard-cli \
-c tech-test \
-s slackboard-server:29800
TMBDLCPBSE
POST /notify-directly
Post to Slack
Slide 10
Slide 10 text
slackboard-log
$ ls
$ slackboard-log \
-c tech-test \
-s slackboard-server:29800 -- ls not found
TMBDLCPBSE
POST /notify-directly
Post to Slack
Slide 11
Slide 11 text
Slackboard APIs
• POST /notify
• POST /notify-directly
• GET /stat/go
• GET /config/app
See slackboard/SPEC.md about details
Slide 12
Slide 12 text
POST /notify
{
“tag”: “error-s3”,
“title”: “failed: aws s3 cp …”
“text”: “s3://bucket/path/to/xxx.log.gz”,
“level”: “warn”,
“sync”: false
}
[[tags]]
tag = “error-s3”
channel = “#alert-sre”
user = “package”
■ slackboard.toml
■ request body
■ posted message
Difference between /notify
and /notify-directly
• POST /notify
• receives original payload
• tag-based notification
• POST /notify-directly
• proxies payload to Slack directly
• channel-based notification
Slide 15
Slide 15 text
Asynchronous or synchronousʁ
• slackboard proxies message to slack asynchronous by
default
• “sync”: false
• slackboard returns 200 OK to client at once
• “sync”: true
• slackboard return 200 OK after posted message to
Slack
Slide 16
Slide 16 text
Attachments
• Slack provides mechanism for rich-formatted
message with Attachments
• coloring, tiltling, authoring
• embed image
• long text
• etc…
Slide 17
Slide 17 text
Colorize message by POST /notify
echo information | slackboard-cli -t test -s slackboard-server:29800 -l info
echo warning | slackboard-cli -t test -s slackboard-server:29800 -l warn
echo critical | slackboard-cli -t test -s slackboard-server:29800 -l crit
TMBDLCPBSE
POST /notify
Post to Slack
Slide 18
Slide 18 text
Colorize message by POST /notify-directly
echo good | slackboard-cli -c random -s slackboard-server:29800 -C good
echo bad | slackboard-cli -c random -s slackboard-server:29800 -C bad
echo warning | slackboard-cli -c random -s slackboard-server:29800 -C warning
TMBDLCPBSE
POST /notify-directly
Post to Slack
Slide 19
Slide 19 text
Alerting to Slack
Slide 20
Slide 20 text
Alerting to Slack with slackboard-cli
# copy log file to Google Cloud Storage
gsutil cp 2015-03-05T08.log.gz gs://${bucket}/
result=`echo $?`
if [ $result -ne 0 ];
then
echo "@channel: failed to copy log to GCS(2015-03-05T08)” | \
slackboard-cli -s slackboard-host:29800 -t error-gcs
fi
Slide 21
Slide 21 text
Alerting to Slack with slackboard-log
# copy log file to Google Cloud Storage
slackboard-log -s slackboard-host:29800 -t error-gcs -- \
gsutil cp 20150305T08.log.gz gs://${bucket}/
Slide 22
Slide 22 text
fluent-plugin-slackboard
https://github.com/cubicdaiya/fluent-plugin-slackboard
gem install fluent-plugin-slackboard
Alerting to Slack with Fluentd
Slide 23
Slide 23 text
fluent-plugin-slackboard
# required
type slackboard
host 127.0.0.1
port 29800
channel random
fetch_key message
# optional
username slackboard
icon_emoji :clipboard:
parse full
sync false
echo ‘{“message”: “bokko”}’ | \
fluent-cat slackboard
TMBDLCPBSE
qVFOUE
Slide 24
Slide 24 text
By the way,
Slide 25
Slide 25 text
Why is proxy server for Slack required?
• Messages are posted from various servers
• Where are messages posted from?
• Messages are posted from various programs
• Perl?Python?PHP?Ruby? or… Which library?
• Different by engineer and team
Slide 26
Slide 26 text
Slackboard’s approach
• Aggregate messages to Slack and logs by
• slackboard
• Unify notification method by
• slackboard-cli
• slackboard-log
Slide 27
Slide 27 text
In the case @
Slide 28
Slide 28 text
Before
Slide 29
Slide 29 text
It’s painfulʂ
• Incoming Webhooks URL is staggled in
various servers
• What if URL is changed?
• Each client program uses different language
and libraries
• (ϊ ʄДʄ)ϊ~~~ᵲᵲ
Slide 30
Slide 30 text
Now
Slide 31
Slide 31 text
Summary
• Slackboard provides
• Proxy server daemon for Slack
• Client for slackboard
• The goal of Slackboard
• Aggregate messages to Slack
• Unifies method for posting message to Slack
Slide 32
Slide 32 text
We are hiring
Go Engineer!
https://www.mercari.com/jp/jobs/