Upgrade to Pro — share decks privately, control downloads, hide ads and more …

visualization of Docker metrics

E86b46be0f4e61db6b28becc5493bbab?s=47 januswel
December 12, 2015

visualization of Docker metrics

Docker メトリクスの可視化 in Japanese

E86b46be0f4e61db6b28becc5493bbab?s=128

januswel

December 12, 2015
Tweet

Transcript

  1. VISUALIZATION OF DOCKER METRICS 2015/12/09 by janus_wel

  2. WHO? 高木健介 janus_wel archtect/programmer for full-stack

  3. PURPOSE OF VISUALIZATION ?

  4. TO CHECK LIFE OR DEATH ? should be covered by

    orchestration ex) consul
  5. TO USE RESOURCES EFFICIENTLY !! figure out bottlenecks decision a

    number of containers on a machine
  6. TO MAKE DOCUMENTS for customers for executives

  7. TO FOLLOW THE PDCA CYCLE figure out influences of changes

  8. COLLECTING METRICS USUALLY is important to trace back

  9. TOOLS Docker remote API InfluxDB

  10. DOCKER REMOTE API https://docs.docker.com/engine/reference/api/docker_remote_

  11. REST LIKE API The API tends to be REST, but

    for some complex commands, like attach or pull, the HTTP connection is hijacked to transport STDOUT, STDIN, and STDERR.
  12. AVAILABILITY maybe, all operations in commands are available. https://docs.docker.com/engine/reference/api/docker_remote_

  13. FOR EXAMPLE List containers Create a container Start a container

    Stop a container Remove a container List Images Build image from a Dockerfile Create an image Display system-wide information Show the docker version information ...
  14. REQUIREMENTS needs some settings for access control $DOCKER_OPTS in /etc/init/docker.conf

    on Ubuntu $OPTIONS in /etc/sysconfig/docker on other Linux if 192.0.2.1 is a client, D O C K E R _ O P T S = " - H 1 9 2 . 0 . 2 . 1 : 2 3 7 6 - H u n i x : / / / v a r / r u n / d o c k e r . s o c k " O P T I O N S = " - H 1 9 2 . 0 . 2 . 1 : 2 3 7 6 - H u n i x : / / / v a r / r u n / d o c k e r . s o c k " https://docs.docker.com/engine/userguide/basics/#bind- docker-to-another-host-port-or-a-unix-socket
  15. NOTE make sure to hear from unix domain socket. O

    P T I O N S = " - H 1 9 2 . 0 . 2 . 1 : 2 3 7 6 - H u n i x : / / / v a r / r u n / d o c k e r . s o c k " # ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ or, docker commands are not available.
  16. FOR DOCKER MACHINE machines created by docker-machine needs https by

    default, in your ~/.docker/machine/machines/(machine name)/ cert.pem key.pem this path is set by "$(docker-machine env (machine name))" as $DOCKER_CERT_PATH
  17. INFO w g e t h t t p :

    / / ( m a c h i n e i p ) : 2 3 7 6 / i n f o - O - - q | j q . for docker-machine w g e t \ - - n o - c h e c k - c e r t i f i c a t e \ - - c e r t i f i c a t e = $ { D O C K E R _ C E R T _ P A T H } / c e r t . p e m \ - - p r i v a t e - k e y = $ { D O C K E R _ C E R T _ P A T H } / k e y . p e m \ h t t p s : / / $ ( d o c k e r - m a c h i n e i p ( m a c h i n e n a m e ) ) : 2 3 7 6 / i n f o \ - O - - q | j q . "curl" can't handle non-checking local cert, so use "wget"
  18. None
  19. STATS API to get container's metrics, use stats API GET

    /containers/(id)/stats https://docs.docker.com/engine/reference/api/docker_remote_ container-stats-based-on-resource-usage
  20. None
  21. INFLUXDB https://influxdb.com/

  22. None
  23. POINTS schemaless SQL like query REST like interface

  24. SCHEMALESS there is no need to define somethings when writing

    data, InfluxDB needs only TABLE NAME and VALUE these are optional timestamp tags https://influxdb.com/docs/v0.9/concepts/schema_and_data_la
  25. SQL LIKE QUERY to read S E L E C

    T * F R O M c p u _ l o a d _ s h o r t W H E R E h o s t = s e r v e r 0 1 A N D r e g i o n = u s - w e s t - - ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ - - t a b l e n a m e t a g s https://influxdb.com/docs/v0.9/query_language/data_explor
  26. REST LIKE INTERFACE # C R E A T E

    D A T A B A S E c u r l - G h t t p : / / l o c a l h o s t : 8 0 8 6 / q u e r y - - d a t a - u r l e n c o d e " q = C R E A T E D A T A B A S E m y d b " # I N S E R T c u r l - i - X P O S T ' h t t p : / / l o c a l h o s t : 8 0 8 6 / w r i t e ? d b = m y d b ' - - d a t a - b i n a r y ' c p u _ l o a d _ # S E L E C T c u r l - G ' h t t p : / / l o c a l h o s t : 8 0 8 6 / q u e r y ? d b = m y d b ' - - d a t a - u r l e n c o d e " q = S E L E C T v a l u https://influxdb.com/docs/v0.9/guides/writing_data.html https://influxdb.com/docs/v0.9/guides/querying_data.html
  27. SYSTEM pull archtecture collect metrics minutely run with various OSs

    Linux Windows others can run Docker for now, test only in Linux...
  28. None
  29. METRICS COLLECTOR get metrics from Docker remote API and insert

    them InfluxDB in Ruby requires gem "clockwork" runs a collection minutely
  30. VISUALIZE in HTML and JavaScript use chart.js use some EcmaScript

    2015 features "let" statement Promise Object.assign http://www.chartjs.org/ now available with only Google Chrome
  31. DEMO http://52.193.71.92/

  32. METRICS legend description type rx_bytes received bytes counter tx_bytes transferred

    bytes counter total_cpu_usage cpu usage time % counter
  33. SOURCES https://github.com/januswel/docker-metrics

  34. PROBLEMS load stats API is heavy pull architecture calling API

    is proportional to number of containers handling mass containers id is not human-friendly
  35. PUSH ARCHITECTURE AND AGENT to collect with low load, we

    need inspect /sys c a t / s y s / f s / c g r o u p / m e m o r y / d o c k e r / < c o n t a i n e r - i d > / m e m o r y . s t a t c a t / s y s / f s / c g r o u p / c p u a c c t / d o c k e r / < c o n t a i n e r - i d > / c p u a c c t . s t a t pull architecture is unfit to cat local files agent is needed https://docs.docker.com/engine/articles/runmetrics/
  36. AFTER increase collectable metrices I don't know Docker metrics means

    what is... agent-ize collector