Upgrade to Pro — share decks privately, control downloads, hide ads and more …

delivering_security_products.pdf

ShaD
October 15, 2018

 delivering_security_products.pdf

Delivering security products without shooting in a foot

Talk about improving the infrastructure for developing, testing and delivering security tools. Our experience of smoothing the difference between security idealism and engineering friendliness.

ShaD

October 15, 2018
Tweet

More Decks by ShaD

Other Decks in Programming

Transcript

  1. Delivering security
    products
    without shooting in a foot
    Dmytro Shapovalov
    Infrastructure Engineer @ Cossack Labs

    View full-size slide

  2. We will talking about:
    — Approaches, not instruments
    — CI/CD
    — Infrastructure
    — Workflow
    — Usability

    View full-size slide

  3. Why is the
    delivery a bit
    different in
    security?

    View full-size slide

  4. An attractive
    target for
    attackers

    View full-size slide

  5. Highly
    qualified
    and
    very
    demanding
    community

    View full-size slide

  6. Significant risks
    and bigger cost
    of each mistake
    Public
    responsibility

    View full-size slide

  7. What is the problem?
    Security teams create complex
    security products,
    developers want to have an
    easy way to use them

    View full-size slide

  8. Why developing is so hard?
    — ingenious ideas
    — chaotic strategy changing
    — do not care about
    compatibility
    — testing in hothouse conditions
    — hard to deploy
    — hard to use properly

    View full-size slide

  9. Developers
    expect stability
    from libraries,
    but tend to
    make
    breaking
    changes in
    own products

    View full-size slide

  10. Avoid creating
    useless products!

    View full-size slide

  11. Take care
    of the
    delivery!

    View full-size slide

  12. Let's look for
    a solution!

    View full-size slide

  13. Our company specific:
    — we create crypthographic products
    — compact size of team
    — mostly engineering staff
    — horizontal structure
    — open-source products
    — multiple languages / platforms

    View full-size slide

  14. What is critical for us?
    — automation
    — clear, transparent processes
    — responsibility
    — openness
    — safety
    — use own products
    — write docs

    View full-size slide

  15. Let's do
    something!

    View full-size slide

  16. Let's divide solution into the parts:
    — architecture
    — workflow standards and rules
    — infrastructure
    — CI/CD

    View full-size slide

  17. Are you
    involved
    into the
    planning process?

    View full-size slide

  18. What questions
    should we answer
    during planning?

    View full-size slide

  19. What are
    customers'
    values?

    View full-size slide

  20. How
    will it be
    arranged?

    View full-size slide

  21. — on what platforms should our application work?
    — how many and what are the types of the
    components?
    — usability bounds of the different components?
    — what are the possible topology variants?

    View full-size slide

  22. — how will they communicate?
    — how do we plan to scale?
    — what external dependencies will the product have?
    — what are possible vulnerabilities and risks in the
    planned architecture?

    View full-size slide

  23. Use existing standards
    wherever possible!

    View full-size slide

  24. How will it be
    delivered?

    View full-size slide

  25. — source code
    — package management systems
    — docker
    — pre-built images

    View full-size slide

  26. Give the
    choice
    for a
    lazy user!

    View full-size slide

  27. What are the external systems
    and how do we plan to integrate with?
    — basic functional systems
    — an application in which we integrate
    — databases
    — libraries
    — auxiliary systems
    — monitoring
    — SIEM

    View full-size slide

  28. How will it be
    monitored?

    View full-size slide

  29. How will application
    handle errors?
    — standard messages
    — standard codes
    — common library

    View full-size slide

  30. How do you plan to do
    logging, tracing
    and metrics gathering?

    View full-size slide

  31. Logging
    — events
    — human-readable
    — standard output methods
    Formats and outputs
    File STDERR Syslog
    Plain text ✔ ✔ -
    JSON ✔ ✔ ✔
    CEF ✔ ✔ ✔
    syslog - - ✔

    View full-size slide

  32. Metrics
    — availability
    — components' health
    — performance
    — statistics
    Output
    — prometheus standard

    View full-size slide

  33. Tracing
    — detail analyze of successive data
    processing
    — intrusion detection
    — performance tuning
    — deep debugging
    Output
    — OpenTracing
    — OpenCensus
    — JSON

    View full-size slide

  34. How will it be
    supported?
    — fast deploy scripts
    — troubleshooting information
    gathering tools

    View full-size slide

  35. Workflow standards and rules
    — synchronize all team members
    — unify approaches
    — make the development results more expected and clear
    for our customers
    — simplify automation

    View full-size slide

  36. The main idea is
    to simplify
    internal processes!

    View full-size slide

  37. The rules must:
    — be as simple and clear as possible
    — be kept actual
    — have a minimal impact on productivity
    — only correct, not break existing workflow

    View full-size slide

  38. Our internal standards and rules
    — versioning
    — naming
    — git flow
    — CI/CD

    View full-size slide

  39. Versioning
    Semantic Versioning
    https://semver.org
    Format: a.b.c, where:
    a — major version, increases with significant changes of
    product
    b — minor version, increases with current releases,
    backward compatibility may not be guaranteed
    c — build version, increases with bug hotfixes, minor
    features' improvements

    View full-size slide

  40. Naming
    Element Rule Example
    application CamelCase AcraServer
    binary debian-style acra-connector
    package debian-style libthemis-
    dev0.10.0+stretch
    amd64.deb

    View full-size slide

  41. Git flow and CI/CD
    Name Usage CI/CD
    automation
    stable Releases all tests, packaging,
    delivering
    master Current state all tests, packaging,
    delivering
    username/
    GH999_issue_des
    c
    Dev, fixes manual

    View full-size slide

  42. The earlier you implement
    the internal standards,
    the cheaper the support
    will be later!

    View full-size slide

  43. Infrastructure
    Part Requirements
    core safety, stable
    developing speed up, easy cross-
    communications
    CI/CD full automation, tests,
    packaging, delivery

    View full-size slide

  44. CircleCI
    — quick check of basic
    functionality

    View full-size slide

  45. Buildbot
    — unit tests
    — multiple platforms
    — crossplatform tests
    — integration tests
    — pre-release checks
    — packages building

    View full-size slide

  46. Buildbot architecture
    — core
    — CI/CD engine
    — scenarios

    View full-size slide

  47. CI/CD rules
    — emulate real customer environment
    — use system libraries
    — strict configurations
    — scenarios according to docs
    — easily maintained and extended
    — developers can add scenarios

    View full-size slide

  48. Exact methodology
    do not change rules on the fly

    View full-size slide

  49. Docker image sizes
    — large image size is typical
    — most developers do not pay
    attention to the size
    — there are ways to significantly
    reduce the image
    — we reduced the size from
    ≈950MB to ≈15MB

    View full-size slide

  50. What technologies did we use?
    — multi-stage builds
    — build from scratch
    You can find all dockerfiles at:
    https://github.com/cossacklabs/acra/tree/master/docker

    View full-size slide

  51. Take care of the
    consumer!

    View full-size slide

  52. Image tags
    Static tags
    — version tag
    — git commit tag
    Sliding tags
    — stable, latest
    — master, current

    View full-size slide

  53. Image labels
    — label-schema standard
    — Cossack Labs internal standard

    View full-size slide

  54. How do we use
    docker compose?
    As a demonstration stand!

    View full-size slide

  55. What was uncomfortable?
    — generate and manually place keys
    — configure parameters inside docker-compose file
    manually
    — one very simple example of configuration
    — topology was unclear
    — security perimeter was not visible

    View full-size slide

  56. What now?
    — multiple demonstation stands
    — all stands can be launched in
    one action
    — confiurations can be easily
    extended and secured
    *
    — self-documented
    — topology described

    View full-size slide

  57. Summary
    — complex security products require careful and accurate
    handling
    — iplemented standards
    — built large testing platform
    — release cycle: from once a year to once a couple weeks
    — from burst developing to planned releases
    — increased flexibility of our team
    — from concept researching to implementing usable
    features

    View full-size slide

  58. Our products became closer
    to consumers!

    View full-size slide

  59. Links
    — Cossack Labs official site
    https://cossacklabs.com
    — GitHub
    https://github.com/cossacklabs
    — Docker Hub
    https://hub.docker.com/r/cossacklabs/

    View full-size slide

  60. Dmytro Shapovalov
    [email protected]
    shadinua
    shad.in.ua
    shad.in.ua

    View full-size slide