Upgrade to Pro — share decks privately, control downloads, hide ads and more …

delivering_security_products.pdf

ShaD
October 15, 2018

 delivering_security_products.pdf

Delivering security products without shooting in a foot

Talk about improving the infrastructure for developing, testing and delivering security tools. Our experience of smoothing the difference between security idealism and engineering friendliness.

ShaD

October 15, 2018
Tweet

More Decks by ShaD

Other Decks in Programming

Transcript

  1. Delivering security
    products
    without shooting in a foot
    Dmytro Shapovalov
    Infrastructure Engineer @ Cossack Labs

    View Slide

  2. We will talking about:
    — Approaches, not instruments
    — CI/CD
    — Infrastructure
    — Workflow
    — Usability

    View Slide

  3. Why is the
    delivery a bit
    different in
    security?

    View Slide

  4. An attractive
    target for
    attackers

    View Slide

  5. Highly
    qualified
    and
    very
    demanding
    community

    View Slide

  6. Significant risks
    and bigger cost
    of each mistake
    Public
    responsibility

    View Slide

  7. What is the problem?
    Security teams create complex
    security products,
    developers want to have an
    easy way to use them

    View Slide

  8. Why developing is so hard?
    — ingenious ideas
    — chaotic strategy changing
    — do not care about
    compatibility
    — testing in hothouse conditions
    — hard to deploy
    — hard to use properly

    View Slide

  9. Developers
    expect stability
    from libraries,
    but tend to
    make
    breaking
    changes in
    own products

    View Slide

  10. Avoid creating
    useless products!

    View Slide

  11. Take care
    of the
    delivery!

    View Slide

  12. Let's look for
    a solution!

    View Slide

  13. Our company specific:
    — we create crypthographic products
    — compact size of team
    — mostly engineering staff
    — horizontal structure
    — open-source products
    — multiple languages / platforms

    View Slide

  14. What is critical for us?
    — automation
    — clear, transparent processes
    — responsibility
    — openness
    — safety
    — use own products
    — write docs

    View Slide

  15. Let's do
    something!

    View Slide

  16. Let's divide solution into the parts:
    — architecture
    — workflow standards and rules
    — infrastructure
    — CI/CD

    View Slide

  17. Are you
    involved
    into the
    planning process?

    View Slide

  18. What questions
    should we answer
    during planning?

    View Slide

  19. What are
    customers'
    values?

    View Slide

  20. How
    will it be
    arranged?

    View Slide

  21. — on what platforms should our application work?
    — how many and what are the types of the
    components?
    — usability bounds of the different components?
    — what are the possible topology variants?

    View Slide

  22. — how will they communicate?
    — how do we plan to scale?
    — what external dependencies will the product have?
    — what are possible vulnerabilities and risks in the
    planned architecture?

    View Slide

  23. Use existing standards
    wherever possible!

    View Slide

  24. How will it be
    delivered?

    View Slide

  25. — source code
    — package management systems
    — docker
    — pre-built images

    View Slide

  26. Give the
    choice
    for a
    lazy user!

    View Slide

  27. What are the external systems
    and how do we plan to integrate with?
    — basic functional systems
    — an application in which we integrate
    — databases
    — libraries
    — auxiliary systems
    — monitoring
    — SIEM

    View Slide

  28. How will it be
    monitored?

    View Slide

  29. How will application
    handle errors?
    — standard messages
    — standard codes
    — common library

    View Slide

  30. How do you plan to do
    logging, tracing
    and metrics gathering?

    View Slide

  31. Logging
    — events
    — human-readable
    — standard output methods
    Formats and outputs
    File STDERR Syslog
    Plain text ✔ ✔ -
    JSON ✔ ✔ ✔
    CEF ✔ ✔ ✔
    syslog - - ✔

    View Slide

  32. Metrics
    — availability
    — components' health
    — performance
    — statistics
    Output
    — prometheus standard

    View Slide

  33. Tracing
    — detail analyze of successive data
    processing
    — intrusion detection
    — performance tuning
    — deep debugging
    Output
    — OpenTracing
    — OpenCensus
    — JSON

    View Slide

  34. How will it be
    supported?
    — fast deploy scripts
    — troubleshooting information
    gathering tools

    View Slide

  35. Workflow standards and rules
    — synchronize all team members
    — unify approaches
    — make the development results more expected and clear
    for our customers
    — simplify automation

    View Slide

  36. The main idea is
    to simplify
    internal processes!

    View Slide

  37. The rules must:
    — be as simple and clear as possible
    — be kept actual
    — have a minimal impact on productivity
    — only correct, not break existing workflow

    View Slide

  38. Our internal standards and rules
    — versioning
    — naming
    — git flow
    — CI/CD

    View Slide

  39. Versioning
    Semantic Versioning
    https://semver.org
    Format: a.b.c, where:
    a — major version, increases with significant changes of
    product
    b — minor version, increases with current releases,
    backward compatibility may not be guaranteed
    c — build version, increases with bug hotfixes, minor
    features' improvements

    View Slide

  40. Naming
    Element Rule Example
    application CamelCase AcraServer
    binary debian-style acra-connector
    package debian-style libthemis-
    dev0.10.0+stretch
    amd64.deb

    View Slide

  41. Git flow and CI/CD
    Name Usage CI/CD
    automation
    stable Releases all tests, packaging,
    delivering
    master Current state all tests, packaging,
    delivering
    username/
    GH999_issue_des
    c
    Dev, fixes manual

    View Slide

  42. The earlier you implement
    the internal standards,
    the cheaper the support
    will be later!

    View Slide

  43. Infrastructure
    Part Requirements
    core safety, stable
    developing speed up, easy cross-
    communications
    CI/CD full automation, tests,
    packaging, delivery

    View Slide

  44. CI/CD

    View Slide

  45. View Slide

  46. CircleCI
    — quick check of basic
    functionality

    View Slide

  47. Buildbot
    — unit tests
    — multiple platforms
    — crossplatform tests
    — integration tests
    — pre-release checks
    — packages building

    View Slide

  48. Buildbot architecture
    — core
    — CI/CD engine
    — scenarios

    View Slide

  49. CI/CD rules
    — emulate real customer environment
    — use system libraries
    — strict configurations
    — scenarios according to docs
    — easily maintained and extended
    — developers can add scenarios

    View Slide

  50. Exact methodology
    do not change rules on the fly

    View Slide

  51. CI/CD
    flow

    View Slide

  52. Docker

    View Slide

  53. Docker image sizes
    — large image size is typical
    — most developers do not pay
    attention to the size
    — there are ways to significantly
    reduce the image
    — we reduced the size from
    ≈950MB to ≈15MB

    View Slide

  54. What technologies did we use?
    — multi-stage builds
    — build from scratch
    You can find all dockerfiles at:
    https://github.com/cossacklabs/acra/tree/master/docker

    View Slide

  55. Take care of the
    consumer!

    View Slide

  56. Image tags
    Static tags
    — version tag
    — git commit tag
    Sliding tags
    — stable, latest
    — master, current

    View Slide

  57. Image labels
    — label-schema standard
    — Cossack Labs internal standard

    View Slide

  58. How do we use
    docker compose?
    As a demonstration stand!

    View Slide

  59. What was uncomfortable?
    — generate and manually place keys
    — configure parameters inside docker-compose file
    manually
    — one very simple example of configuration
    — topology was unclear
    — security perimeter was not visible

    View Slide

  60. What now?
    — multiple demonstation stands
    — all stands can be launched in
    one action
    — confiurations can be easily
    extended and secured
    *
    — self-documented
    — topology described

    View Slide

  61. Summary
    — complex security products require careful and accurate
    handling
    — iplemented standards
    — built large testing platform
    — release cycle: from once a year to once a couple weeks
    — from burst developing to planned releases
    — increased flexibility of our team
    — from concept researching to implementing usable
    features

    View Slide

  62. Our products became closer
    to consumers!

    View Slide

  63. Links
    — Cossack Labs official site
    https://cossacklabs.com
    — GitHub
    https://github.com/cossacklabs
    — Docker Hub
    https://hub.docker.com/r/cossacklabs/

    View Slide

  64. Dmytro Shapovalov
    [email protected]
    shadinua
    shad.in.ua
    shad.in.ua

    View Slide