"Блеск и нищета SAST", Андрей Ковалев, Яндекс

OWASP Russia MeetUp #7

Transcript

2. Andrey Kovalev, Security Engineer Splendors an Miseries of the SAST
3. Agenda 1 SAST at Yandex 2 Appropriate cases and true positive examples 3 Inappropriate cases and false positive examples 4 Statistics and summary
4. SAST at Yandex Splendors an Miseries of the SAST
5. SAST as a process — Yandex approach 5 Dashboards SAST Code analysis Configuration check Dependency inspection Search of secrets Project tracker Data storage Product team
6. Why SAST? 6 At Yandex: › Solve SAST-specific cases: dependency inspection or search of secrets › Perform analysis for platforms which don’t have good DAST tools (for example on Windows) › Protect the products with poor fuzzing’s or unit-test’s coverage
7. Why SAST? 7 In other cases › Perform regular analysis without powerful testing infrastructure › One-time security check without writing additional fuzz-tests › Security assessment without writing additional code (ideal case)
8. Code analyzers we use 8 › For C\C++ : Clang Static Analyzer, Coverity › For Java: Findbugs and Find security bugs › JavaScript: Coverity › For Python: Bandit
9. Main search targets for code static analysis 9 For web applications: › Parsing dangerous serialization data formats › Insecure platform API or technology usage › Unsafe processing of data in SQL queries › Improper encoding or escaping of output
10. Main search targets for code static analysis 10 For binary applications: › Race conditions: time-of-check-time-of-use problem, unsafe thread access problem (Coverity) › Memory corruptions: out of bounds, use after free, type confusion issues
11. Appropriate and true positive examples Brilliance and poverty of the SAST
12. Coverity detects simple memory corruptions 12 Char array overrun 
 
 
 

13. Bandit warns about potentially XXE 13 Unsafe XML Parsing 
 
 
 

14. Inappropriate and false positive examples Brilliance and poverty of the SAST
15. Bandit warns about dangerous serialization 15 Is the usage of the Pickle always bad? It depends… 
 
 
 

16. Output escaping false positive 16 Coverity doesn’t recognize custom escaping implementation 
 
 
 

17. Output escaping false positive 17 Coverity doesn’t understand shared pointer semantics 
 
 
 

18. Statistics and summary Splendors an Miseries of the SAST
19. Statistics §for vulnerabilities 19 0 7,5 15 22,5 30 Coverity Bandit FindSecBugs High Medium
20. Statistics for true positives / false positives 20 0 10 20 30 40 Coverity Bandit FindSecBugs True False
21. Where is SAST good enough? 21 Good precision (70%+ ): › Dangerous format data parsing: yaml, lxml, etc › Insecure API usage: shell commands, insecure randoms, debug modes, etc. › SQL injections › Unitialized variables and members for C++ Average precision (55% - 70%) › Improper encoding or escaping of output
22. Where is SAST have lot’s of false positives? 22 Awful precision (20%): › Memory corruption in modern C++ applications › Data leakage — wrapper escaping
23. Implementation best practice 23 › Analyze only relevant code: exclude tests and third-party › Tune checking rules, switch off false positive checkers › Divide analysis output in two parts: one (with tuned proven checker) for developers, second — for security team › Get feedback from developers: try to find enthusiasts (or security champions) and ask them to help to classify results › Don’t create tickets for developers if issue is not critical and checker is not proven
24. Our point of view (general conclusions) 24 › Some of SAST checkers (Bandit) are regexp-based analyzers of AST › Some analyzers (Coverity, FindSecBugs) try to do taint analysis, but it’s not very good - they still can’t recognize user-controlled input › In general, SAST is good enough for insecure API usage, dangerous format parsing, input validation etc., but only for standard cases › For memory corruptions in modern applications — use DAST (sanity testing, fuzzing), these technics are more effective
25. Andrey Kovalev Security engineer @L1kvID avkov@yandex-team.ru Thank you! Questions?