Make your and other programmers' life easier with static analysis (Unreal Engine 4)

The PVS-Studio team talks about how to use static analysis to look for code errors in the Unreal Engine 4 game engine.
Here are our results of checking Unreal Engine 4 with PVS-Studio: https://pvs-studio.com/en/blog/posts/cpp/0517/

Transcript

1. Make Your and Other Programmers’ Life Easier with Static Analysis (Unreal Engine 4) George Gribkov pvs-studio.com
2. 1. What is static analysis and what is it for? 2. How does static analysis work? (Unreal Engine 4) 3. How to introduce static analysis into your project: best practices Contents 2
3. What Is Static Analysis and What Is It for? 3
4. Challenges 4 Programmers QA Engineers
5. ▪ Programmers miss bugs ▪ QA Engineers spend time to find them ▪ Programmers spend time to fix them ▪ Cost to fix bugs rises rapidly Challenges 5
6. Challenges 6
7. Challenges 7
8. But sometimes tests miss errors… 8
9. Ariane 4 9
10. Ariane 4 10 113 successful launches Ariane 4
11. Ariane 5 11
12. 12 Ariane 5 370 000 000 $ blown up!
13. ▪ Integer overflow caused the explosion ▪ Rocket carried four satellites ▪ Losses amounted to $ 370 000 000 Example of a Very Expensive Error 13
14. ▪ Unit testing ▪ Integration testing ▪ System testing ▪ … ▪ Dynamic analysis ▪ Static analysis Ways to Find Errors 14
15. ▪ Static analysis tools: check code when it’s not executed ▪ Dynamic analysis tools: check code when it’s being executed Automated Code Analysis Tools 15 ▪ Both approaches complement each other very well
16. Static Analysis Automatically Searches for Bugs 16
18. ▪ Covers the entire code ▪ Works fast ▪ Is convenient for all sizes of projects ▪ Saves programmers’ time ▪ Saves QA Engineers’ time Static Analysis Pros 18
19. Modern Static Analysis Tools 19 • PVS-Studio • ReSharper • Coverity • SonarQube • Klocwork • Clang Static Analyzer • IntelliJ IDEA • And others
20. How It Works: Static Analysis in UE 4 20
21. How It Started 21
22. ▪ My boss found errors in UE 4 and wrote an article ▪ The developers of Epic Games liked the article a lot ▪ They wanted to fix more errors and entrusted it to us How It Started 22
23. The First Check (or How We Screwed Up) 23
24. ▪ The most convenient way: check the project via Visual Studio ▪ It’s great that UE has a set of scripts for .vcxproj- files The First Check 24
25. 1.Generate project files 2.Build the project 3.Start the analysis via Visual Studio 4.??????? 5. The First Check 25
26. 1.Generate project files 2.Build the project 3.Start the analysis via Visual Studio 4.??????? 5. EPIC GAMES FAIL The First Check 26
27. ▪ Generated project files are just wrappers ▪ These wrappers call the Unreal Build Tool ▪ Unreal Build Tool calls cl.exe (or clang for Linux builds) ▪ The analyzer cannot collect the parameters required for compilation because of all these layers Unreal Engine Build System 27
28. The Second Check (or How We Did Everything Right) 28
29. ▪ What if we try to find compiler calls directly? ▪ We’re lucky to have a special utility to monitor compilation The Second Check 29
30. 1.Start the compilation monitoring utility before building the project 2.The utility builds all the necessary data 3.Right after the build, run the analysis 4.??????? 5. The Second Analysis Attempt 30
31. The Second Analysis Attempt 31 1.Start the compilation monitoring utility before building the project 2.The utility builds all the necessary data 3.Right after build run the analysis 4.??????? 5. EPIC WIN!!!
32. Analysis Results 32
33. Analysis Results ▪ 1192 top level warnings (Level 1) ▪ 629 second level warnings (Level 2) ▪ 1821 warnings in total (without Level 3) 33
35. Why this is classic ▪ Notepad++ – 3 810 warnings 35
36. Why this is classic ▪ WinMerge – 6 830 warnings 36
37. Why this is classic ▪ Media Player Classic Home Cinema – 24 872 warnings! 37
38. How We Fixed Bugs 38
39. ▪ At night we built the final version of UE 4 ▪ We analyzed each build ▪ In the morning, we got a new report with errors found ▪ What’s more, we could check the build right away How We Fixed Bugs 39
40. Our Progress 40 40
41. The Number of Warnings 41 0 5 10 15 20 25 1 2 3 4 5 Warnings 0 5 10 15 20 25 1 2 3 4 5 Warnings ▪ Expectation ▪ Reality
42. After we fixed errors, we found four new warnings The Icing on The Cake 42
43. ▪ The developers of Epic Games were pleased ▪ They started using a continuous static code analysis, as we did ▪ Now they receive warnings about errors promptly ▪ As for us… we wrote another article :) Results 43
44. The Use of Static Analysis in Your Project: Best Practices 44
45. ▪ Run the analysis in the early stages ▪ Run the analysis regularly Two Main Approaches 45
46. Why is That Necessary 46
47. Why is That Necessary 47 Static Analysis
48. Why is That Necessary 48 Static Analysis You
49. ▪ May be used locally on developers’ computers (plugins for IDEs, compilation monitoring system) Introducing Static Analysis 49
50. ▪ May be used in Continuous Integration Systems (command- line utilities, CI-system plugins, monitoring systems) Introducing Static Analysis 50
51. Continuous Analysis 51
52. Fix Old Warnings Gradually 52
53. Suppress files are responsible for the "mass suppression of analyzer messages" mechanism. Suppress files 53
54. ▪ Hide old errors and work as usual ▪ Starting that moment you’ll get only new warnings ▪ Work on errors in new code only ▪ Don’t forget about hidden errors! Get back to them and fix one-by-one. How to Work with Suppress Files 54
55. ▪ A very convenient method is a “ratchet mechanism” ▪ Suppress file is committed to the version control system ▪ Changes are allowed only if they don’t increase warnings total number How to Work with Suppress Files 55
56. Ratchet Mechanism 56
57. Ratchet Mechanism 57
58. Ratchet Mechanism 58
59. ▪ NO ▪ The best approach: static + dynamic analysis Should I Use Static Analysis Only? 59
60. Conclusion 60
61. Conclusion 61 Help programmers find bugs and make your life easier!
62. Q&A George Gribkov pvs-studio.com