Keep Calm, it's Reverse Engineering Time - PyTN2016

As developers, sometimes we have to investigate a bug, or add a new feature in a codebase that is completely new to us. Sometimes it can be even developed in technology that we are unfamiliar with, often with no one available to ask anything about that code. How can we do this?
In this talk I would like to share some ideas, tips and tools related to reverse engineering of various code components (server side, frontend, etc) – that can help developers to successfully overcome this challenge.

Transcript

1. Alissa Bonas mikeyteva
2. “Run Forrest, Run”
3. When is it needed? • Add a new feature • Fix a bug • Investigate a problem • Maintain legacy code • Integration with another system or component
4. “My momma always said, life was like a box of chocolates. You never know what you’re gonna get”
5. How can we succeed?
6. Ask the person who developed it
8. However...
9. Is it an open source project? • Check the docs / wiki • Ask in the community ◦ mailing list ◦ forum ◦ irc ◦ open an issue / ticket with a question
11. Back to basics
12. Back to basics • Technology / Programming language • Source control • Running tests • Build / Run / Deploy
13. Detecting languages on Github
14. Detecting languages on Github
15. Not 100% accurate
16. Look for hints • File extensions • .gitignore • README.md
17. Look for more hints • Requirements.txt / setup.py • pom.xml / build.xml • Gemfile / gemspec • bower.json
18. Scope the task
19. Minor changes • Fix typos • Add more info to a log statement • Persist a field that already exists in an object
20. In search we trust • Search for strings in the code repository • Search existing logs for object names
21. Medium size changes • Add a new field to an object / model ◦ process and persist ◦ use it in all code layers • Upgrade the version of a third party library ◦ regression is your nemesis ◦ minor version upgrade != major version upgrade
22. Follow the trail • Follow existing fields ◦ understand the flow • Follow the tests ◦ learn how to add new tests ◦ check if tests pass after code changes
23. Major changes • Refactor a module / component • Add a new functionality which requires a deep understanding of the system • Performance is key and must not be harmed
24. Understand • The architecture of the system • Relevant flows and use cases • The main path • Fallback, retries, etc.
25. Number 1 factor is...
26. Time
28. Goal: Identify entry points • UI / REST / Backend • Class hierarchy • Identify and learn frameworks
29. Get to know the code better • Is there a dependency injection mechanism? • How do the components communicate • Configuration keys to enable more info in the logs
30. Get to know the system better • Identify persistency ◦ data store type ◦ how the data is stored • Know your container ◦ web server ◦ application server
31. Product configuration files • Static files location • Check how they are used and changed ◦ directly ◦ loaded into the database
32. Multi process system • Identify the processes • How do they communicate with each other • High availability / symmetric
33. Multithreaded considerations • Sync or async code flow • Usage of thread pools
34. After several iterations • Create a sequence diagram • UML draw.io Free online Visio alternative
35. Iterating over the code is like solving a puzzle
37. Creating anchors “In climbing, a piton (/ˈpiːtɒn/; also called a pin or peg) is a metal spike (usually steel) that is driven into a crack or seam in the rock with a hammer, and which acts as an anchor to protect the climber against the consequences of a fall, or to assist progress in aid climbing” (wikipedia)
38. Creating anchors • Read the code • If we can debug - put breakpoints • Read existing logs • Understand through unitests
39. Creating anchors continued • Change the log level • Add more logging statements • Check history in source control - commits and comments
40. Creating anchors - IDE Bookmarks
41. Creating anchors - the analog way • Works too!
42. Backend
43. Example - utilities to explore the JVM • JConsole • VisualVM • (more tools exist)
44. Getting more hints • Explore threads names • Thread dump - check thread source • Take a heap dump - explore classes • Check which objects a map contains
45. Hint on the database Hint on the scheduler
46. Network analyzer / packet sniffer • Software that intercepts and logs traffic ◦ decodes data ◦ shows packet fields • For example - Wireshark • Define filters for efficiency
47. Example
48. Goal • Send a SOA envelope to a service • Sender - Ruby code • Target - Java SOA service
49. Reports Engine 1 Reports Engine 2 Reports Engine 3 UI Reports Engine 4 http /engine1 http /engine2 http /engine3 http /engine4 SOA Tomcat SOA SOA SOA SOA
50. Steps • Basic understanding what SOA is • Intercept traffic of the existing Java code • Learn how a SOA envelope should look like • Write Ruby code to create a SOA envelope
51. Frontend
52. Frontend • Which libraries are used • Search for strings / translation files • Search for icon files location and usage • Learn how UI calls backend
53. Browser developer tools • FireBug • Chrome developer tools
54. Following the code • Which css and js are downloaded • Add watches in the script tab • Set a breakpoint ◦ Use ‘debugger;’ keyword in code ◦ Set a breakpoint • Inspect elements
55. Inspect the logo 160px * 159px File type and location Press here to switch to element selection mode Inspecting elements
56. Inspect the logo 160px * 159px Inspecting elements
57. Useful options • Network tab • Copy as curl • Copy response • Copy link address
58. Following network requests Alissa Bonas @ PyTN 2016
59. Analyzing response Alissa Bonas @ PyTN 2016
60. Analyzing response - zoom in JSON response
61. REST Client • Learn how REST API is designed
62. In short...
64. Thank you! mikeyteva
65. Graphics credits • Signpost by Lubos Volkov • Clock by Kid Mountain • Idea by Matt Wasser • Gauge by Joel Avery from the Noun Project • Tomcat-logo by The Apache Tomcat Project Team • Loop by useiconic.com from the Noun Project • Magnifying Glass by Musket from the Noun Project • Puzzle Piece by icon 54 from the Noun Project • Writing by Creative Stall from the Noun Project • Sherlock by James Keuning, the Noun Project • Twitter by Lubos Volkov, the Noun Project • Mug by Alex Getty from the Noun Project • Diamond by MarkieAnn Packer from the Noun Project • Light Bulb by artworkbean, the Noun Project • Questions by Rediffusion from the Noun Project • cancel by Jevgeni Striganov from the Noun Project • Movies posters - http://www.impawards.com/