Building a Graph User-Interface for Malware-Analysis by Stefan Hausotte

Transcript

1. Building a Graph User-Interface for Malware- Analysis
2. Stefan Hausotte Team-Lead: „Automated Threat Analysis“ @ G DATA CyberDefense AG Associate Professor for IT-Security @ TU Dortmund Ethan Hasson Senior Web-Developer @ Expero Inc.
3. 01 Domain What kind of problem do we want to solve? 02 Backend GraphQL backend to make data available. 03 Frontend Web-frontend for the users to interact with. 04 Demo Show the interactive web- ui. Building a Graph User-Interface for Malware-Analysis Overview
4. ၈ ၈ What kind of problem do we want to solve? Domain Building a Graph User-Interface for Malware-Analysis Overview
5. ▪ Half a million new malware files each day ▪ Analyzed in sandboxes ▪ All results are stored in a graph database ▪ JanusGraph + Scylla Building a Graph User-Interface for Malware-Analysis Domain Sandbox (VM) Analysis Potentially Malicious Data GRID (Graph Intelligence DB) Static Analysis Analysis
6. ▪ Extracted information ▪ Requested URLs/domains ▪ Accessed Files ▪ Acessed RegKeys ▪ … ▪ Huge knowledge graph with billions of nodes and vertices Building a Graph User-Interface for Malware-Analysis Domain
7. ▪ Provide an easy to use interface to the data ▪ Should be interactive ▪ Reflect the underlying graph structure ▪ Targeted at Malware Analysts Building a Graph User-Interface for Malware-Analysis Domain
8. ၈ ၈ GraphQL backend to make data available. Backend Building a Graph User-Interface for Malware-Analysis Overview
9. Building a Graph User-Interface for Malware-Analysis Backend Scylla Big table database to persist our data. JanusGraph Abstraction over Scylla to model the tables as a graph. GraphQL HTTP interface to query the graph.
10. Building a Graph User-Interface for Malware-Analysis GraphQL More flexible alternative to REST due to its nature as a full query language. type Project { name: String tagline: String contributors: [User] } { project(name: "GrapQL") { tagline } } { "project": { "tagline": "A query language for APIs" } } Type: Query: Result:
11. Building a Graph User-Interface for Malware-Analysis From Schema to JanusGraph + C#
12. Building a Graph User-Interface for Malware-Analysis From Schema to GraphQL
13. Building a Graph User-Interface for Malware-Analysis From Schema to C# Code <vertex label='File' isPartitioned="false"> <properties> <property>Sha256</property> <property></property> <property>SMd5ize</property> </properties> <indices> <uniqueIndex> <type>Composite</type> <property>Sha256</property> </uniqueIndex> <uniqueIndex isOptional="true"> <type>Composite</type> <property>Md5</property> </uniqueIndex> </indices> </vertex> public class FileType : GridVertexType<File> { public FileType(IGrIDAccess access, IOptions<GrIDAccessConfig> options) : base(access, options) { Name = "File"; Field<StringGraphType>("sha256", resolve: x => x.Source.Sha256); Field<StringGraphType>("md5", resolve: x => x.Source.Md5); Field<IntGraphType>("size", resolve: x => x.Source.Size); Field<FileToPEFileFeaturesType>("toPEFileFeatures", resolve: context => access. GetOutgoingEdgeAsync<FileToPEFileFeatures, File, PEFileFeatures>( context.Source )); Field<FileToIconFeaturesType>("toIconFeatures", resolve: context => access. GetOutgoingEdgeAsync<FileToIconFeatures, File, IconFeatures>( context.Source )); ... } } code generation <edge label='FileToIconFeatures'> <multiplicity>MANY2ONE</multiplicity> <connection> <from>File</from> <to>IconFeatures</to> </connection> </edge>
14. Building a Graph User-Interface for Malware-Analysis Graph query with Gremlin Graph gremlin> g.V().has('File','Sha256','0b0d860d8f24a…218216331f3e41f34e60c72dbad90d5'). out('FileToHiveRun'). out('HiveRunToInMemoryResult’). out('InMemoryResultToInMemoryDetection’). values('Name') ==>emotet6 Gremlin Query
15. Building a Graph User-Interface for Malware-Analysis Graph query with GraphQL query { fileBySha256( sha256: "0b0d860d8f24a6…9b33e41f34e60c72dbad90d5" ) { toHiveRun { to { toInMemoryResult { to { toInMemoryDetection { to { name } } } } } } } } GraphQL Request Graph { "data": { "fileBySha256": { "toHiveRun": [ { "to": { "toInMemoryResult": { "to": { "toInMemoryDetection": [ { "to": { "name": "emotet6" } } ] } } } } ] } } } JSON Response
16. ၈ ၈ Web-frontend for the users to interact with. Frontend Building a Graph User-Interface for Malware-Analysis Overview
17. Building a Graph User-Interface for Malware-Analysis Core Open Source Front-End Technologies Cytoscape.js: Open-source graph theory (a.k.a. network) library written in JS. You can use Cytoscape.js for graph analysis and visualization. Tabulator: Create interactive tables in seconds from any HTML Table, JavaScript Array, AJAX data source or JSON formatted data. Apollo Client: A complete state management library for JavaScript apps. Simply write a GraphQL query, and Apollo Client will take care of requesting and caching your data, as well as updating your UI.
18. Building a Graph User-Interface for Malware-Analysis Cytoscape.js Quick Facts • Open Source: Permissive open source license (MIT) • Battle Tested: Large suite of tests that can be run in the browser or the terminal. • Pick a Graph Style: Directed graphs, undirected graphs, mixed graphs, loops, multigraphs, compound graphs, etc. • Good Documentation: Includes live code examples. • Graph Layout Capabilities: Uses layouts for automatically or manually positioning nodes. • Ease of filtering and styling: Supports selectors for terse filtering and graph querying. • Touch Events: Abstracted and unified touch events on top of a familiar event model.
19. Building a Graph User-Interface for Malware-Analysis Apollo Quick Facts • Declarative data fetching: Write a query and receive data without manually tracking loading states • Excellent developer experience: Enjoy helpful tooling for TypeScript, Chrome DevTools, and VS Code • Designed for modern React: Take advantage of the latest React features, such as hooks • Incrementally adoptable: Drop Apollo into any JavaScript app seamlessly • Universally compatible: Use any build setup and any GraphQL API • Community driven: Share knowledge with thousands of developers, thanks to our active open source community
20. Building a Graph User-Interface for Malware-Analysis GraphQL to Visualizing a Graph Using Cytoscape.js
21. Building a Graph User-Interface for Malware-Analysis GraphQL to Visualizing a Graph Using Cytoscape.js query($ids: [Int], $limit: Int!) { vertices(vertexIDs: $ids) { vertexID incomingEdgesCount outgoingEdgesCount incomingEdges(limit: $limit) { from { vertexID incomingEdgesCount outgoingEdgesCount } } outgoingEdges(limit: $limit) { to { vertexID incomingEdgesCount outgoingEdgesCount } } } } { "data": { "vertices": [ { "vertexID": 123, "incomingEdges": [ { "from": {"vertexID": 456} } ], "outgoingEdges": [ { "to": {"vertexID": 789} } ] } ] } } { "nodes": [ {"data": {"id": 123}}, {"data": {"id": 456}}, {"data": {"id": 789}} ], "edges": [ {"data": { "source": 456, "target": 123}}, {"data": {"source": 123,"target": 789}} ] } Gql Response Converted for Cytoscape.js
22. Building a Graph User-Interface for Malware-Analysis Cytoscape.js Core Concepts • Creation • Styles • Layouts • Animations • Data considerations
23. Building a Graph User-Interface for Malware-Analysis Creating the Graph • Initialize Cytoscape • Store Reference for future graph interactions • Supply elements (in this case, inferred nodes and edges) var cy = cytoscape({ container: document.getElementById('cy'), elements: [ { data: { id: 'a' } }, { data: { id: 'b' } }, { data: { id: 'ab', source: 'a', target: 'b' } }] });
24. Building a Graph User-Interface for Malware-Analysis Creating the Graph
25. Building a Graph User-Interface for Malware-Analysis Styling the Graph • During Initialization • Selectors • Styles: Many CSS styles are supported var cy = cytoscape({ {...elementsAndContainer} style: [ { selector: 'node', style: { shape: 'hexagon', 'background-color': 'red' } }] });
26. Building a Graph User-Interface for Malware-Analysis Styling the Graph
27. Building a Graph User-Interface for Malware-Analysis Layouts • During Initialization • Several layouts to choose from including 1st and 3rd party layouts var cy = cytoscape({ {...elementsAndContainer} layout: {name: 'grid'} });
28. Building a Graph User-Interface for Malware-Analysis Layouts • After initialization we can also layout the graph. • This is useful when we give the ability to change layouts to a user. const layout = cy.layout({name: 'grid'}); layout.run();
29. Building a Graph User-Interface for Malware-Analysis Default Cytoscape Layouts • Preset: Puts nodes in the positions you specify manually. • Grid: Places nodes in a well-spaced grid. • Circle: The circle layout puts nodes in a circle. • Concentric: Positions nodes in concentric circles, based on a metric that you specify to segregate the nodes into levels • Breadthfirst: Puts nodes in a hierarchy, based on a breadthfirst traversal of the graph. • Cose: Uses a physics simulation to lay out graphs.
30. Building a Graph User-Interface for Malware-Analysis Animations • Need access to nodes and/or edges in Cytoscape graph • Use the animate function with a node/edge or group Animates a node’s opacity to 0 from its current value const aNode = cy.nodes()[0]; aNode.animate({ duration: 500, style: { opacity: 0 }, easing: 'ease-in-sine', });
31. Building a Graph User-Interface for Malware-Analysis Animations
32. Building a Graph User-Interface for Malware-Analysis Tricks to Proper Animation • Whoa, what about the edge? • We need to also animate the connected edges belonging to the node being hidden Animates a node's edges' opacity to 0 from its current value. … aNode.connectedEdges().animate({ duration: 500, style: { opacity: 0 }, easing: 'ease-in-sine' })
33. Building a Graph User-Interface for Malware-Analysis Tricks to Proper Animation
34. Building a Graph User-Interface for Malware-Analysis Data Considerations There are several strategies we used to help performance and usability • The minimal dataset to accomplish the problem rendered to the screen. (don't render everything) • Search functionality to find nodes of interest. • Ability to hide and show nodes along with collapsing and expanding nodes. • Ability to save a snapshot for sharing or later use. • Ability to only request the x number of connected nodes (in/out). → This was a big one • Ability to remove/add aggregation nodes which are not always helpful to analysts.
35. ၈ ၈ Show the interactive web-ui. Demo Building a Graph User-Interface for Malware-Analysis Overview
36. Building a Graph User-Interface for Malware-Analysis Demo