Dirk Mahler on Software Analyse mit jQAssistant & Neo4j

Transcript

1. by
   Software Analysis Using
   jQAssistant and Neo4j
   eJUG Vienna
   Graph Database Austria
   01/2016
   Dirk Mahler
   

   View Slide

2. 2
   AGENDA
    jQAssistant
    Software As A Graph
    About Structures, Rules and Code
    Verifying Rules With The Graph Model
    Wrap Up
    Q&A
   

   View Slide

3. jQAssistant
   Software Analysis Using jQAssistant And Neo4j
   3
   

   View Slide

4. 4
   http://jQAssistant.org
    Open Source: GPLv3
    Current release: 1.1.1
    initiated: 03/2013
    first stable release: 04/2015
    Neo4j Community Edition
    http://neo4j.org
    embedded, no installation necessary
   

   View Slide

5. 5
    The Idea
    Scan software structures
    Store in a database
    Execute queries
    Explore
    Add high level concepts
    Find constraint violations
    Create Reports
   Database
   Find
   all…
   

   View Slide

6. 6
   Command
   Line/
   Maven
   Scan
   Analyze
   Report
   Plugins
   Plugins
   Plugins
   

   View Slide

7. 7
    Available scanner plugins
    Plugin API is public
   *.class
   RDBMS Schema
   Git
   JaCoCo
   FindBugs
   CheckStyle
   Liquibase
   JAR, WAR, EAR
   ZIP
   application.xml
   web.xml
   MANIFEST.MF /META-INF/services/*
   beans.xml
   pom.xml surefire-reports.xml
   GZ
   *.properties
   *.xsd
   M2 Repository
   *.yaml
   

   View Slide

8. 8
    Getting Started – Command Line
    unzip jqassistant.distribution-1.1.1.zip
    cd jqassistant.distribution-1.1.1/bin
    jqassistant.sh scan –f model.jar
    jqassistant.sh scan –f acme.war
    jqassistant.sh scan –f acme.ear
    jqassistant.sh scan –u http://somewhere.com/acme.ear
    jqassistant.sh scan –u
   maven:repository::http://host/releases
    jqassistant.sh scan –u
   rdbms:schema::jdbc:oracle:thin:user/secret@host:1521:sid
    jqassistant.sh server
    http://localhost:7474
   

   View Slide

9. 9
    Getting Started – Maven Project
   
   
   
   com.buschmais.jqassistant.scm
   jqassistant-maven-plugin
   1.1.1
   
   
   
    mvn install
    mvn jqassistant:scan
    mvn jqassistant:server
    http://localhost:7474
   

   View Slide

10. Software As A Graph
    Software Analysis Using jQAssistant And Neo4j
    10
    

    View Slide

11. 11
     All we need is…
     Nodes
     Labels
     Properties
     Relationships
     Modeling is just…
     Taking a pen
     Drawing the structures on a whiteboard (i.e. the database)
     We don‘t need…
     Foreign keys
     Tables and schemas
     Deep knowledge in graph theory
    Type
    EXTENDS
    fqn:org.springframework.samples.
    petclinic.model.Owner
    

    View Slide

12. Project
    Artifact
    Maven
    CONTAINS
    Package
    name:org
    CONTAINS
    Package
    name:springframework CONTAINS
    Package
    name:petclinic
    Package
    CONTAINS
    name:model
    name:petclinic
    CONTAINS
    Package
    name:samples
    CREATES
    

    View Slide

13. RETURNS
    READS
    Field
    DECLARES
    name:address
    Package
    name:model
    Class
    Type
    CONTAINS
    fqn:org.springframework.
    samples.petclinic.
    model.Person
    name:Person
    Annotation
    ANNOTATED_BY
    Type
    OF_TYPE
    fqn:javax.persistence.MappedSuperclass
    Type
    EXTENDS
    Class
    fqn:org.springframework.
    samples.petclinic.
    model.Owner
    name:Owner
    Type
    OF_TYPE
    fqn:java.lang.String
    DECLARES
    Method
    name:getAddress()
    Annotation
    ANNOTATED_BY
    Type
    OF_TYPE
    fqn:javax.persistence.Entity
    

    View Slide

14. 14
    Type
    Type
    Class
    EXTENDS
     Explore an application using queries
     Which class extends from another class?
     Let‘s convert this to ASCII art…
     () as nodes
     -[]-> as directed relationships
    

    View Slide

15. 15
    Type
    Type
    Class
    EXTENDS
     Explore an application using queries
     Which class extends from another class?
     Let‘s convert this to ASCII art…
     () as nodes
     -[]-> as directed relationships
    ()-[]->()
    

    View Slide

16. 16
    C2
    Type
    C1
    Type
    Class
    EXTENDS
     Explore an application using queries
     Which class extends from another class?
     Let‘s convert this to ASCII art…
     () as nodes
     -[]-> as directed relationships
    (c1)-[]->(c2)
    

    View Slide

17. 17
    C2
    Type
    C1
    Type
    Class
    EXTENDS
     Explore an application using queries
     Which class extends from another class?
     Let‘s convert this to ASCII art…
     () as nodes
     -[]-> as directed relationships
    (c1)-[:EXTENDS]->(c2)
    

    View Slide

18. 18
     Explore an application using queries
     Which class extends from another class?
     Let‘s convert this to ASCII art…
     () as nodes
     -[]-> as directed relationships
    (c1:Class)-[:EXTENDS]->(c2:Type)
    C2
    Type
    C1
    Type
    Class
    EXTENDS
    

    View Slide

19. 19
     Explore an application using queries
     Which class extends from another class?
     Pattern matching is the core principle of Cypher
    MATCH
    (c1:Class)-[:EXTENDS]->(c2:Type)
    RETURN
    c1.fqn, c2.fqn
    C2
    Type
    C1
    Type
    Class
    EXTENDS
    

    View Slide

20. Demo #1
    Software Analysis Using jQAssistant And Neo4j
    20
    

    View Slide

21. About Structures, Rules And Code
    Analysis of software systems using jQAssistant and Neo4j
    21
    

    View Slide

22. 22
     Sketch of an architecture
    

    View Slide

23. 23
     Sketch of an architecture
    My Big Fat Demo Application
    

    View Slide

24. 24
     Sketch of an architecture
     Business modules
    Shopping Cart
    Usermanagement
    

    View Slide

25. 25
     Sketch of an architecture
     Allowed dependencies between business modules
    Shopping Cart
    Usermanagement
    

    View Slide

26. 26
     Sketch of an architecture
     Technical layering
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

27. 27
     Sketch of an architecture
     Allowed dependencies between technical layers
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

28. 28
     Sketch of an architecture
     Allowed dependencies between business modules and
    technical layers
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

29. 29
     Sketch of an architecture
     Allowed dependencies between business modules and
    technical layers
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

30. 30
     Sketch of an architecture
     Abstraction between layers (API vs. Implementation)
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

31. 31
     Sketch of an architecture
     Forbidden dependency
    JSF-UI REST
    Business Logic
    Persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

32. 32
     Sketch of an architecture
     Allowed external dependencies per layer
    JSF-UI REST
    EJB
    JPA
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

33.  Translation of architecture rules into a project
    structure
     Java language element: Package
    org.jqassistant.demo
    33
    

    View Slide

34.  Translation of architecture rules into a project
    structure
     Java language element: Package
     Definition of business modules on „top level“
    34
    Shopping Cart
    „org.jqassistant.demo.cart“
    Usermanagement
    „org.jqassistant.demo.user“
    

    View Slide

35. 35
     Translation of architecture rules into a project
    structure
     Java language element: Package
     Technical layers
     …demo.cart.ui
     …demo.cart.rest
     …demo.cart.logic
     …demo.cart.persistence
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

36. 36
     Translation of architecture rules into a project
    structure
     Java language element: Package
     Technical layers
     …demo.cart.ui
     …demo.cart.rest
     …demo.cart.logic.api
     …demo.cart.logic.impl
     …demo.cart.persistence.api
     …demo.cart.persistence.impl
    JSF-UI REST
    Business Logic
    Persistence
    

    View Slide

37. 37
     Package Names
     All packages in a Maven module must be prefixed with
    ${groupId}.${artifactId}
     Example:
    groupId=org.jqassistant
    artifactId=demo
    => org.jqassistant.demo
    

    View Slide

38. 38
     Class Names
     Message Driven Beans must have a suffix „MDB“.
    

    View Slide

39. 39
     Class location
     JPA entities must be located in „model“ packages.
    

    View Slide

40. 40
     Test design
     Every test method must contain at least one assertion.
     Each assertion must provide a human readable message.
    

    View Slide

41. 41
     Abstraction
     Remote APIs must be interfaces declaring only primitives or
    immutables as parameter or return types.
     OSGi-Bundles must only export dedicated API packages.
    

    View Slide

42. 42
    Source Code
    Java, Properties,
    XML, YML
    Complexity Erosion
    Maintainability
    „Code Quality“
    

    View Slide

43. 43
    Source Code
    Java, Properties,
    XML, YML
    People
    Architect, Developer,
    Test
    Documentation
    UML, Wiki,
    readme.txt
    Tools
    Compiler, Static Code
    Analysis, CI
    

    View Slide

44. 44
     Source Code – Abstraction levels
    Folder, File
    File System
    Package, Class, Field, Method, Annotation
    Java
    Abstraction, Immutable, Factory
    Design
    Module, Layer, Dependency
    Architecture
    

    View Slide

45. 45
     Source Code – Automated validation
    Folder, File
    File System
    Package, Class, Field, Method, Annotation
    Java
    Abstraction, Immutable, Factory
    Design
    Module, Layer, Dependency
    Architecture
    

    View Slide

46. 46
     Java elements represent higher level concepts
     Package  Module
     Annotated Class  Entity
     Constraints apply to these concepts
     JPA entities must be located in „model“ packages
    

    View Slide

47. 47
     Concepts and Constraints
     How to make them visible?
     Is there a language to describe them?
    

    View Slide

48. Verifying Rules With The Graph Model
    Software Analysis Using jQAssistant And Neo4j
    48
    

    View Slide

49. 49
     Analyze
     Execution of rules
     Defined in AsciiDoc or XML documents
     Concepts
     Enrich data model
     Constraints
     Detect violations
     Group
     Allow different execution profiles
    

    View Slide

50. 50
    Constraint
    Group
    Concept Concept
    Concept
    Includes
    Requires Requires
    Requires
    

    View Slide

51. 51
     Concept
    == JPA Entities
    [[jpa2:Entity]]
    .Labels all types annotated with @javax.persistence.Entity with
    Jpa and Entity.
    [source,cypher,role=concept]
    ----
    MATCH
    (t:Type)-[:ANNOTATED_BY]->()-[:OF_TYPE]->(a:Type)
    WHERE a.fqn ="javax.persistence.Entity"
    SET t:Jpa:Entity
    RETURN t AS Entity
    ----
    

    View Slide

52. 52
     Concept
    Entity
    Jpa
    Type
    Class
    fqn:org.springframework.
    samples.petclinic.
    model.Owner
    name:Owner
    Annotation
    ANNOTATED_BY
    Type
    OF_TYPE
    fqn:javax.persistence.Entity
    

    View Slide

53. 53
     Constraint
    [[model:JpaEntityInModelPackage]]
    .All JPA entities must be located in packages named "model".
    [source,cypher,role=constraint,requiresConcepts="jpa2:Entity"]
    ----
    MATCH
    (package:Package)-[:CONTAINS]->(entity:Jpa:Entity)
    WHERE
    package.name <> "model"
    RETURN
    entity AS EntityInWrongPackage
    ----
    

    View Slide

54. 54
     Group
    [[default]]
    [role=group,includesConstraints="model:JpaEntityInModelPackage"]
    == Naming Rules
    The following naming rules apply:
    - <>
    

    View Slide

55. 55
     AsciiDoc
     Markup Language
     Rendering to
     DocBook
     HTML
     PDF
     Executable specification
     Coding rules
     Design & Architecture
    

    View Slide

56. Demo #2
    Software Analysis Using jQAssistant And Neo4j
    56
    http://github.com/buschmais/spring-petclinic
    

    View Slide

57. Wrap Up
    Software Analysis Using jQAssistant And Neo4j
    57
    

    View Slide

58. 58
     Holistic and extensible view on code structure and system
    architecture
     Exploration of existing structures
     Validation of rules using concepts from architecture & design
    language
     Executable architecture and design specification
    

    View Slide

59. Mail: info@jqassistant.org
    Web: jqassistant.org
    Twitter: @jqassistant
    Thank You! – Questions?
    by
    

    View Slide