Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MoSKito for JUGF

MoSKito for JUGF

Slides from MoSKito presentation at the Java User Group Frankfurt

Leon Rosenberg

November 13, 2013
Tweet

More Decks by Leon Rosenberg

Other Decks in Technology

Transcript

  1. for JUGF
    by Leon Rosenberg!
    @dvayanu
    1

    View Slide

  2. View Slide

  3. 3
    Who am I?
    Leon Rosenberg - Architect, DevOps. !
    1997 - Started programming with Java. !
    2000 - Started building Portals.!
    2003 - Founded anotheria.net.!
    2007 - Started MoSKito.

    View Slide

  4. View Slide

  5. View Slide

  6. 6
    Today
    Brief Intro - Some Words!
    Live Demo!
    Integration!
    Some More Words!
    Success Stories

    View Slide

  7. Because running a production System 

    without monitoring is like ...
    Why MoSKito
    7

    View Slide

  8. View Slide

  9. Why MoSKito
    MoSKito is a framework that simplifies your
    efforts to measure and optimize the
    performance and runtime behavior of your
    application and makes your architectural
    decisions visible and measurable.
    9

    View Slide

  10. What is MoSKito
    MoSKito is a multi-purpose, non-invasive,
    interval based monitoring system kit for
    collection, storage and instant analysis of
    application’s performance and behavior data.
    10

    View Slide

  11. Key Features
    Collect and Store.!
    Inspect and Monitor.!
    Analyze and Alert.!
    Continuos production profiling without
    performance impacts with Journeys.
    11

    View Slide

  12. Interval based.
    The behavior of a system depends on hour,
    weekday, weather, holidays and good karma.!
    Large amounts of collected data make
    monitoring nonsensitive to anomalies. !
    Inspection of short intervals offers more
    understanding about system’s behavior.
    5m
    t
    5m 5m 5m 5m 5m 5m
    15m 15m
    now
    12

    View Slide

  13. Core Concepts
    13
    Do something measureable, produce stats.

    Service, Filter, Action, Resource, Gateway, Payment Provider.
    Statistic of a use case, i.e. method name, url,

    cumulated producer statistics
    Value type, i.e. request count, avg duration, error count,

    cache hits, payments etc.
    Container for different values for intervals

    View Slide

  14. Core Sections
    Producers and Stats - gather monitoring data.!
    Thresholds - monitor changes in critical
    sections of the application.!
    Accumulators - builds trends and allow visual
    analysis. !
    Journeys - make inner life of the application
    visible.
    14

    View Slide

  15. 15
    A picture is worth 1000 words...
    ... and live presentation is worth 1000 pictures.


    View Slide

  16. Integration
    AOP / CDI / Spring!
    Proxies!
    WEB!
    !
    Guide: https://
    confluence.opensource.anotheria.net/display/
    MSK/Integration+Guide
    16

    View Slide

  17. AOP
    17
    @Monitor
    public class YourClass {
    public class YourClass {
    @Monitor public void firstMonitoredMethod(){...
    @Monitor public void secondMonitoredMethod(){...
    public void notMonitoredMethod(){...
    @Monitor
    public class YourClass {
    public void thisMethodWillBeMonitored(){...
    @DontMonitor public void thisMethodWillBeExcludedFromMonitoring(){
    @Count
    public class PaymentCounter {
    @Count
    public class PaymentCounter {
    /**
    * Electronic card payment (lastchrifteinzug in germany).
    */
    public void ec(){}
    /**
    * Credit card payment.
    */
    public void cc(){}
    /**
    * Payment via paypal.
    */
    public void paypal(){}
    }

    View Slide

  18. AOP + MAVEN
    18


    net.anotheria
    moskito-core
    2.3.2


    net.anotheria
    moskito-aop
    2.3.2




    org.codehaus.mojo
    aspectj-maven-plugin
    1.4



    net.anotheria
    moskito-aop


    1.6
    1.6




    compile






    View Slide

  19. CDI
    19
    @Monitor
    @ProducerRuntime(producerId=”Foo”, category=”my”)
    public class YourClass {
    public class YourClass {
    @Monitor public void firstMonitoredMethod(){...
    @Monitor(“dao”) public void secondMonitoredMethod(){...
    public void notMonitoredMethod(){...
    @Monitor(MonitoringCategorySelector.WEB)
    public class YourClass {
    public void thisMethodWillBeMonitored(){...
    @DontMonitor public void thisMethodWillBeExcludedFromMonitoring(){
    @Count
    public class PaymentCounter {
    @Count
    public class PaymentCounter {
    /**
    * Electronic card payment (lastchrifteinzug in germany).
    */
    public void ec(){}
    /**
    * Credit card payment.
    */
    public void cc(){}
    /**
    * Payment via paypal.
    */
    public void paypal(){}
    }

    View Slide

  20. CDI
    20
    xsi:schemaLocation="!
    http://java.sun.com/xml/ns/javaee!
    http://java.sun.com/xml/ns/javaee/beans_1_0.xsd">!
    !
    net.anotheria.moskito.integration.cdi.CountInterceptor!
    net.anotheria.moskito.integration.cdi.CallInterceptor!
    !
    net.anotheria.moskito.integration.cdi.WebCallInterceptor!
    net.anotheria.moskito.integration.cdi.ServiceCallInterceptor!
    net.anotheria.moskito.integration.cdi.DaoCallInterceptor!
    !
    de.ayn.integration.moskito.ApiCallInterceptor!
    de.ayn.integration.moskito.MobileApiCallInterceptor!
    de.ayn.integration.moskito.WebCallInterceptor!
    !

    View Slide

  21. Proxies
    21
    public interface SimpleService{
    void doSomethingMethod();
    }
    public class SimpleServiceImpl implements SimpleService{
    public void doSomethingMethod(){
    }
    }
    SimpleService service = ProxyUtils.createServiceInstance(new SimpleServiceImpl(), "default", SimpleService.class);
    SimpleService unmonitoredInstance = new SimpleServiceImpl();
    MoskitoInvokationProxy proxy = new MoskitoInvokationProxy(
    unmonitoredInstance,
    new ServiceStatsCallHandler(),
    new ServiceStatsFactory(),
    "SimpleService",
    "service",
    "test-sub-system",
    SimpleService.class
    );
    SimpleService monitoredInstance = (SimpleService)proxy.createProxy();

    View Slide

  22. WEB
    22

    RequestURIFilter
    net.anotheria.moskito.web.filters.RequestURIFilter

    limit
    1000



    RequestURIFilter
    /*


    DomainFilter
    net.anotheria.moskito.web.filters.DomainFilter

    limit
    50



    DomainFilter
    /*

    View Slide

  23. WebUI
    23
    web.xml



    MoskitoUIFilter
    net.anotheria.moskito.webui.MoskitoUIFilter

    path
    /mui/



    MoskitoUIFilter
    /mui/*





    net.anotheria.moskito.webui.util.StartStopListener




    net.anotheria.moskito.web.session.SessionCountProducer




    org.anotheria.moskitodemo.threshold.presentation.listener.SetupThresholds


    View Slide

  24. Multi-purpose
    MoSKito can collect data about almost
    everything:

    Threads.

    Memory.

    Caches.

    Storages.

    Services.

    !
    Averages.

    ExecutionTime.

    RequestCount.
    CPU Usage.

    IOStats.

    !
    Registrations.

    Payments.

    Conversion.

    Partner Performance.

    Online Users.

    ScamScores.
    Load distribution.

    User-dependent load
    (guest/member).

    Call traversal.

    Path-dependent
    measurement.

    !
    ... whatever you
    need!
    24

    View Slide

  25. What else ?
    Memory leak detection in storages (maps etc)
    with put/get ratios, efficiency checks with get/
    put/remove/miss ratios.!
    Cache/Proxies hit rates in warmed and cold
    states.!
    Error monitoring, which components produces
    (handled or unhandled) errors.!
    Concurrent load on components (number of
    parallel requests). Lock contention
    optimization.
    25

    View Slide

  26. Capacity planing
    What do I have to buy to handle 5.000 more
    concurrent users?!
    Memory per user. !
    Requests per user.!
    CPU Time per user.!
    Separation by domain (multiple sites), guest/
    member/paying member traffic, male/female
    traffic.
    26

    View Slide

  27. more?
    Post release analysis (what have changed).!
    Network traffic optimization.!
    Call trace over multiple VMs.!
    ...!
    ...!
    EVERYTHING IS MEASURABLE!
    27

    View Slide

  28. What else
    MoSKito Control !
    !
    MoSKito Central!
    !
    MoSKito @ Barbecue
    28

    View Slide

  29. View Slide

  30. 30

    View Slide

  31. 31
    MoSKito Control

    View Slide

  32. MoSKito Central
    Central server for snapshot storage.!
    Attachable storages can store data into sql- or
    nosql- based databases, xml/json files etc.!
    Runs in remote or embedded mode.
    32

    View Slide

  33. MoSKito To Go
    33

    View Slide

  34. 34

    View Slide

  35. Success stories
    Far too many, but here are some :-)
    35

    View Slide

  36. I
    After a release of a new version huge traffic
    increase on one of the databases was
    detected.!
    The database in question was used by a
    service. There were 20 clients (code
    components) using this service. !
    MoSKito showed that 55% of the traffic to the
    service came from one client. With MoSKito
    inspection we were able to detect which client
    was producing most traffic.
    36

    View Slide

  37. I
    Closer inspection (code review) of the client
    revealed a bug which led to double calls to the
    service. !
    Incident solved in 30 minutes.
    Most traffic
    Created here
    37

    View Slide

  38. II
    Application overall performance was
    insufficient.!
    With moskito journeys and call tree analysis
    we were able to find redundant calls to the
    backend and remove them.!
    Request duration reduced to 50% with 4 hours
    analysis and 4 hours coding effort.
    38

    View Slide

  39. II
    same call over net repeated thrice
    time in
    milliseconds
    39

    View Slide

  40. III
    A bug only reproduceable by one user.!
    Second page of received messages won’t
    show. No exception logged.
    40

    View Slide

  41. Broken Call
    red color indicates
    that the method
    was terminated
    abnormaly
    (exception)
    possible reason - user didn’t exist in the db and the code didn’t know how to
    handle it.
    41

    View Slide

  42. End of Data.
    Thank you.!
    42
    http://www.moskito.org
    https://github.com/anotheria/moskito
    https://github.com/anotheria/moskito-control
    http://search.maven.org/#search%7Cga%7C1%7Cmoskito
    https://itunes.apple.com/de/app/moskito-ui/id531387262?l=en&mt=8
    https://github.com/anotheria/moskito-examples

    View Slide

  43. 43

    View Slide

  44. 44

    View Slide

  45. End of Data.
    Thank you.!
    45
    http://www.moskito.org
    https://github.com/anotheria/moskito
    https://github.com/anotheria/moskito-control
    http://search.maven.org/#search%7Cga%7C1%7Cmoskito
    https://itunes.apple.com/de/app/moskito-ui/id531387262?l=en&mt=8
    https://github.com/anotheria/moskito-examples

    View Slide

  46. 46
    One More Thing
    Where: Berlin, Dahlem!
    !
    When: 18.11.2013!
    !
    What: Zaunberg Talks - DevOps in Real Life
    http://www.zaunberg-talks.de/berlin2013

    View Slide