Autofunk: a fast and scalable framework for building formal models from production systems (DEBS 2015)

This paper proposes a model inference framework for production systems distributed over multiple devices exchanging thousands of events. Building models for such systems and keeping them up to date is time consuming and expensive, thus not adequately taken care of. Our framework, called Autofunk and designed with the collaboration of our industrial partner Michelin, combines formal model-driven engineering and expert systems to infer formal models that can be used to perform analyses, e.g. test case generation, or help diagnose faults in production by highlighting faulty behaviours. Given a large set of production events, we infer exact models that only capture the functional behaviours of a system under analysis. In this paper, we introduce and evaluate our framework on a real Michelin manufacturing system, showing that it can be used in practice.

Online slides: http://slides.williamdurand.fr/debs2015/
Sources: https://github.com/willdurand-slides/debs2015

Transcript

1. Autofunk, a fast and scalable
   framework for building formal
   models from [Michelin]
   production systems.
   Sébastien Salva, William Durand — July 1st, 2015 / DEBS'15
   

   View Slide

2. View Slide

3. Quick Tour @ Michelin
   

   View Slide

4. A factory is divided into several workshops,
   one for each step of the manufacturing process.
   

   View Slide

5. A production system is composed of devices, production
   machines, and one or more software to control them.
    In our case, we target a single workshop only.
   

   View Slide

6. Software exchange information with points and machines by
   sending and receiving production events.
   1
   7
   -
   J
   u
   n
   -
   2
   0
   1
   5 2
   3
   :
   2
   9
   :
   5
   9
   .
   5
   0
   |
   1
   7
   0
   1
   1
   |
   M
   S
   G
   _
   I
   N [
   p
   i
   d
   : 1
   ] [
   n
   s
   e
   c
   : 8
   ] [
   p
   o
   i
   n
   t
   : 1
   ] .
   .
   .
   1
   7
   -
   J
   u
   n
   -
   2
   0
   1
   5 2
   3
   :
   2
   9
   :
   5
   9
   .
   6
   1
   |
   1
   7
   0
   2
   1
   |
   M
   S
   G
   _
   O
   U
   T [
   p
   i
   d
   : 1
   ] [
   n
   s
   e
   c
   : 8
   ] [
   p
   o
   i
   n
   t
   : 3
   ] .
   .
   .
   1
   7
   -
   J
   u
   n
   -
   2
   0
   1
   5 2
   3
   :
   2
   9
   :
   5
   9
   .
   7
   0
   |
   1
   7
   0
   1
   1
   |
   M
   S
   G
   _
   I
   N [
   p
   i
   d
   : 2
   ] [
   n
   s
   e
   c
   : 8
   ] [
   p
   o
   i
   n
   t
   : 2
   ] .
   .
   .
   A simple example of 3 events in a human readable format
   

   View Slide

7. Production events are exchanged in a binary format (custom
   protocols), through centralized exchanging systems.
   

   View Slide

8. Each production event is tied to a product (e.g. tire),
   identified by a product identifier (p
   i
   d
   ).
   Gathering all production events related to a product
   allows to retrieve what happened to it.
   

   View Slide

9. Background
   

   View Slide

10. Developement Teams POV
    100+ applications running in production
    Not (fully) covered by tests
    Documentation most likely outdated
    MUST be maintained for ~20 years!
    

    View Slide

11. Customers (Factories) POV
    Stability over anything else
    Maintenance periods are planned,
    but rather long (> 1 week)
    1h (unexpected) downtime = 50k $
    

    View Slide

12. Testing such production systems is complex,
    and takes a lot of time as it implies the physical
    devices, and there are numerous behaviours.
    

    View Slide

13. These behaviours could be formally described into a model.
    But writing such models is an heavy task and error prone.
     Not suitable for Michelin applications.
    

    View Slide

14. Our Approach (1/2)
    By leveraging the information carried by the production events,
    we build formal and exact models (STS) that describe functional
    behaviours of a production system under analysis.
    

    View Slide

15. Our Approach (2/2)
    Michelin's exchanging systems guarantee the order in which the
    production events occured. We now capture the events directly
    into these systems to avoid event loss, reordering, and/or
    duplication of the production events.
    

    View Slide

16. The Big Picture
    

    View Slide

17. In Depth Autofunk
    

    View Slide

18. Autofunk
    Combines different fields: model inference,
    expert systems, and (now) machine learning
    Written in Java 8, reusing powerful libraries
    (e.g. , )
    More a Proof of Concept than a production-
    ready tool
    To be open sourced (no ETA yet)
    Spark Drools
    

    View Slide

19. Architecture
    

    View Slide

20. Experimentation
    10 million production events (20 days)
    
    161,035 traces
    
    S R
    (
    S
    )
    77,058 branches 1,587 branches
    43,536 branches 1,585 branches
     2 entry points here
    It took 5 minutes to build the two models.
    

    View Slide

21. Work In Progress
    

    View Slide

22. Offline Passive Testing
    Inferred models are used as specifications
    Another set of traces is collected on a system
    under test S
    U
    T
    (new or upgraded)
    
    Does S
    U
    T
    conforms to the specifications?
    

    View Slide

23. Conclusion
    Fast and efficient technique to infer formal models
    The more production events, the better!
    But a few technical issues to tackle (memory
    consumption for instance)
    

    View Slide

24. Future Work
    Deploying Autofunk as a real solution (WIP)
    Offline passive testing (WIP)
    Online passive testing
    

    View Slide

25. Thank You.
    Questions?
    

    View Slide