Autofunk: a fast and scalable framework for building formal models from production systems (DEBS 2015)

This paper proposes a model inference framework for production systems distributed over multiple devices exchanging thousands of events. Building models for such systems and keeping them up to date is time consuming and expensive, thus not adequately taken care of. Our framework, called Autofunk and designed with the collaboration of our industrial partner Michelin, combines formal model-driven engineering and expert systems to infer formal models that can be used to perform analyses, e.g. test case generation, or help diagnose faults in production by highlighting faulty behaviours. Given a large set of production events, we infer exact models that only capture the functional behaviours of a system under analysis. In this paper, we introduce and evaluate our framework on a real Michelin manufacturing system, showing that it can be used in practice.

Online slides: http://slides.williamdurand.fr/debs2015/
Sources: https://github.com/willdurand-slides/debs2015

Transcript

1. Autofunk, a fast and scalable framework for building formal models from [Michelin] production systems. Sébastien Salva, William Durand — July 1st, 2015 / DEBS'15
3. Quick Tour @ Michelin
4. A factory is divided into several workshops, one for each step of the manufacturing process.
5. A production system is composed of devices, production machines, and one or more software to control them.  In our case, we target a single workshop only.
6. Software exchange information with points and machines by sending and receiving production events. 1 7 - J u n - 2 0 1 5 2 3 : 2 9 : 5 9 . 5 0 | 1 7 0 1 1 | M S G _ I N [ p i d : 1 ] [ n s e c : 8 ] [ p o i n t : 1 ] . . . 1 7 - J u n - 2 0 1 5 2 3 : 2 9 : 5 9 . 6 1 | 1 7 0 2 1 | M S G _ O U T [ p i d : 1 ] [ n s e c : 8 ] [ p o i n t : 3 ] . . . 1 7 - J u n - 2 0 1 5 2 3 : 2 9 : 5 9 . 7 0 | 1 7 0 1 1 | M S G _ I N [ p i d : 2 ] [ n s e c : 8 ] [ p o i n t : 2 ] . . . A simple example of 3 events in a human readable format
7. Production events are exchanged in a binary format (custom protocols), through centralized exchanging systems.
8. Each production event is tied to a product (e.g. tire), identified by a product identifier (p i d ). Gathering all production events related to a product allows to retrieve what happened to it.
9. Background
10. Developement Teams POV 100+ applications running in production Not (fully) covered by tests Documentation most likely outdated MUST be maintained for ~20 years!
11. Customers (Factories) POV Stability over anything else Maintenance periods are planned, but rather long (> 1 week) 1h (unexpected) downtime = 50k $
12. Testing such production systems is complex, and takes a lot of time as it implies the physical devices, and there are numerous behaviours.
13. These behaviours could be formally described into a model. But writing such models is an heavy task and error prone.  Not suitable for Michelin applications.
14. Our Approach (1/2) By leveraging the information carried by the production events, we build formal and exact models (STS) that describe functional behaviours of a production system under analysis.
15. Our Approach (2/2) Michelin's exchanging systems guarantee the order in which the production events occured. We now capture the events directly into these systems to avoid event loss, reordering, and/or duplication of the production events.
16. The Big Picture
17. In Depth Autofunk
18. Autofunk Combines different fields: model inference, expert systems, and (now) machine learning Written in Java 8, reusing powerful libraries (e.g. , ) More a Proof of Concept than a production- ready tool To be open sourced (no ETA yet) Spark Drools
19. Architecture
20. Experimentation 10 million production events (20 days)  161,035 traces  S R ( S ) 77,058 branches 1,587 branches 43,536 branches 1,585 branches  2 entry points here It took 5 minutes to build the two models.
21. Work In Progress
22. Offline Passive Testing Inferred models are used as specifications Another set of traces is collected on a system under test S U T (new or upgraded)  Does S U T conforms to the specifications?
23. Conclusion Fast and efficient technique to infer formal models The more production events, the better! But a few technical issues to tackle (memory consumption for instance)
24. Future Work Deploying Autofunk as a real solution (WIP) Offline passive testing (WIP) Online passive testing
25. Thank You. Questions?