Une plateforme d'expertise en Algèbre Linéaire Creuse un mariage réussi avec DIET - Ronan Guivarch

Une plateforme d'expertise en Algèbre Linéaire Creuse un mariage réussi avec DIET.
Présentation de Ronan Guivarch (Maître de Conférences, CNRS, IRIT, ENSEEIHT) lors des Rencontres SaaS, Cloud & innovation organisées par SysFera le 23 mai à Clamart.

1. Rencontres SaaS, Cloud & innovation 23 mai 2012 - Clamart organisées par Grid-TLSE - une plateforme d'expertise en Algèbre Linéaire Creuse : un mariage réussi avec DIET Grid-TLSE - une plateforme d'expertise en Algèbre Linéaire Creuse : un mariage réussi avec DIET P. Amestoy, F. Camillo, M. Daydé, R. Guivarch, A. Hurault, JY. L'Excellent (*), C. Puglisi IRIT­INPT(ENSEEIHT) (Toulouse) (*)LIP­ENS (Lyon) http://gridtlse.org
2. june 2010 OUTLINE OUTLINE ➢ Overview of the project ➢ Platform Specifications ➢ GRID­TLSE & DAGDA ➢ GRID­TLSE & SysFera
3. june 2010 Overview of the project Overview of the project The GRID­TLSE project has been initially funded by the French Ministery through ACI "Globalisation des Ressources Informatiques et des Données". It has started in 2003. Currently, it is supported by the ANR (Agence National de la Recherche) through: ➢ the COOP project (ANR­09­COSI­001) funded by the French ANR COSINUS program. ➢ The FP3C project (ANR­JTIC 2010­2013) Previously, the GRID­TLSE project was part of other projects: ➢ the SOLSTICE project (ANR­06­CIS6­010). ➢ the ANR LEGO project 2005­2009 (ANR­CICG05­11). ➢ the ReDIMSoPS project through the CNRS/JST (Japan) cooperation.
4. june 2010 Goal of GRID-TLSE site Goal of GRID-TLSE site T TEST FOR L LARGE S SYSTEMS OF EQUATIONS T TEST: L LARGE S SYSTEMS OF EQUATIONS: ✔ It gives facilities to share matrices ✔ It provides a test environment for expert and non-expert users of sparse linear algebra software ✔ It helps non-expert users in choosing the right solvers and its parameters for a given problem
5. june 2010 Test on sparse matrices Test on sparse matrices The GRID­TLSE web site allows an environmental testing Examples of expertise:  Memory required to factor a given matrix  Error analysis as a function of the threshold pivoting value  Minimum time on a given computer to factor a given unsymmetric matrix  Which ordering heuristic is the best one for solving a given problem? Each question corresponds to a «scenario scenario» OP1 OP2 IN OUT
6. june 2010 Goal of GRID-TLSE site Goal of GRID-TLSE site GRID GRID­TLSE Why using the Grid? ✔ Sparse linear algebra software makes use of sophisticated algorithms for (pre-/post-) processing the matrix. ✔ Multiple parameters interfere for efficient execution of a sparse direct solver: ✔ Ordering; ✔ Amount of memory; ✔ Architecture of computer; ✔ Available libraries. ✔ Determining the best combination of parameter values is a multi-parametric problem. => Well-suited for execution over a Grid
7. june 2010 Goal of GRID-TLSE site Goal of GRID-TLSE site The GRID­TLSE web site provides facilities to:  Access to collections of public matrices  Upload matrices  Create private groups to share matrices Sharing test problems
8. june 2010 Platform Specifications Platform Specifications U Users sers Two types of users ✔ Non-expert users that want to proceed to some tests over their problems (matrices) ✔ Expert users that deploy tools (solvers) and specify scenarios
9. june 2010 Platform Specifications Platform Specifications Expert U Expert User ser Expert GRID-TLSE Data Base Prune: Solver Descriptor Geos: Scenario Editor User
10. june 2010 An example of scenario An example of scenario Comparison of direct solvers with their default parameters
11. june 2010 SCENARIO: RUN_SOLVE_DIRECT SCENARIO: RUN_SOLVE_DIRECT Direct solver Geos: scenario editor Geos: scenario editor
12. june 2010 SCENARIO: RUN_SOLVE_DIRECT SCENARIO: RUN_SOLVE_DIRECT Direct solver A : symmetric real Geos: scenario editor Geos: scenario editor
13. june 2010 Platform Specifications Platform Specifications Non­expert U Non­expert User ser ✔ Connects to WebSolve ✔ Uploads his matrices ✔ Chooses a scenario and Construct an Expertise ✔ Fills the expertise inputs (In our example: ✔ Computers ✔ Matrices) User Computer Grid Computer Grid user request
14. june 2010 SCENARIO: RUN_SOLVE_DIRECT SCENARIO: RUN_SOLVE_DIRECT Direct solver A : symmetric real soft1 soft2 soft3 Parameters and constraint description
15. june 2010 Platform Specifications Platform Specifications Non­expert U Non­expert Users sers ✔ Connects to WebSolve ✔ Uploads his matrices ✔ Chooses a scenario and Construct an Expertise ✔ Fills the expertise inputs (In our example: ✔ Computers ✔ Matrices) ✔ Run the Expertise User Computer Grid Computer Grid results user request
16. User Computer Grid Computer Grid Matrices Solvers User request Expertise Request Workflow (set of experiments) XML description of experiments Results Scenario Executions Execution of a scenario OP1 OP2 IN OUT
17. User Computer Grid Computer Grid User request Expertise Request Workflow (set of experiments) XML description of experiments Results Scenario Display of results Executions Execution of a scenario OP1 OP2 IN OUT
18. june 2010 Results Results
19. june 2010 Grid-TLSE & DAGDA • Context: ➢ in an expertise, many experiments with the same matrix and many solvers (or a solver with different parameter values) and many computers ➢ experiments launched in parallel • Our previous solution: ➢ one data transfer (matrix) from the computer where we store the matrices to the selected computers ➢ drawbacks: ➢ many transfers at the same time (huge utilization of the network) ➢ many instances of the same matrix on a computer
20. june 2010 Grid-TLSE & DAGDA • Utilization of Dagda: • New client and service ➢ two main parameters: a matrix and a computer ➢ purpose: find if the matrix is registered with DAGDA and is placed on the selected computer ➢ if the matrix is not yet registered, an alias is created to associate the name of the matrix (unique) with its ID and the matrix is transferred on the selected computer • How do we do now? ➢ a client is launched for each experiment in sequential ➢ we are sure that when we, now, launch the computational experiments, the matrix is registered in DAGDA and exists (with a single instance) on the selected computers
21. june 2010 Grid-TLSE & DAGDA • Advantages: ➢ an unique data transfer from the computer where we store our matrices to the GRID (the other transfers are managed by DAGDA) ➢ an unique instance of the matrix on a computer ➢ some time results: Matrix - WithoutShields.mm (176359Kb) durée expertise (sec) 00:12:40 00:12:58 00:13:15 00:13:32 00:13:49 00:14:07 00:14:24 00:14:41 00:14:59 00:15:16 Run 1 DAGDA (tlse-ma-sed) Run 2 DAGDA Run 3 DAGDA OFF Run 4 DAGDA (ma-sed)
22. june 2010 Grid-TLSE & DAGDA • Things to improve ➢ many runs of the new client are unnecessary: once the matrix is registered and placed on a computer, all the runs with the same matrix and the same computer will do nothing (work on the GRID-TLSE side) ➢ when the DIET/DAGDA infrastructure is stopped, we lose everything (work with the DIET/SysFera team)
23. june 2010 GRID-TLSE & SysFera • http://www.sysfera.fr/-Solveurs- • A simple scenario is available with simulated solvers