Performance and Regression tests for Simulation

7th LHCb Computing Workshop

Transcript

1. Performance and Regression tests for Simulation Sasha Mazurov, Nigel Watson, Timothy Williams 2016-06-03
2. Content • Regression tests • LHCb Performance and Regression (PR) service 2016-06-03 7th LHCb Computing Workshop
3. Simulation Regression Tests 1. Analysis of cross sections for different particles recorded in specific materials 2. Energy deposition, energy flow and stopping power in a sampling calorimeter 3. Analysis of radiation length 4. Analysis of energy deposition by charged particles in the Velo's silicon 5. Analysis of photon emission by charged particles in the Velo's silicon 2016-06-03 7th LHCb Computing Workshop
4. 2016-06-03 7th LHCb Computing Workshop LHCbPR2: Overview
5. 2016-06-03 7th LHCb Computing Workshop LHCbPR2: Services • API service: • https://gitlab.cern.ch/lhcb-core/LHCbPR2BE • https://hub.docker.com/r/mazurov/lhcbpr2be • Web Frontend • https://gitlab.cern.ch/lhcb-core/LHCbPR2FE • https://hub.docker.com/r/mazurov/lhcbpr2fe • ROOT service (return ROOT objects in JSON format): • https://gitlab.cern.ch/lhcb-core/LHCbPR2ROOT • https://hub.docker.com/r/mazurov/lhcbpr2root • Apache proxy server and security control: • https://gitlab.cern.ch/amazurov/LHCbPR2 • https://hub.docker.com/r/mazurov/lhcbpr2all • Nightly Build and Test tools: • https://gitlab.cern.ch/lhcb-core/LbNightlyTools(lhcbpr2 branch) • https://gitlab.cern.ch/lhcb-core/LHCbPR2HD
6. 2016-06-03 7th LHCb Computing Workshop Database Schema
7. REST API • https://lblhcbpr2.cern.ch/api - outputs all possible service endpoints • https://lblhcbpr2.cern.ch/api/docs - interactive documentation 2016-06-03 7th LHCb Computing Workshop
8. REST API features • We can use it for all CRUD operations (create, read, update, delete) • Can output arbitrary JSON (examples later) • Authentication plugins: e.g. shibboleth for web and tokens for console applications • Output pagination, throttling and requests’ cache allow to avoid service overloading. 7th LHCb Computing Workshop 2016-06-03
9. Examples • Jobs list - /jobs • Job details - /jobs/1 – We can expand any object’s relationships. • Nested requests - /jobs/1/results/ – Output job attributes’ values • Output filtering - /compare/?ids=1,2&contains=Event – Show and group attributes of jobs with id 32 and 32. Attribute name should contain “Event” string 7th LHCb Computing Workshop 2016-06-03
10. Web Frontend 7th LHCb Computing Workshop Only http requests from browser to REST server, no backend behind http://lblhcbpr2.cern.ch (from the CERN network) 2016-06-03
11. Reusable Web Components = <div search-jobs="search-jobs" on-found="onJobsFound(searchParams)”> </div> onJobsFound – client’s custom callback function that get search form results: ids of application, options, Versions New one <jsroot> - Draw JSROOT plots (next slides) More components in development 2016-06-03 7th LHCb Computing Workshop
12. JSROOT • Root service. It returns objects from the ROOT files in the JSON format: – https://lblhcbpr2.cern.ch/root/?files=5%2FFTFP_p roton_Al.root&items=Total 2016-06-03 7th LHCb Computing Workshop
13. JSROOT plots 2016-06-03 7th LHCb Computing Workshop
14. 2016-06-03 7th LHCb Computing Workshop
15. For Developers • We have prepared a set of reusable JavaScript components that simplifies work with REST service and introduce common UI elements for LHCbPR • Well organized modules’ directory structure 7th LHCb Computing Workshop 2016-06-03
16. Frontend and API Summary • API – Updated database schema for proper support of application versions, build slots, thresholds, attribute groups. – Ready for production RESTful service that can be used from any application – Fixed a structure of modules in web frontend – Prepared a set of web components and services. – Compare jobs’ results module. Framework is ready for developing (migration) of the rest of analysis modules. 7th LHCb Computing Workshop 2016-06-03
17. Geant4 Standalone Tests in LHCbPR2 2016-06-03 7th LHCb Computing Workshop Geant4 Standalone Tests 1. Analysis of cross sections for different particles recorded in specific materials (hadronic tests) 2. Energy deposition, energy flow and stopping power in a sampling calorimeter (electromagnetic tests)
18. 2016-06-03 7th LHCb Computing Workshop Hadronic Tests in LHCbPR2 https://lblhcbpr2.cern.ch/#/app/geantstandalone
19. Electromagnetic Tests • https://lblhcbpr2.cern.ch/#/geanttestem3 • https://lblhcbpr2.cern.ch/#/geanttestem5 • Issues with JSROOT library: could not draw complex object. • Need to write user friendly analysis modules at the web frontend as it was done for the hadronic tests 2016-06-03 7th LHCb Computing Workshop
20. Migration to Geant4 v10 • Tests does not work yet: – Release a new version of GEANT4 Files updated for G4r10 (LHCBGAUSS-750) • Create new package to host LHCb version of G4 examples (LHCBGAUSS-747) 2016-06-03 7th LHCb Computing Workshop
21. 2016-06-03 7th LHCb Computing Workshop http://chep2016.org/
22. Summary • Simulation regression tests are ready in Geant and Gauss. Need to be validated in Geant4 v10 • LHCbPR2: – virtual machine is setup and configured through central configuration management. – all services are dockerized to simplify development and deployment. API is protected with CERN SSO. Documentation need to be updated. – Need to update web analysis modules – Jenkins and DIRAC should be configured to use new nightly tools in production. 2016-06-03 7th LHCb Computing Workshop