Physics and Systems Behaviour NURETH-16 Conference, Chicago, USA September 2, 2015 A Methodology for Global Sensitivity Analysis of Transient Code Output Applied to a Reflood Experiment Model using TRACE D. Wicaksono, O. Zerkak, and A. Pautz
Benchmark Background and Motivation Problem: Simulation FEBA Reflood SETF in thermal- hydraulics code (OECD/NEA PREMIUM Benchmark) Goal: Derive the uncertainty of code (reflood) model parameters using experimental data (temperature) Steps Involved: Parameter Selection, Model Parameters Uncertainty Quantification, Uncertainty Propagation in [mm] H = 4.114 [m] FEBA Take away message from the Project: • Methodology that has large uncertainty range (minor update) on the parameter able to cover experimental results of other facility • Methodology that updated the prior distribution of a few selected parameters failed to cover experimental results of other facility • Symptom of over-fitting
perturbation affect code output that is a function? Background and Motivation 1. How to characterize the variation of time-dependent code output? 2. How such variations can be decomposed and apportioned to the inputs variation? Reflood Curves at mid asssembly height of 100 Monte-Carlo Samples Problem: Simulation FEBA Reflood SETF in thermal- hydraulics code (OECD/NEA PREMIUM Benchmark) Goal: Derive the uncertainty of code (reflood) model parameters using experimental data (temperature) Steps Involved: Parameter Selection, Model Parameters Uncertainty Quantification, Uncertainty Propagation To investigate how the prior distribution affects the model output and gain better understanding on (TRACE reflood) model behavior
= 1, … , = 0, … − 1 • Realizations (Code runs) • Output given at each = 0, … − 1 • Total data points × Describing the variation in functional output: Discrete Dataset Describing Variation in Functional Output (1) 1 Using a predefined scalar function • Relevant to the application • Problem-specific (max., min., saddle point, etc.) = 1 =1 − ത 2 Variations of the output described by the variance of some quantities of interest (QoI) Max. Temperature Time of Quenching (time at maximum negative curvature) Which QoI to choose?
in functional output Describing Variation in Functional Output (3) How to have parsimonious description of functional data that capture the shape variation?
and its relevance for data analysis fPCA: Eigenvalues and fPC scores KL is an optimal orthogonal basis functions expansion and functional Principal Component (fPC) scores contains the random character of each curves = ത + =1 ∞ ⋅ fPC scores, calculated for each reflood curve Quantities of Interest describing functional variation • Karhunen-Loève Transform (KLT) of the covariance function reads: , = =1 ∞ ⋅ ⋅ () • : Eigenvalues • : Orthogonal Eigenfunctions (fPC) = න[ − ത ] ⋅
quantitative measure of sensitivity globally with less model assumption Sobol’ Indices 1 , 2 , 3 = 1 + 2 + 3 1 , 2 , 3 = 1 3 + 2 0.8 + 3 20 1 , 2 , 3 = 1 ⋅ 2 + 1 ⋅ 3 + 1 ⋅ 2 ⋅ 3 Additive, Linear Additive, Non-Linear Interacting, Non-Linear Examples: Three kinds of 3-parameter model 2 Sensitivity measures from the decomposition: Sobol’ Indices Definition Meaning Main-effect (1st –order) ≡ ~ Output variance cut if fixed additive effect of each inputs separately Total Effect ≡ 1 − ~ ~ Output variance left if other than fixed total effect of with all its interactions
(of 26) are influential in the reflood temperature transient Notes on Screening • Parameter screening using the Morris Screening method • Closure laws, boundary conditions, and spacer grid model parameters are relatively influential (10 parameters) • 16 parameters are excluded from further analysis • Generate 2’000 MC samples on the 10-parameter model What does influential mean? Check run with 50 MC samples for each parameter set
in-house scripting and open-source tools Simulation Experiment Design of Experiment Sensitivity Analysis Post-processor Functional Data Analysis Post-processor • Simple Random Sampling (SRS) • Latin Hypercube Sampling (LHS) • Sobol’ Sequence • Morris factorial design TRACE Launcher • Basis function expansion (B-spline) • Curve registration • Functional Principal Component Analysis • Sobol’ Indices (1st- and total) • Elementary effects • … • Parallel Execution • Output management Results Reporting Simulation experiment on TRACE reflood model to decompose mid-assembly temperature variance: • Sobol’ Indices estimated using Janon et al. estimator (1st) and Jansen est. (total) • 2’000 Sobol’ sequence samples on the 10-parameter model (24’000 TRACE runs) • 10’000 bootstrap samples for MC error assessment
static quantities in reflood, TRACE model is additive (minor interaction) Sensitivity Analysis of Static Quantities 48% 12% 8% 8% Spacer Grid 18% 24% 22% 20% Other 1st Order 10% Other 1st Order 15% Interaction 7% Interaction 9% Max. Temperature σ = 60 [K], σ = % Quenching Time σ = 61 [s], σ = % (66%) DFFB Closure Laws Bootstrap samples boxplot (CI: Confidence Interval) Median 50% 95% CI 1 Using a predefined scalar function
the functional variation of a functional dataset fPCA: Eigenfunctions Eigenfunction [K] Temperature [K] 1 , 2 ≈ =1 3 ⋅ 1 ⋅ 2 Perturbation (±) around the mean by the eigenfunction assists in their interpretation Temperature Ramp Temperature Descent Quenching 55% 32% 5% KL Transform in finite series Using fPCA on functional dataset 2
is driven by DFFB and Grid HT parameters (no interact.) Sensitivity Analysis of 1st fPC scores • The model is additive with respect to temperature raise • Given temperature data, these parameters (and their uncertainties) by structure of the model can be estimated 21% 21% 25% 20% Other Main-Effect 5% Interaction 8% DFFB Parameters (66%) Sum of Main-Effect Indices = 92%
is driven by the interactions of all model parameters Sensitivity Analysis of 2nd fPC Scores Small 1st-order effect: σ = % Boundary Conditions Physical Model Parameters Δ indices = Total Interaction Effect • The model is interacting w.r.t to this functional behavior • Parameters estimation might suffer from non-identifiability (non-uniqueness)
output that is a function? 1. Using a scalar function (problem specific but relevant) 2. Using functional principal component analysis (parsimonious, but less straightforward) Combining fPC and Variance-Based sensitivity analysis gives more complete description of model behavior; functional output variation and attributing these variations to the responsible input … and apply them to the OECD/NEA Benchmark Problem using TRACE… • Reducing the variation of the max. temperature and time of quenching is possible in terms of the structure of the model • Parameter interaction exists for some form of functional variation (possible non-identifiability issue, but which with which not known yet) Decompose them using Sobol’-Saltelli Method
Perret, LRS – Paul Scherrer Institut • Dr. Carl Adamsson, Vattenfall AB • Swiss Federal Nuclear Safety Inspectorate (ENSI) • Swiss Federal Office of Energy (BFM)
Ihle and K. Rust, “FEBA-Flooding Experiments with Blocked Arrays,” Evaluation Report, Kernforschungzentrum Karlsruhe, 1994 2. US NRC, “TRACE v5.0p3 Theory Manual,” United States Regulatory Commission, Washington D.C., 2012 3. Perez et al., “Uncertainty and sensitivity Analysis of a LBLOCA in a PWR NPP: Results of the Phase V of the BEMUSE programme,” Nucl. Eng. Des., vol. 241, pp. 4206-4222, 2011 4. J. Ramsay and B. W. Silverman, “Functional Data Analysis,” 2nd Edition, New York: Springer Science+Business Media, LLC, 2005. 5. G. Li et al., “High Dimensional Model Representations,” J. Phys. Chem. A, 105(33), pp. 7765-7777 (2001) 6. I. M. Sobol’, “Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates,” Math. Comput. Simul., 55, pp. 271-280, 2001 7. T. Homma and A. Saltelli, “Importance measures in global sensitivity analysis of nonlinear models,” Reliab. Eng. Sys. Saf., 52(1), pp. 1-17, 1996 8. A. Saltelli, “Making best use of model evaluations to compute sensitivity indices,” Comput. Phys. Commun., 181(2), pp. 259-270, 2010
Janon et al, “Asymptotic normality and efficiency of two Sobol’ index estimators,” ESAIM Probab. Stat., 18, pp. 342-364, 2014 10. M. D. Morris, “Factorial Sampling Plans for Preliminary Computational Experiments,” Technometrics, vol. 33, no. 2, pp. 161 – 174, 1991. 11. F. Campolongo, J. Cariboni, and A. Saltelli, “An effective screening design for sensitivity analysis for large models,” Environmental Modelling & Software, vol. 22, pp. 1509 – 1518, 2007. 12. Ramsay et al., “fda: Functional Data Analysis,” R Package version 2.4.0, The Comprehensive R Archive Network (CRAN), 2013 13. R Core Team, “R: a Language and Environment for Statistical Computing,” R Foundation for Statistical Computing, Vienna, Austria, 2014, http://www.R- project.org
dcwicaksono
508shuto
ncu_ds
scientia
yamahiro
lcolladotor
tagtag
tlyu0419
psasa
aronwalsh
meshidenn
sshimizu2006
climbteam
dougneiner
bryan
phodgson
jacobian
ufuk
destraynor
chrisshort
addyosmani
moore
shlominoach
zakiyama
geoffreycrofte
