ISC’23 Agenda • Vector Engine 3.0 (VE30) is the first major update to NEC’s Vector Engine series of vector processors [1,2]. • In this paper, we conduct the first performance evaluation of VE30. • Specifically, we • Describe the overall architecture of VE30 and improvements from its predecessor. • Evaluate the basic performance using industry-standard benchmarks. • Analyze the impact of architectural enhancements/ • Evaluate the real-world performance using workloads including SPEChpc 2021. • Present several performance tuning techniques for VE30. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 2 [1] K. Komatsu et al., “Performance Evaluation of a Vector Supercomputer SX-Aurora TSUBASA,” SC’18. [2] R. Egawa et al., “Exploiting the Potentials of the Second Generation SX-Aurora TSUBASA,” PMBS 2020.
ISC’23 SX-Aurora TSUBASA Vector Supercomputer • SX-Aurora TSUBASA (SX-AT) • Latest generation of NEC’s SX-series supercomputers based on the Vector Engine (VE) processor. • VE is implemented as a PCIe card and attached to the host. • Application runs on the VE and “reverse” offload syscalls to the host. • Vector Engine (VE) • A vector processor tightly integrated with High Bandwidth Memory (HBM), primarilty targeting memory-bound applications. • Can be programmed with standard parallel programming models (MPI+OpenMP). • Powerful autovectorizing compilers for C/C++ and Fortran are provided. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 3 Vector Engine Vector Host (x86) Vector Engine PCIe Switch … InfiniBand HCA https://www.nec.com/en/global/solutions/hpc/sx/ vector_engine.html MPI Syscalls
ISC’23 Improvements from VE20 • Per-core private L3 cache • L3 Cache can be bypassed by software. • Compute-capable LLC • Each LLC bank contains a compute unit to perform accumulation in the LLC. • Better FP32 performance • VE30 only requires 4-byte alignment for FP32 data, while VE20 required 8-byte alignment. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 5 VE Type 20A VE Type 30A VE30 Improve ment # of Cores 10 16 1.6x FP64 Perf./Socket [TFLOP/s] 3.07 4.91 1.6x Memory B/W [TB/s] 1.53 2.45 1.6x Memory Capacity [GB] 48 96 2.0x LLC B/W [TB/s] 3.0 6.4 2.1x LLC Capacity [MB] 16 64 4.0x
ISC’23 Benchmarks for basic performance measurements • HPL1: Compute-intensive benchmark that solves a dense linear system using LU decomposition with pivoting. • BabelStream2: Benchmark that measures the effective memory bandwidth. • HPCG1: Memory-intensive benchmark that solves a sparse linear system using the conjugate gradient method and a geometric multigrid preconditioner. • Himeno benchmark: Memory-intensive benchmark that solves the Poisson equation using the Jacobi method. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 8
ISC’23 Compute-capable LLC Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 13 for (int i = 0; i < n; i++) { y[l[i]] = y[l[i]] + x[i]; } On VE20, the user was responsible for choosing from: • Scalar: Computes using scalar instructions only (compiler’s default). • ivdep: Computes using vector instructions only. Must ensure that l[i] do not overlap (requires use of compiler directive). • list_vector: Computes using vectorized instructions, then corrects results for overlapped indices using scalar instructions (requires use of compiler directive). On VE30: • vlfa: Dedicated instruction for index vector accumulation (compiler’s default). Core LLC Memory l[i] x[i] y[l[i]] Each LLC bank has a compute unit Indexed vector accumulation used in finite element method, particle method, etc.
ISC’23 Hardware support for indexed vector accumulation Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 14 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1 2 4 8 12 16 20 24 28 32 GFLOP/s # of Overlapping Indices VE20 scalar VE20 list_vector VE30 scalar VE30 list_vector VE30 vlfa Single-core performance of a microbenchmark that performs indexed vector accumulation with varying degree of address overlaps (x out of 32 addresses overlap). vlfa falls behind scalar vlfa is 3.48x faster than list_vector Since vlfa is always faster than list_vector and high degree of overlaps rarely happens in real-world applications, the user can always use vlfa.
ISC’23 SPEChpc 2021 • Benchmark suite developed by the Standard Performance Evaluation Corporation (SPEC) • “a set of application benchmark suites using a comprehensive measure of real-world performance for the state-of-the-art HPC systems”1 • Programming models: • Used MPI+OpenMP on VE20/30, A64FX and IceLake-SP, and MPI+OpenACC on A100. • All benchmarks are executed as-is (no source modification) on all platforms. • Workload sizes: • Tiny (9 benchmarks, requires ~60 GB of memory): Executed using the minimum possible number of sockets on each platform. Speedup is normalized by the number of used sockets. • Medium (6 benchmarks, requires ~4 TB of memory): Executed using 128 sockets on all platforms. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 17 1 https://www.spec.org/hpc2021/
ISC’23 SPEChpc 2021 tiny workload performance analysis • LBM, TeaLeaf, CloverLeaf, POT3D • Memory-bound and achieves good performance on VE. • In CloverLeaf, kernels that perform gather are slower than A100. • SPH-EXA: • Octree traversal for nearest neighbor search cannot be vectorized. • Could benefit from (reverse) offloading nearest neighbor search to the host CPU. • HPGMG-FV • Inner-most loop is too short (32 iterations) for VE where a register holds 256 elements. • Could benefit from collapsing loops to increase the average vector length. • miniWeather • Memory-bound kernels are faster than A100 but compute-bound kernels are slower. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 19
ISC’23 • Trend looks similar to tiny since the both the problem size and # of nodes are increased. • VE30’s MPI communication performance is worse than A100 in CloverLeaf, POT3D and miniWeather – requires further investigation (see paper for MPI profiling results). Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 20 SPEChpc 2021 medium workload results 0 5 10 15 20 25 30 35 40 LBM TeaLeaf CloverLeaf POT3D HPGMG-FV miniWeather Speedup over Baseline System VE20 x128 VE30 x128 A100 40GB x128 A64FX x128 IceLake-SP x128
ISC’23 Impact of selective caching in Himeno benchmark Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 22 250 255 260 265 270 275 280 285 290 Watt Cache all Bypass all Cache p only 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 GFLOP/s per Watt 0 100 200 300 400 500 600 700 800 900 S M L XL GFLOP/s Problem Size Cache all Bypass all Cache p only Size Dimensions S 64x64x128 M 128x128x256 L 256x256x512 XL 512x512x1024 +6.9% w/ selective caching p does not fit in L3C +5.7% w/ selective caching Performance Power (L size) Power Efficiency (L) VE20: 2.14 GFLOP/s/W A100: 2.21 GFLOP/s/W +8.2% -0.6% +6.5% w/ selective caching
ISC’23 Partitioning mode • Partitioning mode splits a single VE into two NUMA nodes • Each NUMA node has half the cores, LLC and HBM (capacity and B/W are both halved) • Alleviates congestion in the NoC and increases total effective LLC B/W • Cache-intensive applications will benefit from partitioning mode Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 23 Core Core Core Core Core Core LLC LLC Core Core Core Core Core Core Core Core Core Core HBM2E HBM2E HBM2E HBM2E HBM2E HBM2E 0 100 200 300 400 500 600 700 800 900 VE20 VE30 GFLOP/s w/o Partitionig Mode w/ Partitionig Mode +7.1% w/ partitioning mode NUMA node #0 NUMA node #1 Himeno benchmark
ISC’23 Summary • VE30 attains massive speedup in memory-intensive standard benchmarks. • Speedup exceeds improvement in peak compute and memory performance, indicating the benefits of the newly introduced L3 cache and improved LLC. • VE30 outperforms other processors in many real-world workloads including SPEChpc without any source code modification. • New architectural features could be used to further improve performance. Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 24 VE30 achieves high sustained performance equal to or greater than latest GPUs and CPUs, while allowing programmers to use conventional programming models.
ISC’23 Real-world kernel with indexed vector accumulation • A kernel extracted from a real-world CFD application (4 out of 256 indices overlap, two pairs of identical indices) • Using vlfa reduces the runtime from 175.6s to 12.0s (14.6x speedup) Performance Evaluation of a Next-Generation SX-Aurora TSUBASA Vector Supercomputer 28 DO N = nstart,nend IF(flag3(N)==1) THEN COF(7,WI(N),WJ(N),WK(N))=COF(7,WI(N),WJ(N),WK(N))+W_TAUWC(N) * W_AREA_1(N) SOC(WI(N),WJ(N),WK(N))=SOC(WI(N),WJ(N),WK(N))+W_TAUWS(N) * W_AREA_1(N) ENDIF ENDDO
keichi
hayataka88
saltcooky12
ynakano
genomethica
kamakiri1225
s1ok69oo
miyayou
joisino
icoxfog417
tiffany_04192
norihirosuzuki
ykamit
holman
danielanewman
lara
quramy
maltzj
destraynor
hursman
hannesfritz
eileencodes
jensimmons
aarron