Matrix Multiplication

Parallel Computing in Shared Memory using OpenMP - Matrix Multiplication problem.

Transcript

1. Matrix Multiplication Parallel Computing in Shared Memory using OpenMP Gabriel Moro - KNOWLEDGE TRANSFER - KT, Porto Alegre - November 2018
2. Matrix Multiplication A B C
3. Matrix Multiplication A B C
4. Matrix Multiplication A B C
5. Matrix Multiplication A B C
6. Matrix Multiplication A B C
7. Matrix Multiplication A B C
8. Matrix Multiplication A B C
9. Matrix Multiplication A B C
10. Matrix Multiplication A B C
11. Ways to improve the performance to this algorithm - Algorithm complexity - Parallelism
12. Ways to improve the performance to this algorithm - Algorithm complexity - Parallelism
13. Ways to improve the performance to this algorithm - Algorithm complexity - Parallelism - Shared Memory - Distributed Memory
14. Ways to improve the performance to this algorithm - Algorithm complexity - Parallelism - Shared Memory - Distributed Memory
15. Parallel OpenMP Model A C T1 T2 T3
16. Parallel OpenMP Model A C T1 T2 T3
17. Turing - Processor - 4 x Intel Xeon X7550 Nehalem - 32 physical cores - HyperThreading - Memory - 128GB DDR3 - GPPD-UFRGS
18. Version: normal_seq for(i=0;i < size; i++) { for(j=0;j < size; j++) { tmp=0; for(k=0; k < size; k++) tmp = tmp + A[i][k] * B[k][j]; C[i][j] = tmp; } }
19. Version: normal_par #pragma omp parallel for private(i,j,k,tmp) for(i=0;i < size; i++) { for(j=0;j < size; j++) { tmp=0; for(k=0; k < size; k++) tmp = tmp + A[i][k] * B[k][j]; C[i][j] = tmp; } }
20. Version: continuos_seq for(i=0;i < size; i++) { for(j=0;j < size; j++) { tmp=0; for(k=0; k < size; k++) tmp = tmp + A[i * size + k] * B[k * size + j]; C[i * size + j] = tmp; } }
21. Version: continuos_par #pragma omp parallel for private(i,j,k,tmp) for(i=0;i < size; i++) { for(j=0;j < size; j++) { tmp=0; for(k=0; k < size; k++) tmp = tmp + A[i * size + k] * B[k * size + j]; C[i * size + j] = tmp; } }
22. Version: tiling_seq register int jj,kk,i,j,k; double tmp=0; for(jj=0;jj < size; jj=jj+block) { for(kk=0; kk < size; kk=kk+block) { for(i=0; i < size; i++) { for(j=jj; j < min(jj+block, size); j++) { tmp=0; for(k=kk; k < min(kk+block,size); k++) { tmp = tmp + A[i][k] * B[k][j]; } R[i][j] = tmp; } } } }
23. Version: tiling_par register int jj,kk,i,j,k; double tmp=0; for(jj=0;jj < size; jj=jj+block) { for(kk=0; kk < size; kk=kk+block) { #pragma omp parallel for private(i,j,k,tmp) schedule(static) for(i=0; i < size; i++) { for(j=jj; j < min(jj+block, size); j++) { tmp=0; for(k=kk; k < min(kk+block,size); k++) { tmp = tmp + A[i][k] * B[k][j]; } R[i][j] = tmp; } } } }
24. Links - Top 500: https://www.top500.org/lists/2018/11/ - Green 500: https://www.top500.org/green500/lists/2018/11/ - NAS Parallel Benchmark: https://www.nas.nasa.gov/publications/npb.html
25. Thanks! https://github.com/tido4410/knowledge-transfer-gbmoro.git Gabriel Moro - Matrix Multiplication - OpenMP - KT, Porto Alegre - November 2018