Slide 26
Slide 26 text
Ti ps 2 -2: Utili z e G P G P U as M uc h as P ossi bl e
H P C e n vir o n m e nt ( e. g., A B CI) us u all y h as m or e G P Us ( > 5 k) t h a n C P Us ( 2. 4 k).
• Y o u c o ul d s p e e d u p y o ur pr o gr a m m or e t h a n 1 0 ti m es f ast er b y j ust r e writi n g f or G P Us.
• Y o u c a n writ e al m ost t h e s a m e w a y as N u m P y b y usi n g C u P y
f r o m o p t _ e i n s u m i m p o r t c o n t r a c t
x = n p . r a n d o m . r a n d o m ( [ 1 0 0 0 0 , 5 0 ] ) # [ N , D ]
z = n p . r a n d o m . r a n d o m ( [ 1 0 0 0 0 , 1 0 ] ) # [ N , K ]
z / = z . s u m ( a x i s = - 1 , k e e p d i m s = T r u e )
e I = 1 e - 8 * n p . e y e ( x . s h a p e [ 1 ] )
f o r i i i n r a n g e ( 1 0 ) :
z s u m = z . s u m ( a x i s = 0 ) . c l i p ( 1 e - 8 )
p i = z s u m / z . s h a p e [ 0 ] # [ K ]
m u = c o n t r a c t ( " n k , n d - > k d " , z , x ) / z s u m [ : , N o n e ] # [ K , D ]
x m u = x [ : , N o n e ] - m u # [ N , K , D ]
V = c o n t r a c t ( " n k , n k d , n k e - > k d e " , z , x m u , x m u ) / z s u m [ : , N o n e , N o n e ] + e I
V i , ( _ , l d V ) = n p . l i n a l g . i n v ( V ) , n p . l i n a l g . s l o g d e t ( V )
l o g z = n p . l o g ( p i ) - 0 . 5 * ( l d V + c o n t r a c t ( " n k d , k d e , n k e - > n k " , x m u , V i , x m u ) )
z = n p . e x p ( l o g z - l o g s u m e x p ( l o g z , a x i s = - 1 , k e e p d i m s = T r u e ) )
f r o m o p t _ e i n s u m i m p o r t c o n t r a c t
x = c p . r a n d o m . r a n d o m ( [ 1 0 0 0 0 , 5 0 ] ) # [ N , D ]
z = c p . r a n d o m . r a n d o m ( [ 1 0 0 0 0 , 1 0 ] ) # [ N , K ]
z / = z . s u m ( a x i s = - 1 , k e e p d i m s = T r u e )
e I = 1 e - 8 * c p . e y e ( x . s h a p e [ 1 ] )
f o r i i i n r a n g e ( 1 0 ) :
z s u m = z . s u m ( a x i s = 0 ) . c l i p ( 1 e - 8 )
p i = z s u m / z . s h a p e [ 0 ] # [ K ]
m u = c o n t r a c t ( " n k , n d - > k d " , z , x ) / z s u m [ : , N o n e ] # [ K , D ]
x m u = x [ : , N o n e ] - m u # [ N , K , D ]
V = c o n t r a c t ( " n k , n k d , n k e - > k d e " , z , x m u , x m u ) / z s u m [ : , N o n e , N o n e ] + e I
V i , ( _ , l d V ) = c p . l i n a l g . i n v ( V ) , c p . l i n a l g . s l o g d e t ( V )
l o g z = c p . l o g ( p i ) - 0 . 5 * ( l d V + c o n t r a c t ( " n k d , k d e , n k e - > n k " , x m u , V i , x m u ) )
z = c p . e x p ( l o g z - l o g s u m e x p ( l o g z , a x i s = - 1 , k e e p d i m s = T r u e ) )
E M -G M M writt e n wit h N u m P y 9, 9 5 0 ms E M -G M M writt e n wit h C u P y 5 0 4 ms !!
※ M e as ur e d o n a V1 0 0 n o d e (rt _ F =1) of A B CI @ 2 0 2 2/ 0 4/ 0 6
C o n d ucti n g AI R es e ar c h o n Hi g h -P erf or m a n c e C o m p uti n g ( H P C) S yst e ms / 5 0
2 6