Fast Distributed Total Variation

NVIDIA-CDS meeting, LAL, Orsay, March 2015.

Transcript

1. Fast Distributed Total Variation Samuel Vaiter March 30, 2015
2. Total Variation ৙ > ʇ৘1 , ৗ Linear inverse problems ʇ Total Variation regularization ৘Գ ѵ bshnjo ৘ѵϓ঵ 2 3 }}৙ ѿ ʇ৘}}3 3 , ౠ঱)৘* ৘2-2 )౯Ј ৘*৉-৊ > }৘৉-৊ ѿ ৘৉,2-৊ } )౯Ї ৘*৉-৊ > }৘৉-৊ ѿ ৘৉-৊,2 } ঱)৘* > }}Ѵ৘}}2-3 > า ৉-৊ ำ)౯Ј ৘*3 ৉-৊ , )౯Ї ৘*3 ৉-৊ [Rudin-Osher-Fatemi 1992] This talk : How to distribute this minimization ?
3. Proximal Method ৘)ো,2* > qspy ౠ঱ )৘)ো* ѿ ౘѴভ)৘** ৘Գ ѵ bshnjo ৘ѵϓ঵ 2 3 }}৙ ѿ ʇ৘}}3 3 , ౠ঱)৘* Forward-backward ভ)৘* { qspy ౠ঱ )৘* > bshnjo ৚ѵϓ঵ 2 3 }}৘ ѿ ৚}}3 3 , ౠ঱)৚* Proximity operator no closed formula for TV Row/Column splitting [Condat 2013] Augmented method [Chambolle-Pock 2011]
4. A Splitting Scheme ঱)৘* > ঱ even )৘ even * , ঱ odd )৘ odd * nby ৘ even -৘ odd ѿ ঱ even ҄)৘ even * ѿ ঱ odd ҄)৘ odd * ѿ 2 3ౠ }}৘ even , ৘ odd }}3 , ܕ৘ even , ৘ odd - ৙ܖ Primal-dual problem fully decomposable on each square qspy ౠ঱ )৘* > bshnjo ৚ѵϓ঵ 2 3 }}৘ ѿ ৚}}3 3 , ౠ঱)৚* alternate minimization
5. Minimization Over a Square ౣԳ ѵ bshnjo } }ౣ} }3 ӑ2 }} ejw ౣ , Ȣ ৙}}3 3 ܕѴ৘- ౣܖ > ѿܕ৘- ejw ౣܖ ౣ Explicit descent Newton minimization ৘Գ > ejw ౣԳ Empirical observation: one Newton step is enough Dual problem + reprojection on the unit ball ѵ ϓ5
6. Algorithm While Minimize over every even square Minimize over every odd square Over-relaxation (FISTA-like acceleration) easy distributed calculus Compute dual gap dual gap big enough reduction step Implementation in CUDA (2D and 3D) (not open-source yet)
7. Preliminary Benchmark @Xeon E5-2670 / Tesla K20m (linux 3.12, CUDA 5.5) (ms) 2562 5122 10242 20482 40962 81922 1.0 0.4 0.5 0.9 2.3 7.9 79 5.0 1.8 2.6 4.3 13 63 379 10.0 3.9 5.7 11 39 170 892 20.0 9.3 12 24 92 406 1961 dimension of the problem parameter 2D 3D 181 x 217 x 181 35 ms ~100-1000x faster than CPU CP
8. Thanks for your attention Any question ?