Upgrade to Pro — share decks privately, control downloads, hide ads and more …

深層学習フレームワーク概要とChainerの事例紹介

 深層学習フレームワーク概要とChainerの事例紹介

2017年5月19日 Stapy x 理研AIP オープンソース研究会
https://startpython.connpass.com/event/55579/

Kenta Oono

May 19, 2017
Tweet

More Decks by Kenta Oono

Other Decks in Technology

Transcript

  1. େ໺ ݈ଠ • twitter: @delta2323_ • 経歴 • 数学専攻(修⼠) →

    2011.4 PFI → 2014.10 PFN • 担当分野 • バイオプロジェクト • Chainerコアチーム • インターン・採⽤チーム 2
  2. 5FDIOPMPHZTUBDLPGB%-GSBNFXPSL 6 name functions example Graphical visualization DIGITS, TensorBoard Machine

    learning workflow management Dataset prep, Save/Load Training loop Keras, TF slim Computational graph(CG) management Build/Optimize CGs Forward/Back prop Theano, TensorFlow Torch.nn Multi-dimensional array processing High-level array manipulation NumPy, CuPy Eigen, Torch (core) Numerical computation Matrix operation Convolution BLAS(OpenBLAS, MKL), cuBLAS, cuDNN, MKL DNN Computational device CPU, GPU, TPU, FPGA
  3. üconstructing NNs as a Python programming üdynamic NN construction üCPU/GPU

    agnostic code with CuPy 7 Mission Speed up research and development of deep learning and its applications. Features Flexible and intuitive description of complex NNs by http://chainer.org *NN = Neural Network
  4. f g x f g Static graph construction Data feed

    x y f z g Dynamic graph construction Define-And-Run (Most frameworks) Define-By-Run (Chainer) 9 x y f x z Static Dynamic Optimization ✓ △ Flexibility △ ✓
  5. $POTUSVDU//TBT 1ZUIPOQSPHSBNNJOH class MLP(Link): def __int__(self): super(MLP, self).__init__( l1=Linear(784, 1000),

    l2=Linear(1000, 1000), l3=Linear(1000, 10)) def __call__(x): h1 = F.relu(self.l1(x)) h2 = F.relu(self.l2(l1)) return self.l3(h2) Linear l1 x W bias ReLU Linear l2 h1 W bias ReLU Linear l3 h2 W bias 11
  6. 3FMFBTFIJTUPSZ • 2015/06: v1.0.0 • 2015/09: v1.3.0 (CuPy) • 2015/11:

    v1.5.0 (Link/Chain, CuPy in Cython) • 2016/06: [MinPy] • 2016/07: v1.11.0 (Trainer) • 2017/01: [PyTorch, TensorFlow Fold] • 2017/02: v2.0.0a • 2017/04: v2.0.0b • 2017/05: v1.24.0 (Last v1 release)
  7. $IBJOFS W • First major version up that breaks backward

    compatibility. • Important features (almost fixed) • CuPy separation • Unified configuration (chainer.config, esp. train mode) • train argument is removed from many functions • Variable updated: Parameter class, uninitialized var, volatile removed • Funcion.retain_inputs and retain_outputs to reduce memory usage • New-style parameter/child link registration (just setting them as an attribute) • UpdateRule customized for each parameter • Extention.initialize added, invoke_before_training removed • No duplicated memory between training graph and evaluation graph • Input size is made optional in many links (L.Linear(100)) • wscale option is removed from many links 13 Will release on May 30th 2017
  8. $IBJOFS$7 17 ü Dataset Wrapper For well-known CV datasets (CUB,

    Pascal VOC) ü Dataset transformer (random crop, random flop) ü Implements typical workflow in CV
  9. /VN1ZMJLF"1*BDDFMFSBUFEXJUI$6%" # CPU x_cpu = numpy.array([1, 2, 3]) l2_cpu =

    numpy.linalg.norm(x_cpu) # GPU x_gpu = cupy.array([1, 2, 3]) l2_gpu = cupy.linalg.norm(x_gpu) CuPy will be an independent project from Chainer from Chainer v2. >150 NumPy functions are supported
  10. %FWFMPQNFOUUFBN BTPG.BZ • Chainer, CuPy • Core development team: approx.

    10 members • Reviewer team: approx. 10 members • Chainer user group: approx. 5 members • Chainer RL, Chainer MN, Chainer CV: 2, 3 members for each • Paints Chainer: approx. 10 members 22 * some members overlap
  11. $* 23 Travis CI Run all CPU tests of all

    PRs Jenkins installation test
  12. $PNNVOJUZBDUJWJUJFT 25 Chainer meetup (#0 - #4) #5 will be

    held in June 10th 2017 Deep learning mini course @UCSF application for biology Google group (ja, en), Slack (ja, en), Twitter (ja, en)
  13. $PODMVTJPO • Chainer is a Python-based deep learning framework that

    leverages flexible and intuitive description of NNs. • Many libraries and services are being developed on top of Chainer (ChainerRL/MN/CV, PaintsChainer, PonanzaChainer, CuPy). • Introduced the development and the user-group teams of Chainer 26
  14. 5SZ$IBJOFS  http://chainer.org 27 Seiya Tokui Kenta Oono Yuya Unno

    Ryosuke Okuta Chainer core development team Brian Vogel Gentaro Watanabe Shunta Saito Daisuke Nishino and many contributors ! Contact: [email protected] Google Group: Chainer User Group