Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Natural Policy Gradient and TRPO by Andrew Ga...

Avatar for Tokyo Machine Learning Society Tokyo Machine Learning Society
October 28, 2016
870
1

A Natural Policy Gradient and TRPO by Andrew Gambardella - RRS #1

Avatar for Tokyo Machine Learning Society

Tokyo Machine Learning Society

October 28, 2016

More Decks by Tokyo Machine Learning Society

See All by Tokyo Machine Learning Society
Natural Gradient Learning by Shun'ichi Amari, RIKEN - RRS #1
Avatar for Tokyo Machine Learning Society tmls
1
440
Generative Adversarial Networks by Hasanov Vagif - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
3
540
Character-Level Machine Translation with Bi-scale RNNs by Joe Bullard - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
2
420
NVIDIA Deep Learning Platform and latest information by Takeshi Izaki, Nvidia - TMLS #4　
Avatar for Tokyo Machine Learning Society tmls
1
360
Neural Translation by David and Stefano - TMLS#3
Avatar for Tokyo Machine Learning Society tmls
3
340
Extending Cloud Vision API by Ray Sakai -TMLS#3
Avatar for Tokyo Machine Learning Society tmls
2
200
Bayesian Optimization with TensorFlow/Keras by Keisuke Kamataki - TMLS #2
Avatar for Tokyo Machine Learning Society tmls
1
9.3k

Featured

See All Featured
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
275
41k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
HDC tutorial
Avatar for Michiel Stock michielstock
2
790
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
183
10k
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
31
10k
Measuring & Analyzing Core Web Vitals
Avatar for Philip Tellis bluesmoon
9
950
First, design no harm
Avatar for Per Axbom axbom
PRO
2
1.2k
AI Search: Implications for SEO and How to Move Forward - #ShenzhenSEOConference
Avatar for Aleyda Solis aleyda
1
1.3k
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
Avatar for Liv Day livdayseo
0
170
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.9k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.7k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
360

Transcript

  1. "/BUVSBM1PMJDZ(SBEJFOU
 
 5SVTU3FHJPO1PMJDZ0QUJNJ[BUJPO "OESFX(BNCBSEFMMB
 3FBDUJWF3FTFBSDI4FNJOBS

  2. 3FJOGPSDFNFOU-FBSOJOH

  3. "QQSPBDIFT ↟ 7BMVFCBTFE ↟ 1PMJDZCBTFE ↟ -FBSOBQPMJDZ BDUPS BOETUBUFWBMVFGVODUJPO DSJUJD

     KPJOUMZ ↟ 0UIFST

  4. %FFQ3FJOGPSDFNFOU-FBSOJOH

  5. #VUXIZMFBSOB2GVODUJPO ↟ *ONBDIJOFMFBSOJOH XFPQUJNJ[FGPSUIFUIJOHXFDBSF BCPVU ↟ *OUIJTDBTF JUsTUIFFYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ $BOFNQJSJDBMMZTIPXUIBU%2/JTOPUWFSZHPPE

    DPNQBSFEUPQPMJDZHSBEJFOUBOEBDUPSDSJUJDNFUIPET TFF"TZODISPOPVT.FUIPETGPS%FFQ3FJOGPSDFNFOU -FBSOJOH

  6. 1PMJDZ(SBEJFOU

  7. 3&*/'03$&FTUJNBUPS
 BMTPLOPXOBTMJLFMJIPPESBUJPFTUJNBUPSTDPSF GVODUJPOFTUJNBUPS -FUYCFBSBOEPNWBSJBCMFXJUIY_Q ↟Ƕ 5IFO

  8. .BOZEJGGFSFOUDIPJDFTGPSG

  9. 1SPTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ 0QUJNJ[JOHGPSXIBUXFBDUVBMMZXBOUnNBYJNVN FYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ #FUUFSGPSIJHIEJNFOTJPOBMUBTLT ↟ #FUUFSGPSDPOUJOVPVTUBTLT NBQZPVSTUBUFUPUIFNFBO

    BOEWBSJBODFPGB(BVTTJBOEJTUSJCVUJPO  ↟ $BOCPPUTUSBQPGGPGQPMJDJFTDSFBUFECZJNJUBUJPOMFBSOJOH ↟ $BOMFBSOBTUPDIBTUJDQPMJDZ HPPEJGTPNFTUBUFTBSF BMJBTFE

  10. $POTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ %2/NPSFTBNQMFFGGJDJFOUPODFSUBJOUBTLT

  11. $POTPGQPMJDZHSBEJFOU NFUIPETJOHFOFSBM ↟ 'BJSMZEJGGJDVMUUPEFCVH ↟ &TUJNBUPSPGUIFHSBEJFOUIBTIJHIWBSJBODF ↟ $POWFSHFUPBMPDBMNJOJNVN DBOsUUSBJOCJHOFVSBMOFUT CFDBVTFPGQSFWJPVTQPJOUBOZHVBSBOUFFTBCPVUMPDBM

    NJOJNBPGFYUSFNFMZIJHIEJNFOTJPOBMGVODUJPOTEPOPU IPME

  12. )PXUPEFBMXJUIFTUJNBUPS WBSJBODF ↟ $FSUBJOUSJDLTDBOIFMQVT 4FF3FJOGPSDFNFOU-FBSOJOH /FVSBM5VSJOH.BDIJOFT PSTFF-FBSOJOH0OMJOF "MJHONFOUTXJUI$POUJOVPVT3FXBSET1PMJDZ(SBEJFOU  ↟

    8FDBOBMTPEPHSBEJFOUEFTDFOUVTJOHBEJSFDUJPOBOE PSTUFQTJ[FUIBUNBLFTNPSFTFOTFGPS3-UIBOWBOJMMB 4(%

  13. /BUVSBM1PMJDZ(SBEJFOU ↟ 6TF"NBSJsTUIFPSZUPEFGJOFBNFUSJDCBTFEOPUPOUIFDIPJDF PGDPPSEJOBUFT CVUPOUIFNBOJGPMEUIBUUIFTFDPPSEJOBUFT QBSBNFUFSJ[F ↟ "TTIPXOCZ"NBSJ UIF'JTIFSJOGPSNBUJPONBUSJYJTBO JOWBSJBOUNFUSJDPOUIFTQBDFPGQBSBNFUFSTPGQSPCBCJMJUZ

    EJTUSJCVUJPOT*UEFGJOFTUIFTBNFEJTUBODFCFUXFFOUXPQPJOUT SFHBSEMFTTPGUIFDIPJDFPGDPPSEJOBUFT JFUIF QBSBNFUFSJ[BUJPO  ↟ 5IJTHJWFTTUFFQFTUEFTDFOUEJSFDUJPO(− Ƕ ˜ǵ Ƕ XIFSF˜ǵ Ƕ  JTUIFEFTDFOUEJSFDUJPOHJWFOCZUIFtWBOJMMBuQPMJDZHSBEJFOU

  14. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ 8FXBOUUIFTUBUFTXFTFFXJUIPVSOFXQPMJDZUPCF TJNJMBSUPUIPTFXFTBXXJUIPVSPMEQPMJDZ TPXFDBO SFVTFLOPXMFEHFGSPNQBTUSPMMPVUT ↟ #ZDPOTUSBJOJOHPVSOFXQPMJDZUPCFtDMPTFuUPPVSPME QPMJDZJOUIJTXBZ

    XFDBODPOUSPMUIFBDUJPOEJTUSJCVUJPO DIBOHFBTBQSPYZ

  15. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ *GXFUBLFBTNBMMTUFQJO&VDMJEFBOQBSBNFUFSTQBDF  FWFOBTNBMMTUFQDPVMEDBVTFXJMEDIBOHFTJOUIF BDUJPOEJTUSJCVUJPO*UJTUIFSFGPSFFTTFOUJBMUIBUXFUBLF TUFQTJOUIFOBUVSBMHSBEJFOUEJSFDUJPO ↟ "MTP

    IPXDBOXFDIPPTFBOBQQSPQSJBUFTUFQTJ[F  XIFOUIFTUBUJTUJDTPGPVSEBUBBSFDPOTUBOUMZDIBOHJOH  BTPVSQPMJDZDIBOHFT TPEPFTPVSEBUB
  16. None

  17. "TMJHIUJTTVF ↟ 5IFOBUVSBMQPMJDZHSBEJFOUNVTUGPSNBOEJOWFSUUIF FNQJSJDBM'*. ↟ 5IJTJTUPPFYQFOTJWF

  18. 5SVODBUFE/BUVSBM1PMJDZ (SBEJFOU 5/1( ↟ *OTUFBEPGDPNQVUJOHUIF'*. DPNQVUFUIFTBNF EJSFDUJPOPGOBUVSBMQPMJDZHSBEJFOUVTJOHBDPOKVHBUF HSBEJFOUBMHPSJUINUIBUPOMZSFRVJSFTDPNQVUJOHBO BQQSPYJNBUJPOPG( Ƕ

    WGPSBSCJUSBSZWFDUPSW

  19. 5SVTU3FHJPO1PMJDZ 0QUJNJ[BUJPO 5310 ↟ $POTUSBJOUIF,-EJWFSHFODFCFUXFFOPVSPMEQPMJDZ BOEPVSOFXQPMJDZUPCFXJUIJOTPNFUSVTUSFHJPO SBEJVTDz ↟ #PVOEPO,-EJWFSHFODFJTBMTPBCPVOEPOIPXNVDI PVSQFSGPSNBODFDBOEFHSBEFXJUIJOUIJTPOFTUFQ

  20. 5/1(BOE5310 ↟ 5/1(DBOCFPCUBJOFEBTBTQFDJBMDBTFPG5310XJUIB MJOFBSBQQSPYJNBUJPOUP-BOEBRVBESBUJDBQQSPYJNBUJPO UPUIF%,- DPOTUSBJOU/PUJDFTUFQTJ[FǹJO5/1(
 
 
 
 


    
 
 
 5310 5/1(XBQQSPYJNBUJPO
  21. None

  22. "OEOPXGPSTPNFUIJOH DPNQMFUFMZEJGGFSFOU

  23. (FOFSBUJWF"EWFSTBSJBM /FUXPSLT ↟ "SFDFOUNFUIPEGPSVOTVQFSWJTFEMFBSOJOH ↟ (FOFSBUPS(TBNQMFTGSPNBEJTUSJCVUJPOBOEEJTDSJNJOBUPS% DMBTTJGJFTTBNQMFTBTSFBMPSGBMTF ↟ (JTUSZJOHUPtUSJDLu%CZJODSFBTJOHUIFQSPCBCJMJUZUIBU%XJMM DMBTTJGZ(sTTBNQMFTBTSFBM%JTUSZJOHUPNJOJNJ[FUIF

    QSPCBCJMJUZPGCFJOHUSJDLFE ↟ 5IJTHJWFTVTUIFBEWFSTBSJBMMPTTGVODUJPO
 
 


  24. 8IBUEPFTUIJTIBWFUP EPXJUI3-

  25. *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHTFUUJOH(JWFOUSBKFDUPSJFTGSPNBO FYQFSU MFBSOBQPMJDZ ↟ 3FHVMBSTVQFSWJTFEMFBSOJOH CFIBWJPSBMDMPOJOH XJMM BMXBZTXPSLJOUIFJOGJOJUFEBUBSFHJNF

    ↟ $BOXFNBLFTPNFUIJOHNPSFTBNQMFFGGJDJFOU  LOPXJOHUIBUXFBSFMPPLJOHGPSBO.%1QPMJDZ

  26. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHDBOCFGPSNVMBUFEBTBNJOJNBYQSPCMFN XIFSFUIFBJNJTUPMFBSOUIFDPTUGVODUJPOUIBUNJOJNJ[FT UIFBNPVOUCZXIJDIUIFPQUJNBMQPMJDZPVUQFSGPSNTUIF FYQFSUQPMJDZ ↟ #VUMFBSOJOHBDPTUGVODUJPOJTFRVJWBMFOUUPNJOJNJ[JOHUIF SFHVMBSJ[FEEJTUBODFCFUXFFOTUBUFBDUJPOPDDVQBODZ

    EJTUSJCVUJPOTGPSUIFFYQFSUQPMJDZBOEMFBSOFEQPMJDZ ↟ #ZVTJOHBEFFQOFVSBMOFUXPSLXJUIUIFDSPTTFOUSPQZMPTT BTUIFEJTUBODFCFUXFFOPDDVQBODZEJTUSJCVUJPOT XFDBO GPSHPMFBSOJOHBDPTUGVODUJPODPNQMFUFMZ

  27. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH

  28. None

  29. 3FTPVSDFT ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH1POHGSPN1JYFMT "OESFK ,BSQBUIZsTCMPH ↟ -FDUVSF1PMJDZ(SBEJFOU %BWJE4JMWFST3-$PVSTF ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH"5VUPSJBM

    0QFO"* ↟ 1PMJDZ(SBEJFOU.FUIPET 4DIPMBSQFEJB

  30. 3FGFSFODFT ↟ "NBSJ 4/BUVSBMHSBEJFOUXPSLTFGGJDJFOUMZJOMFBSOJOH/FVSBM$PNQVUBUJPO      ↟

    %VBO: $IFO9 )PVUIPPGU3 4DIVMNBO+ BOE"CCFFM1#FODINBSLJOHEFFQ SFJOGPSDFNFOUMFBSOJOHGPSDPOUJOVPVTDPOUSPMBS9JWQSFQSJOUBS9JW  ↟ )P + BOE&SNPO 4(FOFSBUJWFBEWFSTBSJBMJNJUBUJPOMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ ,BLBEF 4"OBUVSBMQPMJDZHSBEJFOU*O"EWBODFTJO/FVSBM*OGPSNBUJPO1SPDFTTJOH 4ZTUFNT QQm.*51SFTT  ↟ -VP: $IJV$ +BJUMZ. BOE4VUTLFWFS*-FBSOJOHPOMJOFBMJHONFOUTXJUIDPOUJOVPVT SFXBSETQPMJDZHSBEJFOUBS9JWQSFQSJOUBS9JW  ↟ .OJI7 #BEJB" .JS[B. (SBWFT" -JMMJDSBQ5 )BSMFZ5 4JMWFS% BOE,BWVLDVPHMV, "TZODISPOPVTNFUIPETGPSEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW  

  31. 3FGFSFODFT ↟ .OJI 7 ,BWVLDVPHMV , 4JMWFS % (SBWFT "

    "OUPOPHMPV * 8JFSTUSB % BOE 3JFENJMMFS .1MBZJOH"UBSJXJUIEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ 1GBV % BOE7JOZBMT 0$POOFDUJOHHFOFSBUJWFBEWFSTBSJBMOFUXPSLTBOEBDUPS DSJUJDNFUIPETBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + -FWJOF 4 .PSJU[ 1 +PSEBO . BOE"CCFFM 15SVTUSFHJPOQPMJDZ PQUJNJ[BUJPOBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + .PSJU[ 1 -FWJOF 4 +PSEBO . BOE"CCFFM 1)JHIEJNFOTJPOBM DPOUJOVPVTDPOUSPMVTJOHHFOFSBMJ[FEBEWBOUBHFFTUJNBUJPOBS9JWQSFQSJOUBS9JW   ↟ ;BSFNCB 8 BOE4VUTLFWFS *3FJOGPSDFNFOUMFBSOJOHOFVSBMUVSJOHNBDIJOFT BS9JWQSFQSJOUBS9JW 

  32. 2VFTUJPOT