Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

A Natural Policy Gradient and TRPO by Andrew Ga...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Tokyo Machine Learning Society Tokyo Machine Learning Society
October 28, 2016
880
1

A Natural Policy Gradient and TRPO by Andrew Gambardella - RRS #1

Avatar for Tokyo Machine Learning Society

Tokyo Machine Learning Society

October 28, 2016

More Decks by Tokyo Machine Learning Society

See All by Tokyo Machine Learning Society
Natural Gradient Learning by Shun'ichi Amari, RIKEN - RRS #1
Avatar for Tokyo Machine Learning Society tmls
1
460
Generative Adversarial Networks by Hasanov Vagif - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
3
550
Character-Level Machine Translation with Bi-scale RNNs by Joe Bullard - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
2
440
NVIDIA Deep Learning Platform and latest information by Takeshi Izaki, Nvidia - TMLS #4　
Avatar for Tokyo Machine Learning Society tmls
1
380
Neural Translation by David and Stefano - TMLS#3
Avatar for Tokyo Machine Learning Society tmls
3
360
Extending Cloud Vision API by Ray Sakai -TMLS#3
Avatar for Tokyo Machine Learning Society tmls
2
220
Bayesian Optimization with TensorFlow/Keras by Keisuke Kamataki - TMLS #2
Avatar for Tokyo Machine Learning Society tmls
1
9.3k

Featured

See All Featured
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
57
14k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
280
Public Speaking Without Barfing On Your Shoes - THAT 2023
Avatar for David Neal reverentgeek
1
600
Paper Plane
Avatar for Katie Cordina Lindsey katiecoart
PRO
5
54k
Digital Ethics as a Driver of Design Innovation
Avatar for Per Axbom axbom
PRO
1
460
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.3k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
2
370
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
12k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
10k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
Avatar for Vitaly Friedman smashingmag
254
22k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
560
Efficient Content Optimization with Google Search Console & Apps Script
Avatar for Katarina Dahlin katarinadahlin
PRO
1
930

Transcript

  1. "/BUVSBM1PMJDZ(SBEJFOU
 
 5SVTU3FHJPO1PMJDZ0QUJNJ[BUJPO "OESFX(BNCBSEFMMB
 3FBDUJWF3FTFBSDI4FNJOBS

  2. 3FJOGPSDFNFOU-FBSOJOH

  3. "QQSPBDIFT ↟ 7BMVFCBTFE ↟ 1PMJDZCBTFE ↟ -FBSOBQPMJDZ BDUPS BOETUBUFWBMVFGVODUJPO DSJUJD

     KPJOUMZ ↟ 0UIFST

  4. %FFQ3FJOGPSDFNFOU-FBSOJOH

  5. #VUXIZMFBSOB2GVODUJPO ↟ *ONBDIJOFMFBSOJOH XFPQUJNJ[FGPSUIFUIJOHXFDBSF BCPVU ↟ *OUIJTDBTF JUsTUIFFYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ $BOFNQJSJDBMMZTIPXUIBU%2/JTOPUWFSZHPPE

    DPNQBSFEUPQPMJDZHSBEJFOUBOEBDUPSDSJUJDNFUIPET TFF"TZODISPOPVT.FUIPETGPS%FFQ3FJOGPSDFNFOU -FBSOJOH

  6. 1PMJDZ(SBEJFOU

  7. 3&*/'03$&FTUJNBUPS
 BMTPLOPXOBTMJLFMJIPPESBUJPFTUJNBUPSTDPSF GVODUJPOFTUJNBUPS -FUYCFBSBOEPNWBSJBCMFXJUIY_Q ↟Ƕ 5IFO

  8. .BOZEJGGFSFOUDIPJDFTGPSG

  9. 1SPTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ 0QUJNJ[JOHGPSXIBUXFBDUVBMMZXBOUnNBYJNVN FYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ #FUUFSGPSIJHIEJNFOTJPOBMUBTLT ↟ #FUUFSGPSDPOUJOVPVTUBTLT NBQZPVSTUBUFUPUIFNFBO

    BOEWBSJBODFPGB(BVTTJBOEJTUSJCVUJPO  ↟ $BOCPPUTUSBQPGGPGQPMJDJFTDSFBUFECZJNJUBUJPOMFBSOJOH ↟ $BOMFBSOBTUPDIBTUJDQPMJDZ HPPEJGTPNFTUBUFTBSF BMJBTFE

  10. $POTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ %2/NPSFTBNQMFFGGJDJFOUPODFSUBJOUBTLT

  11. $POTPGQPMJDZHSBEJFOU NFUIPETJOHFOFSBM ↟ 'BJSMZEJGGJDVMUUPEFCVH ↟ &TUJNBUPSPGUIFHSBEJFOUIBTIJHIWBSJBODF ↟ $POWFSHFUPBMPDBMNJOJNVN DBOsUUSBJOCJHOFVSBMOFUT CFDBVTFPGQSFWJPVTQPJOUBOZHVBSBOUFFTBCPVUMPDBM

    NJOJNBPGFYUSFNFMZIJHIEJNFOTJPOBMGVODUJPOTEPOPU IPME

  12. )PXUPEFBMXJUIFTUJNBUPS WBSJBODF ↟ $FSUBJOUSJDLTDBOIFMQVT 4FF3FJOGPSDFNFOU-FBSOJOH /FVSBM5VSJOH.BDIJOFT PSTFF-FBSOJOH0OMJOF "MJHONFOUTXJUI$POUJOVPVT3FXBSET1PMJDZ(SBEJFOU  ↟

    8FDBOBMTPEPHSBEJFOUEFTDFOUVTJOHBEJSFDUJPOBOE PSTUFQTJ[FUIBUNBLFTNPSFTFOTFGPS3-UIBOWBOJMMB 4(%

  13. /BUVSBM1PMJDZ(SBEJFOU ↟ 6TF"NBSJsTUIFPSZUPEFGJOFBNFUSJDCBTFEOPUPOUIFDIPJDF PGDPPSEJOBUFT CVUPOUIFNBOJGPMEUIBUUIFTFDPPSEJOBUFT QBSBNFUFSJ[F ↟ "TTIPXOCZ"NBSJ UIF'JTIFSJOGPSNBUJPONBUSJYJTBO JOWBSJBOUNFUSJDPOUIFTQBDFPGQBSBNFUFSTPGQSPCBCJMJUZ

    EJTUSJCVUJPOT*UEFGJOFTUIFTBNFEJTUBODFCFUXFFOUXPQPJOUT SFHBSEMFTTPGUIFDIPJDFPGDPPSEJOBUFT JFUIF QBSBNFUFSJ[BUJPO  ↟ 5IJTHJWFTTUFFQFTUEFTDFOUEJSFDUJPO(− Ƕ ˜ǵ Ƕ XIFSF˜ǵ Ƕ  JTUIFEFTDFOUEJSFDUJPOHJWFOCZUIFtWBOJMMBuQPMJDZHSBEJFOU

  14. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ 8FXBOUUIFTUBUFTXFTFFXJUIPVSOFXQPMJDZUPCF TJNJMBSUPUIPTFXFTBXXJUIPVSPMEQPMJDZ TPXFDBO SFVTFLOPXMFEHFGSPNQBTUSPMMPVUT ↟ #ZDPOTUSBJOJOHPVSOFXQPMJDZUPCFtDMPTFuUPPVSPME QPMJDZJOUIJTXBZ

    XFDBODPOUSPMUIFBDUJPOEJTUSJCVUJPO DIBOHFBTBQSPYZ

  15. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ *GXFUBLFBTNBMMTUFQJO&VDMJEFBOQBSBNFUFSTQBDF  FWFOBTNBMMTUFQDPVMEDBVTFXJMEDIBOHFTJOUIF BDUJPOEJTUSJCVUJPO*UJTUIFSFGPSFFTTFOUJBMUIBUXFUBLF TUFQTJOUIFOBUVSBMHSBEJFOUEJSFDUJPO ↟ "MTP

    IPXDBOXFDIPPTFBOBQQSPQSJBUFTUFQTJ[F  XIFOUIFTUBUJTUJDTPGPVSEBUBBSFDPOTUBOUMZDIBOHJOH  BTPVSQPMJDZDIBOHFT TPEPFTPVSEBUB
  16. None

  17. "TMJHIUJTTVF ↟ 5IFOBUVSBMQPMJDZHSBEJFOUNVTUGPSNBOEJOWFSUUIF FNQJSJDBM'*. ↟ 5IJTJTUPPFYQFOTJWF

  18. 5SVODBUFE/BUVSBM1PMJDZ (SBEJFOU 5/1( ↟ *OTUFBEPGDPNQVUJOHUIF'*. DPNQVUFUIFTBNF EJSFDUJPOPGOBUVSBMQPMJDZHSBEJFOUVTJOHBDPOKVHBUF HSBEJFOUBMHPSJUINUIBUPOMZSFRVJSFTDPNQVUJOHBO BQQSPYJNBUJPOPG( Ƕ

    WGPSBSCJUSBSZWFDUPSW

  19. 5SVTU3FHJPO1PMJDZ 0QUJNJ[BUJPO 5310 ↟ $POTUSBJOUIF,-EJWFSHFODFCFUXFFOPVSPMEQPMJDZ BOEPVSOFXQPMJDZUPCFXJUIJOTPNFUSVTUSFHJPO SBEJVTDz ↟ #PVOEPO,-EJWFSHFODFJTBMTPBCPVOEPOIPXNVDI PVSQFSGPSNBODFDBOEFHSBEFXJUIJOUIJTPOFTUFQ

  20. 5/1(BOE5310 ↟ 5/1(DBOCFPCUBJOFEBTBTQFDJBMDBTFPG5310XJUIB MJOFBSBQQSPYJNBUJPOUP-BOEBRVBESBUJDBQQSPYJNBUJPO UPUIF%,- DPOTUSBJOU/PUJDFTUFQTJ[FǹJO5/1(
 
 
 
 


    
 
 
 5310 5/1(XBQQSPYJNBUJPO
  21. None

  22. "OEOPXGPSTPNFUIJOH DPNQMFUFMZEJGGFSFOU

  23. (FOFSBUJWF"EWFSTBSJBM /FUXPSLT ↟ "SFDFOUNFUIPEGPSVOTVQFSWJTFEMFBSOJOH ↟ (FOFSBUPS(TBNQMFTGSPNBEJTUSJCVUJPOBOEEJTDSJNJOBUPS% DMBTTJGJFTTBNQMFTBTSFBMPSGBMTF ↟ (JTUSZJOHUPtUSJDLu%CZJODSFBTJOHUIFQSPCBCJMJUZUIBU%XJMM DMBTTJGZ(sTTBNQMFTBTSFBM%JTUSZJOHUPNJOJNJ[FUIF

    QSPCBCJMJUZPGCFJOHUSJDLFE ↟ 5IJTHJWFTVTUIFBEWFSTBSJBMMPTTGVODUJPO
 
 


  24. 8IBUEPFTUIJTIBWFUP EPXJUI3-

  25. *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHTFUUJOH(JWFOUSBKFDUPSJFTGSPNBO FYQFSU MFBSOBQPMJDZ ↟ 3FHVMBSTVQFSWJTFEMFBSOJOH CFIBWJPSBMDMPOJOH XJMM BMXBZTXPSLJOUIFJOGJOJUFEBUBSFHJNF

    ↟ $BOXFNBLFTPNFUIJOHNPSFTBNQMFFGGJDJFOU  LOPXJOHUIBUXFBSFMPPLJOHGPSBO.%1QPMJDZ

  26. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHDBOCFGPSNVMBUFEBTBNJOJNBYQSPCMFN XIFSFUIFBJNJTUPMFBSOUIFDPTUGVODUJPOUIBUNJOJNJ[FT UIFBNPVOUCZXIJDIUIFPQUJNBMQPMJDZPVUQFSGPSNTUIF FYQFSUQPMJDZ ↟ #VUMFBSOJOHBDPTUGVODUJPOJTFRVJWBMFOUUPNJOJNJ[JOHUIF SFHVMBSJ[FEEJTUBODFCFUXFFOTUBUFBDUJPOPDDVQBODZ

    EJTUSJCVUJPOTGPSUIFFYQFSUQPMJDZBOEMFBSOFEQPMJDZ ↟ #ZVTJOHBEFFQOFVSBMOFUXPSLXJUIUIFDSPTTFOUSPQZMPTT BTUIFEJTUBODFCFUXFFOPDDVQBODZEJTUSJCVUJPOT XFDBO GPSHPMFBSOJOHBDPTUGVODUJPODPNQMFUFMZ

  27. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH

  28. None

  29. 3FTPVSDFT ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH1POHGSPN1JYFMT "OESFK ,BSQBUIZsTCMPH ↟ -FDUVSF1PMJDZ(SBEJFOU %BWJE4JMWFST3-$PVSTF ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH"5VUPSJBM

    0QFO"* ↟ 1PMJDZ(SBEJFOU.FUIPET 4DIPMBSQFEJB

  30. 3FGFSFODFT ↟ "NBSJ 4/BUVSBMHSBEJFOUXPSLTFGGJDJFOUMZJOMFBSOJOH/FVSBM$PNQVUBUJPO      ↟

    %VBO: $IFO9 )PVUIPPGU3 4DIVMNBO+ BOE"CCFFM1#FODINBSLJOHEFFQ SFJOGPSDFNFOUMFBSOJOHGPSDPOUJOVPVTDPOUSPMBS9JWQSFQSJOUBS9JW  ↟ )P + BOE&SNPO 4(FOFSBUJWFBEWFSTBSJBMJNJUBUJPOMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ ,BLBEF 4"OBUVSBMQPMJDZHSBEJFOU*O"EWBODFTJO/FVSBM*OGPSNBUJPO1SPDFTTJOH 4ZTUFNT QQm.*51SFTT  ↟ -VP: $IJV$ +BJUMZ. BOE4VUTLFWFS*-FBSOJOHPOMJOFBMJHONFOUTXJUIDPOUJOVPVT SFXBSETQPMJDZHSBEJFOUBS9JWQSFQSJOUBS9JW  ↟ .OJI7 #BEJB" .JS[B. (SBWFT" -JMMJDSBQ5 )BSMFZ5 4JMWFS% BOE,BWVLDVPHMV, "TZODISPOPVTNFUIPETGPSEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW  

  31. 3FGFSFODFT ↟ .OJI 7 ,BWVLDVPHMV , 4JMWFS % (SBWFT "

    "OUPOPHMPV * 8JFSTUSB % BOE 3JFENJMMFS .1MBZJOH"UBSJXJUIEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ 1GBV % BOE7JOZBMT 0$POOFDUJOHHFOFSBUJWFBEWFSTBSJBMOFUXPSLTBOEBDUPS DSJUJDNFUIPETBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + -FWJOF 4 .PSJU[ 1 +PSEBO . BOE"CCFFM 15SVTUSFHJPOQPMJDZ PQUJNJ[BUJPOBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + .PSJU[ 1 -FWJOF 4 +PSEBO . BOE"CCFFM 1)JHIEJNFOTJPOBM DPOUJOVPVTDPOUSPMVTJOHHFOFSBMJ[FEBEWBOUBHFFTUJNBUJPOBS9JWQSFQSJOUBS9JW   ↟ ;BSFNCB 8 BOE4VUTLFWFS *3FJOGPSDFNFOUMFBSOJOHOFVSBMUVSJOHNBDIJOFT BS9JWQSFQSJOUBS9JW 

  32. 2VFTUJPOT