Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

A Natural Policy Gradient and TRPO by Andrew Ga...

Avatar for Tokyo Machine Learning Society Tokyo Machine Learning Society
October 28, 2016
880
1

A Natural Policy Gradient and TRPO by Andrew Gambardella - RRS #1

Avatar for Tokyo Machine Learning Society

Tokyo Machine Learning Society

October 28, 2016

More Decks by Tokyo Machine Learning Society

See All by Tokyo Machine Learning Society
Natural Gradient Learning by Shun'ichi Amari, RIKEN - RRS #1
Avatar for Tokyo Machine Learning Society tmls
1
460
Generative Adversarial Networks by Hasanov Vagif - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
3
550
Character-Level Machine Translation with Bi-scale RNNs by Joe Bullard - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
2
430
NVIDIA Deep Learning Platform and latest information by Takeshi Izaki, Nvidia - TMLS #4　
Avatar for Tokyo Machine Learning Society tmls
1
370
Neural Translation by David and Stefano - TMLS#3
Avatar for Tokyo Machine Learning Society tmls
3
350
Extending Cloud Vision API by Ray Sakai -TMLS#3
Avatar for Tokyo Machine Learning Society tmls
2
210
Bayesian Optimization with TensorFlow/Keras by Keisuke Kamataki - TMLS #2
Avatar for Tokyo Machine Learning Society tmls
1
9.3k

Featured

See All Featured
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
950
Designing for Timeless Needs
Avatar for Cassini Nazir cassininazir
1
510
A Tale of Four Properties
Avatar for Chris Coyier chriscoyier
163
24k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
620
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
41k
Highjacked: Video Game Concept Design
Avatar for Ryan Kendrick rkendrick25
PRO
1
470
Money Talks: Using Revenue to Get Sh*t Done
Avatar for Nikki Halliwell nikkihalliwell
0
510
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
350
Agile Actions for Facilitating Distributed Teams - ADO2019
Avatar for Mark Kilby mkilby
0
290
sira's awesome portfolio website redesign presentation
Avatar for ElsiraPls elsirapls
0
430
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
The World Runs on Bad Software
Avatar for Brandon Keepers bkeepers
PRO
72
12k

Transcript

  1. "/BUVSBM1PMJDZ(SBEJFOU
 
 5SVTU3FHJPO1PMJDZ0QUJNJ[BUJPO "OESFX(BNCBSEFMMB
 3FBDUJWF3FTFBSDI4FNJOBS

  2. 3FJOGPSDFNFOU-FBSOJOH

  3. "QQSPBDIFT ↟ 7BMVFCBTFE ↟ 1PMJDZCBTFE ↟ -FBSOBQPMJDZ BDUPS BOETUBUFWBMVFGVODUJPO DSJUJD

     KPJOUMZ ↟ 0UIFST

  4. %FFQ3FJOGPSDFNFOU-FBSOJOH

  5. #VUXIZMFBSOB2GVODUJPO ↟ *ONBDIJOFMFBSOJOH XFPQUJNJ[FGPSUIFUIJOHXFDBSF BCPVU ↟ *OUIJTDBTF JUsTUIFFYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ $BOFNQJSJDBMMZTIPXUIBU%2/JTOPUWFSZHPPE

    DPNQBSFEUPQPMJDZHSBEJFOUBOEBDUPSDSJUJDNFUIPET TFF"TZODISPOPVT.FUIPETGPS%FFQ3FJOGPSDFNFOU -FBSOJOH

  6. 1PMJDZ(SBEJFOU

  7. 3&*/'03$&FTUJNBUPS
 BMTPLOPXOBTMJLFMJIPPESBUJPFTUJNBUPSTDPSF GVODUJPOFTUJNBUPS -FUYCFBSBOEPNWBSJBCMFXJUIY_Q ↟Ƕ 5IFO

  8. .BOZEJGGFSFOUDIPJDFTGPSG

  9. 1SPTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ 0QUJNJ[JOHGPSXIBUXFBDUVBMMZXBOUnNBYJNVN FYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ #FUUFSGPSIJHIEJNFOTJPOBMUBTLT ↟ #FUUFSGPSDPOUJOVPVTUBTLT NBQZPVSTUBUFUPUIFNFBO

    BOEWBSJBODFPGB(BVTTJBOEJTUSJCVUJPO  ↟ $BOCPPUTUSBQPGGPGQPMJDJFTDSFBUFECZJNJUBUJPOMFBSOJOH ↟ $BOMFBSOBTUPDIBTUJDQPMJDZ HPPEJGTPNFTUBUFTBSF BMJBTFE

  10. $POTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ %2/NPSFTBNQMFFGGJDJFOUPODFSUBJOUBTLT

  11. $POTPGQPMJDZHSBEJFOU NFUIPETJOHFOFSBM ↟ 'BJSMZEJGGJDVMUUPEFCVH ↟ &TUJNBUPSPGUIFHSBEJFOUIBTIJHIWBSJBODF ↟ $POWFSHFUPBMPDBMNJOJNVN DBOsUUSBJOCJHOFVSBMOFUT CFDBVTFPGQSFWJPVTQPJOUBOZHVBSBOUFFTBCPVUMPDBM

    NJOJNBPGFYUSFNFMZIJHIEJNFOTJPOBMGVODUJPOTEPOPU IPME

  12. )PXUPEFBMXJUIFTUJNBUPS WBSJBODF ↟ $FSUBJOUSJDLTDBOIFMQVT 4FF3FJOGPSDFNFOU-FBSOJOH /FVSBM5VSJOH.BDIJOFT PSTFF-FBSOJOH0OMJOF "MJHONFOUTXJUI$POUJOVPVT3FXBSET1PMJDZ(SBEJFOU  ↟

    8FDBOBMTPEPHSBEJFOUEFTDFOUVTJOHBEJSFDUJPOBOE PSTUFQTJ[FUIBUNBLFTNPSFTFOTFGPS3-UIBOWBOJMMB 4(%

  13. /BUVSBM1PMJDZ(SBEJFOU ↟ 6TF"NBSJsTUIFPSZUPEFGJOFBNFUSJDCBTFEOPUPOUIFDIPJDF PGDPPSEJOBUFT CVUPOUIFNBOJGPMEUIBUUIFTFDPPSEJOBUFT QBSBNFUFSJ[F ↟ "TTIPXOCZ"NBSJ UIF'JTIFSJOGPSNBUJPONBUSJYJTBO JOWBSJBOUNFUSJDPOUIFTQBDFPGQBSBNFUFSTPGQSPCBCJMJUZ

    EJTUSJCVUJPOT*UEFGJOFTUIFTBNFEJTUBODFCFUXFFOUXPQPJOUT SFHBSEMFTTPGUIFDIPJDFPGDPPSEJOBUFT JFUIF QBSBNFUFSJ[BUJPO  ↟ 5IJTHJWFTTUFFQFTUEFTDFOUEJSFDUJPO(− Ƕ ˜ǵ Ƕ XIFSF˜ǵ Ƕ  JTUIFEFTDFOUEJSFDUJPOHJWFOCZUIFtWBOJMMBuQPMJDZHSBEJFOU

  14. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ 8FXBOUUIFTUBUFTXFTFFXJUIPVSOFXQPMJDZUPCF TJNJMBSUPUIPTFXFTBXXJUIPVSPMEQPMJDZ TPXFDBO SFVTFLOPXMFEHFGSPNQBTUSPMMPVUT ↟ #ZDPOTUSBJOJOHPVSOFXQPMJDZUPCFtDMPTFuUPPVSPME QPMJDZJOUIJTXBZ

    XFDBODPOUSPMUIFBDUJPOEJTUSJCVUJPO DIBOHFBTBQSPYZ

  15. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ *GXFUBLFBTNBMMTUFQJO&VDMJEFBOQBSBNFUFSTQBDF  FWFOBTNBMMTUFQDPVMEDBVTFXJMEDIBOHFTJOUIF BDUJPOEJTUSJCVUJPO*UJTUIFSFGPSFFTTFOUJBMUIBUXFUBLF TUFQTJOUIFOBUVSBMHSBEJFOUEJSFDUJPO ↟ "MTP

    IPXDBOXFDIPPTFBOBQQSPQSJBUFTUFQTJ[F  XIFOUIFTUBUJTUJDTPGPVSEBUBBSFDPOTUBOUMZDIBOHJOH  BTPVSQPMJDZDIBOHFT TPEPFTPVSEBUB
  16. None

  17. "TMJHIUJTTVF ↟ 5IFOBUVSBMQPMJDZHSBEJFOUNVTUGPSNBOEJOWFSUUIF FNQJSJDBM'*. ↟ 5IJTJTUPPFYQFOTJWF

  18. 5SVODBUFE/BUVSBM1PMJDZ (SBEJFOU 5/1( ↟ *OTUFBEPGDPNQVUJOHUIF'*. DPNQVUFUIFTBNF EJSFDUJPOPGOBUVSBMQPMJDZHSBEJFOUVTJOHBDPOKVHBUF HSBEJFOUBMHPSJUINUIBUPOMZSFRVJSFTDPNQVUJOHBO BQQSPYJNBUJPOPG( Ƕ

    WGPSBSCJUSBSZWFDUPSW

  19. 5SVTU3FHJPO1PMJDZ 0QUJNJ[BUJPO 5310 ↟ $POTUSBJOUIF,-EJWFSHFODFCFUXFFOPVSPMEQPMJDZ BOEPVSOFXQPMJDZUPCFXJUIJOTPNFUSVTUSFHJPO SBEJVTDz ↟ #PVOEPO,-EJWFSHFODFJTBMTPBCPVOEPOIPXNVDI PVSQFSGPSNBODFDBOEFHSBEFXJUIJOUIJTPOFTUFQ

  20. 5/1(BOE5310 ↟ 5/1(DBOCFPCUBJOFEBTBTQFDJBMDBTFPG5310XJUIB MJOFBSBQQSPYJNBUJPOUP-BOEBRVBESBUJDBQQSPYJNBUJPO UPUIF%,- DPOTUSBJOU/PUJDFTUFQTJ[FǹJO5/1(
 
 
 
 


    
 
 
 5310 5/1(XBQQSPYJNBUJPO
  21. None

  22. "OEOPXGPSTPNFUIJOH DPNQMFUFMZEJGGFSFOU

  23. (FOFSBUJWF"EWFSTBSJBM /FUXPSLT ↟ "SFDFOUNFUIPEGPSVOTVQFSWJTFEMFBSOJOH ↟ (FOFSBUPS(TBNQMFTGSPNBEJTUSJCVUJPOBOEEJTDSJNJOBUPS% DMBTTJGJFTTBNQMFTBTSFBMPSGBMTF ↟ (JTUSZJOHUPtUSJDLu%CZJODSFBTJOHUIFQSPCBCJMJUZUIBU%XJMM DMBTTJGZ(sTTBNQMFTBTSFBM%JTUSZJOHUPNJOJNJ[FUIF

    QSPCBCJMJUZPGCFJOHUSJDLFE ↟ 5IJTHJWFTVTUIFBEWFSTBSJBMMPTTGVODUJPO
 
 


  24. 8IBUEPFTUIJTIBWFUP EPXJUI3-

  25. *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHTFUUJOH(JWFOUSBKFDUPSJFTGSPNBO FYQFSU MFBSOBQPMJDZ ↟ 3FHVMBSTVQFSWJTFEMFBSOJOH CFIBWJPSBMDMPOJOH XJMM BMXBZTXPSLJOUIFJOGJOJUFEBUBSFHJNF

    ↟ $BOXFNBLFTPNFUIJOHNPSFTBNQMFFGGJDJFOU  LOPXJOHUIBUXFBSFMPPLJOHGPSBO.%1QPMJDZ

  26. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHDBOCFGPSNVMBUFEBTBNJOJNBYQSPCMFN XIFSFUIFBJNJTUPMFBSOUIFDPTUGVODUJPOUIBUNJOJNJ[FT UIFBNPVOUCZXIJDIUIFPQUJNBMQPMJDZPVUQFSGPSNTUIF FYQFSUQPMJDZ ↟ #VUMFBSOJOHBDPTUGVODUJPOJTFRVJWBMFOUUPNJOJNJ[JOHUIF SFHVMBSJ[FEEJTUBODFCFUXFFOTUBUFBDUJPOPDDVQBODZ

    EJTUSJCVUJPOTGPSUIFFYQFSUQPMJDZBOEMFBSOFEQPMJDZ ↟ #ZVTJOHBEFFQOFVSBMOFUXPSLXJUIUIFDSPTTFOUSPQZMPTT BTUIFEJTUBODFCFUXFFOPDDVQBODZEJTUSJCVUJPOT XFDBO GPSHPMFBSOJOHBDPTUGVODUJPODPNQMFUFMZ

  27. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH

  28. None

  29. 3FTPVSDFT ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH1POHGSPN1JYFMT "OESFK ,BSQBUIZsTCMPH ↟ -FDUVSF1PMJDZ(SBEJFOU %BWJE4JMWFST3-$PVSTF ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH"5VUPSJBM

    0QFO"* ↟ 1PMJDZ(SBEJFOU.FUIPET 4DIPMBSQFEJB

  30. 3FGFSFODFT ↟ "NBSJ 4/BUVSBMHSBEJFOUXPSLTFGGJDJFOUMZJOMFBSOJOH/FVSBM$PNQVUBUJPO      ↟

    %VBO: $IFO9 )PVUIPPGU3 4DIVMNBO+ BOE"CCFFM1#FODINBSLJOHEFFQ SFJOGPSDFNFOUMFBSOJOHGPSDPOUJOVPVTDPOUSPMBS9JWQSFQSJOUBS9JW  ↟ )P + BOE&SNPO 4(FOFSBUJWFBEWFSTBSJBMJNJUBUJPOMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ ,BLBEF 4"OBUVSBMQPMJDZHSBEJFOU*O"EWBODFTJO/FVSBM*OGPSNBUJPO1SPDFTTJOH 4ZTUFNT QQm.*51SFTT  ↟ -VP: $IJV$ +BJUMZ. BOE4VUTLFWFS*-FBSOJOHPOMJOFBMJHONFOUTXJUIDPOUJOVPVT SFXBSETQPMJDZHSBEJFOUBS9JWQSFQSJOUBS9JW  ↟ .OJI7 #BEJB" .JS[B. (SBWFT" -JMMJDSBQ5 )BSMFZ5 4JMWFS% BOE,BWVLDVPHMV, "TZODISPOPVTNFUIPETGPSEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW  

  31. 3FGFSFODFT ↟ .OJI 7 ,BWVLDVPHMV , 4JMWFS % (SBWFT "

    "OUPOPHMPV * 8JFSTUSB % BOE 3JFENJMMFS .1MBZJOH"UBSJXJUIEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ 1GBV % BOE7JOZBMT 0$POOFDUJOHHFOFSBUJWFBEWFSTBSJBMOFUXPSLTBOEBDUPS DSJUJDNFUIPETBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + -FWJOF 4 .PSJU[ 1 +PSEBO . BOE"CCFFM 15SVTUSFHJPOQPMJDZ PQUJNJ[BUJPOBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + .PSJU[ 1 -FWJOF 4 +PSEBO . BOE"CCFFM 1)JHIEJNFOTJPOBM DPOUJOVPVTDPOUSPMVTJOHHFOFSBMJ[FEBEWBOUBHFFTUJNBUJPOBS9JWQSFQSJOUBS9JW   ↟ ;BSFNCB 8 BOE4VUTLFWFS *3FJOGPSDFNFOUMFBSOJOHOFVSBMUVSJOHNBDIJOFT BS9JWQSFQSJOUBS9JW 

  32. 2VFTUJPOT