Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Scalable Job Queue System Built with Docker
Takashi Kokubun
September 08, 2016
Programming
14
21k
Scalable Job Queue System Built with Docker
RubyKaigi 2016
http://rubykaigi.org/2016
Takashi Kokubun
September 08, 2016
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
96
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
2
1.2k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
19
12k
Ruby 3 JIT's roadmap / RubyConf China 2020
k0kubun
0
420
Ruby 3.0 JIT on Rails
k0kubun
8
7.9k
JIT ロードマップ / Ruby 3 さみっと
k0kubun
2
940
JIT compiler improvements in Ruby 2.7 / RubyRussia 2019
k0kubun
0
620
Optimizing Ruby with JIT
k0kubun
13
10k
Ruby 2.7 JIT on Rails / RailsConf 2019
k0kubun
1
590
Other Decks in Programming
See All in Programming
Reactでアプリケーションを構築する多様化
sakito
4
3.1k
2022 FrontEnd Training
mixi_engineers
1
110
Reinventing the wheel ... as a service
mariofusco
3
230
読みやすいコードを書こう
yutorin
0
380
Let's make a contract: the art of designing a Java API
mariofusco
0
160
roadmap to rust 2024
matsu7874
1
620
Groovy Roadmap
paulk
7
13k
アプリのログをチーム外で活用してもらうためにやったこと
shotakashihara
0
130
クリエイティブ系のウェブサイト制作で役立つCSS技法 / CSS for develop creative website
clockmaker
2
1.3k
Develop your CI tools
xgouchet
2
180
Swift Concurrencyによる安全で快適な非同期処理
tattn
2
170
About Type Syntax Proposal
quramy
1
940
Featured
See All Featured
Side Projects
sachag
449
37k
Build your cross-platform service in a week with App Engine
jlugia
219
17k
Typedesign – Prime Four
hannesfritz
33
1.3k
The Invisible Side of Design
smashingmag
289
48k
Unsuck your backbone
ammeep
659
55k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
29
4.3k
Designing the Hi-DPI Web
ddemaree
272
32k
Code Review Best Practice
trishagee
41
6.7k
Stop Working from a Prison Cell
hatefulcrawdad
261
17k
Done Done
chrislema
174
14k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
3
430
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
119
28k
Transcript
4DBMBCMF+PC2VFVF4ZTUFN #VJMUXJUI%PDLFS 5BLBTIJ,PLVCVO$PPLQBE*OD 4FQUFNCFSUI 3VCZ,BJHJ
5BLBTIJ,PLVCVO!LLVCVO %FWFMPQFS1SPEVDUJWJUZ(SPVQ $PPLQBE*OD )BNMJU JUBNBF\ HP NSVCZ^
8FSFIJSJOH IUUQTSFDSVJUDPPLQBEDPN IUUQTJOGPDPPLQBEDPNFODBSFFST
5PEBZTUPQJD w 8IBUJTBKPCRVFVF w *OUSPEVDUJPOUP#BSCFRVF$PPLQBETOFXKPCRVFVFTZTUFN w 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN w
)PXJT#BSCFRVFEFTJHOFE
8IBUJTBKPCRVFVF 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
"KPCRVFVFJT w *OUIFDPOUFYUPGBXFCBQQMJDBUJPO JUTBTZTUFNVTFEUPQFSGPSN UBTLTBTZODISPOPVTMZ w JFQFSGPSNTMPXUBTLTMBUFSJOTUFBEPGEVSJOHSFRVFTUUPBQQ TFSWFS
"KPCRVFVFJT w 8JUIPVUBKPCRVFVF VTFSTIBWFUPXBJUVOUJMBQQTFSWFSDPNQMFUFTBTMPX UBTL VTFS BQQ TMPXUBTL SFRVFTU SFTQPOTF
"KPCRVFVFJT w 8JUIBKPCRVFVF VTFSTEPOUIBWFUPXBJUGPSUIFTMPXUBTLCFDBVTFJUJT QFSGPSNFEBTZODISPOPVTMZ VTFS BQQ RVFVF XPSLFS FORVFVFBUBTL
EFRVFVFBUBTL TMPXUBTL SFRVFTU SFTQPOTF
)PXUPVTFBKPCRVFVFTZTUFNJO3VCZ w %FpOFBKPCVTJOHSVCZHFNGPSBKPCRVFVFTZTUFN w SFTRVF TJEFLJR w $BMMNFUIPEUPFORVFVFBKPC w
.POJUPSUBTLTBOEXPSLFSTXJUIBXFCDPOTPMF
%FpOFBKPC FYBNQMFPGSFTRVFHFN class TestJob @queue = :default def self.perform(user_id) #
do something end end Resque.enqueue(TestJob, 100)
.FUIPETUPFORVFVFBKPC w 6TFBNFUIPEJOUIFHFNGPSBKPCRVFVFTZTUFN w 3FTRVFFORVFVF w 4JEFLJR8PSLFSQFSGPSN@BTZOD w %FMBZFE+PCFORVFVF w
4IPSZVLFO8PSLFSQFSGPSN@BTZOD w *O3BJMT ZPVDBOVTFBOBCTUSBDUJPOMBZFS w "DUJWF+PC&ORVFVJOHQFSGPSN@MBUFS
8FCDPOTPMF w SFTRVFXFC w TJEFLJR XFCJOUFSGBDFCVOEMFE w EFMBZFE@KPC@XFC
8FCDPOTPMF
8IBUEPFTBKPCRVFVFTZTUFNDPOTJTUPG w %BUBTUPSFGPSBRVFVF w 8PSLFSUPQFSGPSNBKPC
%BUBTUPSFGPSBRVFVF w 3FEJT w 3%#.4 w .Z42- 1PTUHSF42- w 3BCCJU.2
w "NB[PO424 w
8PSLFSUPQFSGPSNBKPC GPS3VCZ w 6TJOH3FEJT w SFTRVF w TJEFLJR w 6TJOH3%#.4
w EFMBZFE@KPC w 6TJOH"NB[PO424 w TIPSZVLFO
*OUSPEVDUJPOUP#BSCFRVF $PPLQBETOFXKPCRVFVFTZTUFN 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
#BSCFRVF w IUUQTHJUIVCDPNDPPLQBECBSCFRVF w *NQMFNFOUFEBTSVCZHFN w $PSFPGPVSOFXKPCRVFVFTZTUFN w %PDVNFOUBUJPOJTXPSLJOQSPHSFTT
$PNQPOFOUTJOCBSCFRVFHFN w 8PSLFS w #VJMUXJUI4FSWFSFOHJOF w &YFDVUFBKPCVTJOH%PDLFS w 4MBDLOPUJpDBUJPO w
2VFVFJOHJOUFSGBDF 8FC"1* w 8FCDPOTPMF w .BOBHFSFHJTUFSFEBQQMJDBUJPOTBOEOPUJpDBUJPOT w :PVDBOTFFBMPHTUPSFEJO4
2VFVFJOH"1* w 1045WKPC@FYFDVUJPOT w &ORVFVFBNFTTBHFUP424 w 1BSBNFUFST w BQQMJDBUJPOTUSJOH w
KPCTUSJOH w RVFVFTUSJOH w NFTTBHFBOZ PCKFDU BSSBZ TUSJOH
2VFVFJOH"1*
2VFVFJOH"1*
$MJFOUJOUFSGBDFGPS3VCZ CBSCFRVF@DMJFOUHFN BarbequeClient.configure do |config| config.application = 'blog' config.default_queue =
'default' config.endpoint = 'http://www.example.com' end response = BarbequeClient.enqueue( job: 'NotifyAuthor', message: { article_id: 1 }, ) response.message_id #=> "edb7ef55-f1d0-46cf-bcca-036acb0ebd96" response.status #=> "pending"
3FRVFVFJOH"1* w 1045WKPC@FYFDVUJPOTNFTTBHF@JESFUSJFT w 3FRVFVFBNFTTBHFUP424 w 1BSBNFUFST w EFMBZ@TFDPOETJOUFHFS
3FRVFVFJOH"1*
3FRVFVFJOH"1*
8FCDPOTPMF
8IZEJEXFOFFE BOFXKPCRVFVFTZTUFN 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
"TIPSUIJTUPSZPGKPCRVFVFTZTUFNTBU$PPLQBE w 4PNFMJNJUFEBQQMJDBUJPOTXFSFVTJOHBKPCRVFVF w 3FTRVF 3FEJT JOIPVTFXFCDPOTPMF w 4JEFLJR
3FEJT 4JEFLJR8FC
3FTRVF 3FEJT JOIPVTFXFCDPOTPMF VOJDPSO BQQDPOTPMF BQQTFSWFS SFRVFTU HPE SFEJTTFSWFS
SFEJT NPOJUPS SFTRVF FORVFVF EFRVFVF
3FTRVF 3FEJT JOIPVTFXFCDPOTPMF w 4JODFSFTRVFXPSLTJONVMUJQSPDFTTFT UIFSFTOPOFFEUPUIJOL BCPVUUISFBETBGFUZ w *G3FEJTJTOPUEJTUSJCVUFE
EVQMJDBUFNFTTBHFTBSFOUEFMJWFSFE
4JEFLJR 3FEJT 4JEFLJR8FC w 'BTUFSUIBO3FTRVF w IUUQTHJUIVCDPNNQFSIBNTJEFLJRQFSGPSNBODF w :PVNVTUJNQMFNFOUKPCTJOBUISFBETBGFXBZCFDBVTFXPSLFST
BSFNVMUJUISFBEFE
8IZEJEOUXFVTFBKPCRVFVFTPGSFRVFOUMZ w 8FIBWFLVSPLP XIJDIJTBOVTFGVMTDIFEVMFECBUDITZTUFN w 'FBUVSFSJDIXFCDPOTPMF w 4VQQPSUKPCFYFDVUJPOXJUI%PDLFS w .BOZEFWFMPQFSTMPWFLVSPLPBOEVTFJUUPQFSGPSNBUBTL
BTZODISPOPVTMZ w #VUVTFSTIBWFUPXBJUUIFTDIFEVMFECBUDIFYFDVUJPO w 8FTIPVMEVTFBKPCRVFVFTZTUFNGPSVTFSTDPOWFOJFODF
LVSPLP *UMMCFPQFOTPVSDFEBMJUUMFMBUFS
8IBUEJEXFXBOUGPS$PPLQBETKPCRVFVFTZTUFN w $FOUSBMJ[FENBOBHFNFOUPGXPSLFST w 8FIBWFTPNBOZBQQMJDBUJPOTBOEEPOUXBOUUPNBOBHF JOGSBTUSVDUVSFGPSFBDIBQQMJDBUJPO w &BTZKPCEFQMPZNFOU w LVSPLPMJLFXFCDPOTPMF
)PXUPEFQMPZBKPC w 8FSFBMSFBEZVTJOH%PDLFSUPEFQMPZNPTUPGPVSBQQMJDBUJPOT w EFQMPZNFOUDPOpHVSBUJPOT w 8FVTF&$4UPSVO%PDLFSDPOUBJOFSTJOBDMVTUFS w &$4"NB[PO&$$POUBJOFS4FSWJDF
w 4DBMFJOPVUDBOCFEPOFCZBEKVTUJOHEFTJSFEDBQBDJUZPG "VUP4DBMJOH(SPVQ
)PXJT#BSCFRVFEFTJHOFE 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
%BUBTUPSFGPSBRVFVF"NB[PO424 w .BOBHFENFTTBHFRVFVF w 1SPT w 'BTU TDBMBCMFBOESFMJBCMF w &BTZUPJOUFHSBUFXJUIPUIFS"84DPNQPOFOUTMJLF"NB[PO4/4
w $POT w .BZEFMJWFSEVQMJDBUFNFTTBHFT 2P4JTBUMFBTUPODF w .FTTBHFEFMBZEVSBUJPOJTMJNJUFEVQUPT
2P4BUMFBTUPODF BUNPTUPODFPSFYBDUMZPODF w 2VBMJUZPG4FSWJDFJTEFDJEFECZBEBUBTUPSFGPSBRVFVFBOE JNQMFNFOUBUJPOPGBXPSLFS w 2P4 w BUNPTUPODF6OSFMJBCMFCVUOPEVQMJDBUJPO w
BUMFBTUPODF3FMJBCMFCVUEVQMJDBUFEFMJWFSZPDDVST w FYBDUMZPODF.PTUSFMJBCMFCVUIBSEUPNBLFBTZTUFNTDBMBCMF
8IZEJEXFDIPPTF"NB[PO424 w *UTIPVMECFTDBMBCMFBOESFMJBCMFCFDBVTFJUTVTFECZBMM BQQMJDBUJPOT w %VQMJDBUFEFMJWFSZJTBMMPXFEJOPVSNPTUVTFDBTFT
8PSLFSUPQFSGPSNBKPC w #BSCFRVFXPSLFSJTJNQMFNFOUFEXJUI3VCZ w 4JODFUIFKPCQFSGPSNFEPOBKPCRVFVFTZTUFNUFOETUPCFB TMPXUBTL XFDBOJHOPSFUIFPWFSIFBEJOBXPSLFSMBZFS w 5PQFSGPSNBKPC #BSCFRVFFYFDVUFTBDPNNBOEXJUITPNF
TQFDJBMFOWJSPONFOUWBSJBCMFTVTJOH%PDLFS w 5IFZJODMVEFBTFSJBMJ[FENFTTBHFBOEBKPCOBNF
+PCFYFDVUJPOPO&$4 w #BSCFRVFXPSLFSVTFTIBLPUPQFSGPSNBKPC XIJDIJTBEFQMPZ UPPMGPS%PDLFS w IBLPIUUQTHJUIVCDPNFBHMFUNUIBLP w 8FSFBMSFBEZVTJOHUIJTUPEFQMPZNBOZBQQMJDBUJPOTUP&$4 w
8IJMFIBLPJTPQFOTPVSDFE IBLPBEBQUFSGPS#BSCFRVFJTTUJMM DMPTFETPVSDF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* FORVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* "NB[PO424 FORVFVF FORVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF EFRVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF EFRVFVF 3%4
.Z42- TUPSFTUBUVT
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* &$4CBUDIDMVTUFS BQQ CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF
EFRVFVF 3%4 .Z42- TUPSFTUBUVT FYFDVUFBDPNNBOE
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* &$4CBUDIDMVTUFS BQQ CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF
EFRVFVF TUPSFTUEPVU TUEFSS "NB[PO4 3%4 .Z42- TUPSFTUBUVT FYFDVUFBDPNNBOE
"VUPTDBMJOH w 6TJOH"VUP4DBMJOHHSPVQUPTDBMFBO&$4DMVTUFS w 4DBMFJOPVUBSFSVOEFQFOEJOHPOSFTPVSDFVTBHFPG&$4 DMVTUFS w 4JODFBKPCJTPOFUBTLPO&$4 XFDBOFTUJNBUFBQSFDJTF SFTPVSDFUPFYFDVUFKPCT
w 3FTPVSDFUPSVOBO&$4UBTLJTEFpOFECFGPSFIBOE w 8FDBOSFEVDFDPTUXIFOKPCTBSFOPUQFSGPSNFEGSFRVFOUMZ
$PODMVTJPO w 8FEFTJHOFEBOFXKPCRVFVFTZTUFNUP w $FOUSBMJ[FNBOBHFNFOUPGKPCRVFVFTZTUFNTXJUI%PDLFS w "VUPNBUFTDBMJOHCZ&$4BOE"VUP4DBMJOH(SPVQ w 2VFVFJOH"1* XFCDPOTPMFBOEDPSFPGXPSLFSJTPQFOTPVSDFE
w IUUQTHJUIVCDPNDPPLQBECBSCFRVF