Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Scalable Job Queue System Built with Docker
Takashi Kokubun
September 08, 2016
Programming
14
21k
Scalable Job Queue System Built with Docker
RubyKaigi 2016
http://rubykaigi.org/2016
Takashi Kokubun
September 08, 2016
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
1
1.3k
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
8.4k
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
200
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
3
1.4k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
20
13k
Ruby 3 JIT's roadmap / RubyConf China 2020
k0kubun
0
500
Ruby 3.0 JIT on Rails
k0kubun
9
8.3k
JIT ロードマップ / Ruby 3 さみっと
k0kubun
2
1k
JIT compiler improvements in Ruby 2.7 / RubyRussia 2019
k0kubun
0
720
Other Decks in Programming
See All in Programming
量子コンピュータ時代のプログラミングセミナー / 20221222_Amplify_seminar _route_optimization
fixstars
0
250
tidy_rpart
bk_18
0
590
Rによる大規模データの処理
s_uryu
2
640
社会人 20 年目エンジニア、発信で技術学びなおしてる話
e99h2121
1
140
まだ日本国内で利用できないAppActionsにトライしてみた / MoT TechTalk #15
mot_techtalk
0
110
和暦を正しく扱うための暦の話
nagise
10
6.4k
2023年にクル(かもしれない)通信ミドルウェア技術(仮)
s_hosoai
0
200
Cloudflare Workersと状態管理
chimame
3
480
低レイヤーから始める GUI
fadis
18
9.4k
データドリブンな組織の不正検知
fkubota
0
240
ポケモンで学ぶiOS 16弾丸ツアー 🚅
giginet
PRO
1
610
ちょうぜつ改め21世紀ふつうのソフトウェア設計
tanakahisateru
7
6.4k
Featured
See All Featured
We Have a Design System, Now What?
morganepeng
37
5.9k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
24
4.5k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
Typedesign – Prime Four
hannesfritz
34
1.5k
Web development in the modern age
philhawksworth
197
9.6k
Adopting Sorbet at Scale
ufuk
65
7.8k
Designing the Hi-DPI Web
ddemaree
273
32k
Automating Front-end Workflow
addyosmani
1351
200k
A Tale of Four Properties
chriscoyier
149
21k
Building Applications with DynamoDB
mza
85
5k
Creatively Recalculating Your Daily Design Routine
revolveconf
207
11k
Testing 201, or: Great Expectations
jmmastey
25
5.7k
Transcript
4DBMBCMF+PC2VFVF4ZTUFN #VJMUXJUI%PDLFS 5BLBTIJ,PLVCVO$PPLQBE*OD 4FQUFNCFSUI 3VCZ,BJHJ
5BLBTIJ,PLVCVO!LLVCVO %FWFMPQFS1SPEVDUJWJUZ(SPVQ $PPLQBE*OD )BNMJU JUBNBF\ HP NSVCZ^
8FSFIJSJOH IUUQTSFDSVJUDPPLQBEDPN IUUQTJOGPDPPLQBEDPNFODBSFFST
5PEBZTUPQJD w 8IBUJTBKPCRVFVF w *OUSPEVDUJPOUP#BSCFRVF$PPLQBETOFXKPCRVFVFTZTUFN w 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN w
)PXJT#BSCFRVFEFTJHOFE
8IBUJTBKPCRVFVF 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
"KPCRVFVFJT w *OUIFDPOUFYUPGBXFCBQQMJDBUJPO JUTBTZTUFNVTFEUPQFSGPSN UBTLTBTZODISPOPVTMZ w JFQFSGPSNTMPXUBTLTMBUFSJOTUFBEPGEVSJOHSFRVFTUUPBQQ TFSWFS
"KPCRVFVFJT w 8JUIPVUBKPCRVFVF VTFSTIBWFUPXBJUVOUJMBQQTFSWFSDPNQMFUFTBTMPX UBTL VTFS BQQ TMPXUBTL SFRVFTU SFTQPOTF
"KPCRVFVFJT w 8JUIBKPCRVFVF VTFSTEPOUIBWFUPXBJUGPSUIFTMPXUBTLCFDBVTFJUJT QFSGPSNFEBTZODISPOPVTMZ VTFS BQQ RVFVF XPSLFS FORVFVFBUBTL
EFRVFVFBUBTL TMPXUBTL SFRVFTU SFTQPOTF
)PXUPVTFBKPCRVFVFTZTUFNJO3VCZ w %FpOFBKPCVTJOHSVCZHFNGPSBKPCRVFVFTZTUFN w SFTRVF TJEFLJR w $BMMNFUIPEUPFORVFVFBKPC w
.POJUPSUBTLTBOEXPSLFSTXJUIBXFCDPOTPMF
%FpOFBKPC FYBNQMFPGSFTRVFHFN class TestJob @queue = :default def self.perform(user_id) #
do something end end Resque.enqueue(TestJob, 100)
.FUIPETUPFORVFVFBKPC w 6TFBNFUIPEJOUIFHFNGPSBKPCRVFVFTZTUFN w 3FTRVFFORVFVF w 4JEFLJR8PSLFS
[email protected]
w %FMBZFE+PCFORVFVF w
4IPSZVLFO8PSLFS
[email protected]
w *O3BJMT ZPVDBOVTFBOBCTUSBDUJPOMBZFS w "DUJWF+PC&ORVFVJOH
[email protected]
8FCDPOTPMF w SFTRVFXFC w TJEFLJR XFCJOUFSGBDFCVOEMFE w
[email protected]
@XFC
8FCDPOTPMF
8IBUEPFTBKPCRVFVFTZTUFNDPOTJTUPG w %BUBTUPSFGPSBRVFVF w 8PSLFSUPQFSGPSNBKPC
%BUBTUPSFGPSBRVFVF w 3FEJT w 3%#.4 w .Z42- 1PTUHSF42- w 3BCCJU.2
w "NB[PO424 w
8PSLFSUPQFSGPSNBKPC GPS3VCZ w 6TJOH3FEJT w SFTRVF w TJEFLJR w 6TJOH3%#.4
w
[email protected]
w 6TJOH"NB[PO424 w TIPSZVLFO
*OUSPEVDUJPOUP#BSCFRVF $PPLQBETOFXKPCRVFVFTZTUFN 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
#BSCFRVF w IUUQTHJUIVCDPNDPPLQBECBSCFRVF w *NQMFNFOUFEBTSVCZHFN w $PSFPGPVSOFXKPCRVFVFTZTUFN w %PDVNFOUBUJPOJTXPSLJOQSPHSFTT
$PNQPOFOUTJOCBSCFRVFHFN w 8PSLFS w #VJMUXJUI4FSWFSFOHJOF w &YFDVUFBKPCVTJOH%PDLFS w 4MBDLOPUJpDBUJPO w
2VFVFJOHJOUFSGBDF 8FC"1* w 8FCDPOTPMF w .BOBHFSFHJTUFSFEBQQMJDBUJPOTBOEOPUJpDBUJPOT w :PVDBOTFFBMPHTUPSFEJO4
2VFVFJOH"1* w 1045W
[email protected]
w &ORVFVFBNFTTBHFUP424 w 1BSBNFUFST w BQQMJDBUJPOTUSJOH w
KPCTUSJOH w RVFVFTUSJOH w NFTTBHFBOZ PCKFDU BSSBZ TUSJOH
2VFVFJOH"1*
2VFVFJOH"1*
$MJFOUJOUFSGBDFGPS3VCZ
[email protected]
HFN BarbequeClient.configure do |config| config.application = 'blog' config.default_queue =
'default' config.endpoint = 'http://www.example.com' end response = BarbequeClient.enqueue( job: 'NotifyAuthor', message: { article_id: 1 }, ) response.message_id #=> "edb7ef55-f1d0-46cf-bcca-036acb0ebd96" response.status #=> "pending"
3FRVFVFJOH"1* w 1045W
[email protected]
[email protected]
SFUSJFT w 3FRVFVFBNFTTBHFUP424 w 1BSBNFUFST w
[email protected]
JOUFHFS
3FRVFVFJOH"1*
3FRVFVFJOH"1*
8FCDPOTPMF
8IZEJEXFOFFE BOFXKPCRVFVFTZTUFN 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
"TIPSUIJTUPSZPGKPCRVFVFTZTUFNTBU$PPLQBE w 4PNFMJNJUFEBQQMJDBUJPOTXFSFVTJOHBKPCRVFVF w 3FTRVF 3FEJT JOIPVTFXFCDPOTPMF w 4JEFLJR
3FEJT 4JEFLJR8FC
3FTRVF 3FEJT JOIPVTFXFCDPOTPMF VOJDPSO BQQDPOTPMF BQQTFSWFS SFRVFTU HPE SFEJTTFSWFS
SFEJT NPOJUPS SFTRVF FORVFVF EFRVFVF
3FTRVF 3FEJT JOIPVTFXFCDPOTPMF w 4JODFSFTRVFXPSLTJONVMUJQSPDFTTFT UIFSFTOPOFFEUPUIJOL BCPVUUISFBETBGFUZ w *G3FEJTJTOPUEJTUSJCVUFE
EVQMJDBUFNFTTBHFTBSFOUEFMJWFSFE
4JEFLJR 3FEJT 4JEFLJR8FC w 'BTUFSUIBO3FTRVF w IUUQTHJUIVCDPNNQFSIBNTJEFLJRQFSGPSNBODF w :PVNVTUJNQMFNFOUKPCTJOBUISFBETBGFXBZCFDBVTFXPSLFST
BSFNVMUJUISFBEFE
8IZEJEOUXFVTFBKPCRVFVFTPGSFRVFOUMZ w 8FIBWFLVSPLP XIJDIJTBOVTFGVMTDIFEVMFECBUDITZTUFN w 'FBUVSFSJDIXFCDPOTPMF w 4VQQPSUKPCFYFDVUJPOXJUI%PDLFS w .BOZEFWFMPQFSTMPWFLVSPLPBOEVTFJUUPQFSGPSNBUBTL
BTZODISPOPVTMZ w #VUVTFSTIBWFUPXBJUUIFTDIFEVMFECBUDIFYFDVUJPO w 8FTIPVMEVTFBKPCRVFVFTZTUFNGPSVTFSTDPOWFOJFODF
LVSPLP *UMMCFPQFOTPVSDFEBMJUUMFMBUFS
8IBUEJEXFXBOUGPS$PPLQBETKPCRVFVFTZTUFN w $FOUSBMJ[FENBOBHFNFOUPGXPSLFST w 8FIBWFTPNBOZBQQMJDBUJPOTBOEEPOUXBOUUPNBOBHF JOGSBTUSVDUVSFGPSFBDIBQQMJDBUJPO w &BTZKPCEFQMPZNFOU w LVSPLPMJLFXFCDPOTPMF
)PXUPEFQMPZBKPC w 8FSFBMSFBEZVTJOH%PDLFSUPEFQMPZNPTUPGPVSBQQMJDBUJPOT w EFQMPZNFOUDPOpHVSBUJPOT w 8FVTF&$4UPSVO%PDLFSDPOUBJOFSTJOBDMVTUFS w &$4"NB[PO&$$POUBJOFS4FSWJDF
w 4DBMFJOPVUDBOCFEPOFCZBEKVTUJOHEFTJSFEDBQBDJUZPG "VUP4DBMJOH(SPVQ
)PXJT#BSCFRVFEFTJHOFE 8IBUJTBKPCRVFVF *OUSPEVDUJPOUP#BSCFRVF 8IZEJEXFOFFEBOFXKPCRVFVFTZTUFN )PXJT#BSCFRVFEFTJHOFE
%BUBTUPSFGPSBRVFVF"NB[PO424 w .BOBHFENFTTBHFRVFVF w 1SPT w 'BTU TDBMBCMFBOESFMJBCMF w &BTZUPJOUFHSBUFXJUIPUIFS"84DPNQPOFOUTMJLF"NB[PO4/4
w $POT w .BZEFMJWFSEVQMJDBUFNFTTBHFT 2P4JTBUMFBTUPODF w .FTTBHFEFMBZEVSBUJPOJTMJNJUFEVQUPT
2P4BUMFBTUPODF BUNPTUPODFPSFYBDUMZPODF w 2VBMJUZPG4FSWJDFJTEFDJEFECZBEBUBTUPSFGPSBRVFVFBOE JNQMFNFOUBUJPOPGBXPSLFS w 2P4 w BUNPTUPODF6OSFMJBCMFCVUOPEVQMJDBUJPO w
BUMFBTUPODF3FMJBCMFCVUEVQMJDBUFEFMJWFSZPDDVST w FYBDUMZPODF.PTUSFMJBCMFCVUIBSEUPNBLFBTZTUFNTDBMBCMF
8IZEJEXFDIPPTF"NB[PO424 w *UTIPVMECFTDBMBCMFBOESFMJBCMFCFDBVTFJUTVTFECZBMM BQQMJDBUJPOT w %VQMJDBUFEFMJWFSZJTBMMPXFEJOPVSNPTUVTFDBTFT
8PSLFSUPQFSGPSNBKPC w #BSCFRVFXPSLFSJTJNQMFNFOUFEXJUI3VCZ w 4JODFUIFKPCQFSGPSNFEPOBKPCRVFVFTZTUFNUFOETUPCFB TMPXUBTL XFDBOJHOPSFUIFPWFSIFBEJOBXPSLFSMBZFS w 5PQFSGPSNBKPC #BSCFRVFFYFDVUFTBDPNNBOEXJUITPNF
TQFDJBMFOWJSPONFOUWBSJBCMFTVTJOH%PDLFS w 5IFZJODMVEFBTFSJBMJ[FENFTTBHFBOEBKPCOBNF
+PCFYFDVUJPOPO&$4 w #BSCFRVFXPSLFSVTFTIBLPUPQFSGPSNBKPC XIJDIJTBEFQMPZ UPPMGPS%PDLFS w IBLPIUUQTHJUIVCDPNFBHMFUNUIBLP w 8FSFBMSFBEZVTJOHUIJTUPEFQMPZNBOZBQQMJDBUJPOTUP&$4 w
8IJMFIBLPJTPQFOTPVSDFE IBLPBEBQUFSGPS#BSCFRVFJTTUJMM DMPTFETPVSDF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* FORVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* "NB[PO424 FORVFVF FORVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF EFRVFVF
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF EFRVFVF 3%4
.Z42- TUPSFTUBUVT
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* &$4CBUDIDMVTUFS BQQ CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF
EFRVFVF 3%4 .Z42- TUPSFTUBUVT FYFDVUFBDPNNBOE
#BSCFRVFXPSLFSXPSLqPX BQQ SFRVFTU CBSCFRVF"1* &$4CBUDIDMVTUFS BQQ CBSCFRVFXPSLFS "NB[PO424 FORVFVF FORVFVF
EFRVFVF TUPSFTUEPVU TUEFSS "NB[PO4 3%4 .Z42- TUPSFTUBUVT FYFDVUFBDPNNBOE
"VUPTDBMJOH w 6TJOH"VUP4DBMJOHHSPVQUPTDBMFBO&$4DMVTUFS w 4DBMFJOPVUBSFSVOEFQFOEJOHPOSFTPVSDFVTBHFPG&$4 DMVTUFS w 4JODFBKPCJTPOFUBTLPO&$4 XFDBOFTUJNBUFBQSFDJTF SFTPVSDFUPFYFDVUFKPCT
w 3FTPVSDFUPSVOBO&$4UBTLJTEFpOFECFGPSFIBOE w 8FDBOSFEVDFDPTUXIFOKPCTBSFOPUQFSGPSNFEGSFRVFOUMZ
$PODMVTJPO w 8FEFTJHOFEBOFXKPCRVFVFTZTUFNUP w $FOUSBMJ[FNBOBHFNFOUPGKPCRVFVFTZTUFNTXJUI%PDLFS w "VUPNBUFTDBMJOHCZ&$4BOE"VUP4DBMJOH(SPVQ w 2VFVFJOH"1* XFCDPOTPMFBOEDPSFPGXPSLFSJTPQFOTPVSDFE
w IUUQTHJUIVCDPNDPPLQBECBSCFRVF