지연 시간 순삭! LLM 추론 구조와 효율적 애플리케이션 설계

ߊ಴੗ࣗѐ "*-BC "*ূ૑פয நܼఠ ಕܰࣗա

--. पदр গ೒ܻா੉࣌ ੉ߣߊ಴ীࢲ׮ܖחѪ BVUPSFHSFTTJWF ݽ؛ EFDPEFS - POMZ ౟ےझನݠ
୶ۿ ૑োदр ਽׹ࣘب

੉ߣߊ಴ীࢲ׮ܖ૑ঋחѪ QSVOJOH FODPEFS - POMZ ౟ےझನݠ ࢎ੹೟ण FODPEFS - EFDPEFS
౟ےझನݠ ੹੉೟ण ন੗ച ૑धૐܨ

(JUIVC$PQJMPU 1FSQMFYJUZ ୁࠈ ߡ୶঴ോݢ पदр--.গ೒ܻா੉࣌ IUUQT HJUIVCDPNGFBUVSFTDPQJMPU] IUUQT
XXXQFSQMFYJUZBJ

--. ਬ੷ --.ױة୊ܻ

--. ਬ੷ 3"( ୭न بݫੋౠച ࠺ҕѐ ޹х -FXJT 1BUSJDL
FUBM3FUSJFWBM - BVHNFOUFEHFOFSBUJPOGPSLOPXMFEHF - JOUFOTJWFOMQUBTLT

--. ਬ੷ "*"HFOU ੘স ୭न بݫੋౠച ࠺ҕѐ ޹х

ਃ୒ ੄ب࠙ࢳ ҙ۲ࢿ ಣо ੿ࠁр ਋ࢶࣽਤ ࠗৈ Ә஗য х૑ ਬ೧੿ࠁ
х૑ ఐ࢝੿ࠁ ಿ૕ಣо

--.௾Ӓܿ --.ੑ۱ --.୹۱ --.ղࠗ

--. tఫझ౟੄׮਺ױয৘ஏu

--.ఫझ౟׮਺ױয৘ஏ оਸী֥۞оӝજ਷Ҕ਷ ࢑ आ য٣ ૘ 4/4 Ӕ୊ ߄׮ ઱߸
ੋఠ֔ ѥӝ j --.

--. ೞܖ য়ט੿݈ "VUPSFHSFTTJWFݽ؛ --. о য়ט੿݈ೞܖ --. ӡ য়ט੿݈ೞܖо
4IBOBIBO FUBM3PMFQMBZXJUIMBSHFMBOHVBHFNPEFMT

%FDPEFS - 0OMZ 5SBOTGPSNFS

5SBOTGPSNFSইఃఫ୊ 5SBOTGPSNFS &ODPEFS - %FDPEFS 5
&ODPEFS - 0OMZ #&35 %FDPEFS - 0OMZ (15 7BTXBOJ "TIJTI FUBM"UUFOUJPOJTBMMZPVOFFE ] 3BGGFM $PMJO FUBM&YQMPSJOHUIFMJNJUTPGUSBOTGFSMFBSOJOHXJUIBVOJ fi FEUFYU - UP - UFYUUSBOTGPSNFSu ] %FWMJO +BDPC#FSU 1SF - USBJOJOHPGEFFQCJEJSFDUJPOBMUSBOTGPSNFSTGPSMBOHVBHFVOEFSTUBOEJOH ] 3BEGPSE "MFD FUBM*NQSPWJOHMBOHVBHFVOEFSTUBOEJOHCZHFOFSBUJWFQSF - USBJOJOH

%FDPEFS - 0OMZ5SBOTGPSNFSইఃఫ୊ IUUQT BJNFUBDPNCMPHNFUB - MMBNB ੑ۱
UPLFOT UPLFO ੐߬٬ TFMG BUUFOUJPO ''// ୹۱ UPLFO TFMG BUUFOUJPO ''// "VUPSFHSFTTJWF%FDPEJOH

য়ט զॿ যٸ &04 ੑ۱ ୹۱ ҳܴ ೠ ੼
হח ݉਷ ҳܴ হח ੼ ೠ

୭੸ച$MPTFE2VFTUJPO

ਃ୒ ੄ب࠙ࢳ ҙ۲ࢿ ಣо ੿ࠁр ਋ࢶࣽਤ ࠗৈ Ә஗য х૑ ਬ೧੿ࠁ
х૑ ఐ࢝੿ࠁ ಿ૕ಣо

যڃҗݾਸજই೧ ӣߏਸજইೠ׮ ৉ࢎীҙब੉݆ইҴࢎदр੉ӝ׮۰૓׮

0QFO2VFTUJPO ઱য૓૕ޙী׹ਵ۽੸੺ೠޙ੢ਸ଺ই઻ 2যڃҗݾਸજই೧ ӣߏਸજইೠ׮ ৉ࢎীҙब੉݆ইҴࢎदр੉ӝ׮۰૓׮ rযڃҗݾਸજই೧ sۄח૕ޙীחr৉ࢎীҙब੉ ݆ইҴࢎदр੉ӝ׮۰૓׮sۄחޙ੢੉׹ਵ۽੸ ੺೤פ׮ ೐܁೐౟
୹۱

$MPTFE2VFTUJPO ೐܁೐౟ ୹۱ ઱য૓૕ޙী׹ਵ۽੸੺ೠޙ੢ਸ଺ই઻ э਷ഋधਵ۽׹ਸ೧઻ 2যڃҗݾਸજই೧ ӣߏਸજইೠ׮ ৉ࢎীҙब੉݆ইҴࢎदр੉ӝ׮۰૓׮

ࢿמ࠺Ү IUUQT DPNNVOJUZPQFOBJDPNUHQUP - UPLFOT - QFS - TFDPOE
- DPNQBSBCMF - UP - HQUUVSCP - EBUB - BOE - BOBMZTJTDPNQBSBUJWF - TUBUJTUJDT - UBCMF] IUUQT QMBUGPSNPQFOBJDPNUPLFOJ[FS] NT ష௾ࣻ 0QFO 2VFTUJPO $MPTFE 2VFTUJPO ష௾ࣻ दр NT ݽ؛ HQUP UPLFOTT .FBO 0QFO"*5PLFOJ[FS rযڃҗݾਸજই೧ sۄח૕ޙীח r৉ࢎীҙब੉݆ইҴࢎदр੉ӝ׮۰ ૓׮sۄחޙ੢੉׹ਵ۽੸੺೤פ׮ WT

--.௾Ӓܿ --.ੑ۱ --.୹۱ --.ղࠗ

--.ੑ۱ ੑ۱ UPLFOT UPLFO ੐߬٬ TFMG BUUFOUJPO ''// ୹۱
UPLFO TFMG BUUFOUJPO ''// "VUPSFHSFTTJWF%FDPEJOH

5PLFOJ[FS tഥࢎীب଱೮׮u <tഥu tࢎu tীu tبu t଱u t೮u t׮u tu>
tഥࢎীب଱೮׮u <tഥࢎu tীu tب଱೮׮u tu> DIBSBDUFS - CBTFE TVCXPSE - CBTFE tഥࢎীب଱೮׮u <tഥࢎীu tب଱೮׮u> XIJUFTQBDF

5PLFO&NCFEEJOH t஠஠য়ূఠపੋݢ౟חૌѩ׮u <t஠஠য়u tূఠపੋݢ౟u tחu tૌѩ׮u tu> j
j j j j

5PLFO&NCFEEJOH ૌѩ׮ ূఠపੋݢ౟ ஠஠য় ח ஠஠য় ૌѩ׮ ূఠపੋݢ౟ ח
j j j j

ੑ۱द௫झ য়ט যٸ զॿ о 5PLFO 5PLFO &NCFEEJOH
1PTJUJPOBM &ODPEJOH JE QPTJUJPO j j j j j j j j j j j j

୭੸ച5PLFOJ[FS

ష௼ա੉੷ ష௼ա੉੷୭੸ച IUUQT IVHHJOHGBDFDPMFBSOOMQ - DPVSTFDIBQUFS] IUUQT QMBUGPSNPQFOBJDPNUPLFOJ[FS
tউ֞ೞࣁਃu < > <tউu t֞ೞࣁਃu tu> tউ֞ೞࣁਃu < > <tইu tũu t֗u tżu tೞࣁਃu tu> (15(15 (15P

ࢿמ࠺Ү (15 5PLFOJ[FS (15P 5PLFOJ[FS ష௾ࣻ दр NT
ݽ؛HQUP UPLFOTT .FBO 0QFO"*5PLFOJ[FS JG LBLBP ח஠஠য়Ӓܛ੄ӝࣿ࠺੹ਸҕѐೞҊ ӝࣿ੸ࢿஂী؀೧ѐߊ੗ழޭפ౭৬ೣԋҕਬೞ ח஠஠য়؀಴ѐߊ੗ஶಌ۠झੑפ׮ ݽ؛݃׮ష௾ࢤࢿࣘب UPLFOTT ׮ܰ૑݅ ੉࠺Үীࢲחэ׮ۄҊо੿ IUUQT DPNNVOJUZPQFOBJDPNUHQUP - UPLFOT - QFS - TFDPOE - DPNQBSBCMF - UP - HQUUVSCP - EBUB - BOE - BOBMZTJTDPNQBSBUJWF - TUBUJTUJDT - UBCMF] IUUQT QMBUGPSNPQFOBJDPNUPLFOJ[FS] NT ష௾ࣻ

--.௾Ӓܿ --.ੑ۱ --.୹۱ --.ղࠗ

--.୹۱ ੑ۱ UPLFOT UPLFO ੐߬٬ TFMG BUUFOUJPO ''// ୹۱
UPLFO TFMG BUUFOUJPO ''// "VUPSFHSFTTJWF%FDPEJOH j ୭ઙIJEEFOTUBUF

୭ઙIJEEFOTUBUF ೖ੗ ࣻਭ ୭ઙIJEEFOTUBUF ۄݶ ӣ஖ ৬ ୭ઙIJEEFOTUBUF
j j j TFMG BUUFOUJPO ''// TFMG BUUFOUJPO ''//

-PHJUT ୭ઙIJEEFOTUBUF &NCFEEJOH.BUSJY
j -PHJUT ଵ ࣻਭ Ҋೱ ۄݶ ղੌ j ೖ੗ ଵ ࣻਭ Ҋೱ ۄݶ ղੌ j ೖ੗

ష௾୹۱ഛਯ ೐܁೐౟ ઱য૓૕ޙী׹ਵ۽੸੺ೠޙ੢ਸ଺ই઻ э਷ഋधਵ۽׹ਸ೧઻ 2যڃҗݾਸજই೧ ӣߏਸજইೠ׮
r җݾ զॿ j যڃ JOG JOG JOG JOG JOG JOG JOG -PHJUT

୭੸ച4USVDUVSFE0VUQVU

+40/0VUQVU UZQF.FNCFS \ OBNFTUSJOH BHFOVNCFS ^ \ OBNF.V[J BHF ^
\ OBNF"QFBDI BHF ^

+40/0VUQVU ೐܁೐౟ 0VUQVUB+40/PCKFDUUIBUDPOUBJOTUIF LFZTAOBNFABOEABHFA EFTDSJCJOHUIFj jGFX - TIPUFYBNQMFT

--. \ +40/0VUQVU t --. { 1SPNQU 1SPNQU { "
1SPNQU OBNF --. { "name": "Muzi", "age": 10 }

\ OBNF.V[J BHF ^ \ OBNF"QFBDI BHF ^ +40/0VUQVU UZQF.FNCFS
\ OBNFTUSJOH BHFOVNCFS ^

--. .V[J { "name": 4USVDUVSFE0VUQVU --. { "name": "Muzi",
"age": { "name": "Muzi", "age": 10 } 1SPNQU 1SPNQU { "name": "Muzi", "age": 10 }

ࢿמ࠺Ү 6OTUSVDUVSFE 4USVDUVSFE ష௾ࣻ दр NT
ݽ؛HQUP UPLFOTT .FBO 0QFO"*5PLFOJ[FS { "id": 12345, "name": "John Doe", "email": "[email protected]", "is_active": true } IUUQT DPNNVOJUZPQFOBJDPNUHQUP - UPLFOT - QFS - TFDPOE - DPNQBSBCMF - UP - HQUUVSCP - EBUB - BOE - BOBMZTJTDPNQBSBUJWF - TUBUJTUJDT - UBCMF] IUUQT QMBUGPSNPQFOBJDPNUPLFOJ[FS] NT ష௾ࣻ

0QFOݽ؛ 0VUMJOFT $MPTFEݽ؛ 0QFO"* ֙ਘ୹द
(FNJOJ ֙ਘ୹द 4USVDUVSFE0VUQVU IUUQT HJUIVCDPNEPUUYU - BJPVUMJOFT] IUUQT PQFOBJDPNJOEFYJOUSPEVDJOH - TUSVDUVSFE - PVUQVUT - JO - UIF - BQJ] IUUQT DMPVEHPPHMFDPNWFSUFY - BJHFOFSBUJWF - BJEPDTNVMUJNPEBM DPOUSPM - HFOFSBUFE - PVUQVU

--.௾Ӓܿ --.ੑ۱ --.୹۱ --.ղࠗ

5SBOTGPSNFS࠶۟ ੑ۱ UPLFOT UPLFO ੐߬٬ ୹۱ UPLFO TFMG BUUFOUJPO
''// TFMG BUUFOUJPO ''// "VUPSFHSFTTJWF%FDPEJOH

5SBOTGPSNFS࠶۟ 5PLFO &NCFEEJOH 5SBOTGPSNFS #MPDLT ⃗ E1 ⃗ E2 ⃗
E3 ⃗ E4 TFMG BUUFOUJPO ''// ⃗ E′ 1 TFMG BUUFOUJPO ''// ⃗ E′ 2 ⃗ E′ 3 ⃗ E′ 4 ⃗ E′ ′ ′ ′ ′ ′ ′ 1 ⃗ E′ ′ ′ ′ ′ ′ ′ 2 ⃗ E′ ′ ′ ′ ′ ′ ′ 3 ⃗ E′ ′ ′ ′ ′ ′ ′ 4 য়ט যٸ զॿ о

߹ݽনࢎҗܳੜۋ׮

"UUFOUJPO,FZ 2VFSZ 7BMVF 5PLFO &NCFEEJOH 5SBOTGPSNFS #MPDLT ⃗ E1 ⃗
E2 ⃗ E3 ⃗ E4 TFMG BUUFOUJPO ''// ⃗ E′ 1 TFMG BUUFOUJPO ''// ⃗ E′ 2 ⃗ E′ 3 ⃗ E′ 4 ⃗ E′ ′ ′ ′ ′ ′ ′ 1 ⃗ E′ ′ ′ ′ ′ ′ ′ 2 ⃗ E′ ′ ′ ′ ′ ′ ′ 3 ⃗ E′ ′ ′ ′ ′ ′ ′ 4 , 2 7 , 2 7 , 2 7 , 2 7 , 2 7 , 2 7 , 2 7 , 2 7 ߹ݽন ੜۋ׮ ࢎҗ ܳ

"UUFOUJPO,FZ 2VFSZ 7BMVF ⃗ E1 ⃗ E2 ⃗ E3 ⃗
E4 TFMG BUUFOUJPO ''// , 2 7 , 2 7 , 2 7 , 2 7 WQ WK WV ⃗ K1 ⃗ Q1 ⃗ V1 ⃗ E1 ⃗ E1 ⋅ WK ⃗ E1 ⋅ WQ ⃗ E1 ⋅ WV ߹ݽন ੜۋ׮ ࢎҗ ܳ

"UUFOUJPO,FZ 2VFSZ 7BMVF ⃗ E1 ⃗ E2 ⃗ E3 ⃗
E4 K1 Q1 V1 Q1 ⋅ K1 Q2 ⋅ K1 Q2 ⋅ K2 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V1 ⃗ V2 ⃗ V3 ⃗ V4 K2 Q2 V2 K3 Q3 V3 K4 Q4 V4 Q1 ⋅ K2 Q1 ⋅ K3 Q1 ⋅ K4 Q2 ⋅ K3 Q2 ⋅ K4 Q3 ⋅ K4 9 "UUFOUJPO-BZFS ⃗ E1 ⃗ E2 ⃗ E3 ⃗ E4 ߹ݽন ੜۋ׮ ࢎҗ ܳ

୭੸ച,7$BDIF

,7$BDIFࢎਊ9 9 "UUFOUJPO-BZFS ⃗ E1 K1 Q1 V1 Q1 ⋅
K1 ⃗ V1 ⃗ E1 ⃗ E2 Q2 ⋅ K1 Q2 ⋅ K2 ⃗ V2 K2 Q2 V2 ⃗ E2 ⃗ E3 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 ⃗ V3 K3 Q3 V3 ⃗ E3 ⃗ E4 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K4 Q4 V4 ⃗ E4 IUUQT NFEJVNDPN!KPBPMBHFTLW - DBDIJOH - FYQMBJOFE

,7$BDIFࢎਊ9 9 "UUFOUJPO-BZFS ⃗ E1 K1 Q1 V1 Q1 ⋅
K1 ⃗ V1 ⃗ E1 ⃗ E2 Q2 ⋅ K1 Q2 ⋅ K2 ⃗ V2 K2 Q2 V2 ⃗ E2 ⃗ E3 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 ⃗ V3 K3 Q3 V3 ⃗ E3 ⃗ E4 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K4 Q4 V4 ⃗ E4

,7$BDIFࢎਊ9 "UUFOUJPO ⃗ E1 K1 Q1 V1 ⃗ E1 ⃗
E2 K2 Q2 V2 ⃗ E2 ⃗ E3 K3 Q3 V3 ⃗ E3 ⃗ E4 K4 Q4 V4 ⃗ E4 'FFE - 'PSXBSE /FUXPSL ⃗ E′ 1 ⃗ E′ 2 ⃗ E′ 3 ⃗ E′ 4

,7$BDIFࢎਊ0 ⃗ V2 ⃗ V3 9 "UUFOUJPO-BZFS ⃗ V1 ⃗
E1 K1 Q1 V1 Q1 ⋅ K1 ⃗ E1 ⃗ E2 Q2 ⋅ K1 Q2 ⋅ K2 K2 Q2 V2 ⃗ E2 ⃗ E3 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 K3 Q3 V3 ⃗ E3 ⃗ E4 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K4 Q4 V4 ⃗ E4 IUUQT NFEJVNDPN!KPBPMBHFTLW - DBDIJOH - FYQMBJOFE ⃗ V1 ⃗ V2 ⃗ V3

,7$BDIFࢎਊ0 ⃗ V2 ⃗ V3 9 "UUFOUJPO-BZFS ⃗ V1 ⃗
V1 ⃗ V2 ⃗ V3 ⃗ E4 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K4 Q4 V4 ⃗ E4

,7$BDIFࢎਊ0 "UUFOUJPO ⃗ E4 K4 Q4 V4 ⃗ E4 'FFE
- 'PSXBSE /FUXPSL ⃗ E′ 4

ࢿמ࠺Ү ,7$BDIF 9 ,7$BDIF 0 दр TFD ড ড
ݽ؛NFUB - MMBNB-MBNBC - IB (152CJU2VBOUJ[BUJPO (16" ೐܁೐౟ష௾ࣻ ࢤࢿష௾ࣻ ߓ஖ࢎ੉ૉ )VHHJOH'BDF5SBOTGPSNFST generate(..., use_cache=False) generate(..., use_cache=True) IUUQT IVHHJOHGBDFDPNFUB - MMBNB-MBNBC - IG] IUUQT IVHHJOHGBDFDPEPDTUSBOTGPSNFST] IUUQT IVHHJOHGBDFDPCMPHPWFSWJFX - RVBOUJ[BUJPO - USBOTGPSNFSTHFOFSBUF - TQFFE

୭੸ച߽۳୊ܻ

߽۳୶ۿ য়טզॿоযٸ बबೡٺ ੷֘ݫ׏ח য়ט਷ޖटզ 1SPNQU 5PLFO(FOFSBUJPO ҳܴ ೠ
੼ হח ݉਷ ઁਭ श ߏ җ ػ੢ ଧѱ JG LBLBP ஶಌ۠झ ୐ ߣ૩ զ ஠஠য় ਢొ ਸ ࠊ &04 ೞט

,7$BDIFബਯച ,7$BDIF੄ݫݽܻ੼ਬ֫਺ ,7$BDIFബਯചद زद୶ۿद௫झࣻૐо ੹୓UISPVHIQVUೱ࢚

,7$BDIFബਯച बब ೡ ٺ &04 য়ט ਷ ޖट զ &04
஠஠য় ਢొ ਸ ࠊ &04 JG LBLBP &04 ݫݽܻ౵ಞച द௲झࢤࢿ੹޷ܻࢎਊೡݫݽܻ੼ਬ ష௾੉঴݃݅ఀࢤࢿؼ૑৘ஏࠛо

,7$BDIFബਯച बब ೡ ٺ &04 য়ט ਷ ޖट զ &04
஠஠য় ਢొ ਸ ࠊ &04 JG LBLBP &04 ݫݽܻ౵ಞച द௲झࢤࢿ੹޷ܻࢎਊೡݫݽܻ੼ਬ ష௾੉঴݃݅ఀࢤࢿؼ૑৘ஏࠛо ೠद௲झ੿ܻറ೧׼ҕрࢎਊࠁ੢উؽ

,7$BDIFബਯച য়ט ਷ ޖट զ &04 JG LBLBP &04
ݫݽܻ౵ಞച द௲झࢤࢿ੹޷ܻࢎਊೡݫݽܻ੼ਬ ష௾੉঴݃݅ఀࢤࢿؼ૑৘ஏࠛо ೠद௲झ੿ܻറ೧׼ҕрࢎਊࠁ੢উؽ JG LBLBP ীࢲ যڃ ߊ಴ ٜ ੉ ੓য &04

,7$BDIFബਯച ஠஠য় ਢొ ਸ ࠊ &04 JG LBLBP &04
W--. э਷௼ӝ࠶۾ਵ۽ա־Ҋ п࠶۾ীҊ੿ػࣻష௾, 7੷੢ п࠶۾਷োࣘػҕр೙ਃ9 बब ೡ ٺ &04 য়ט ਷ ޖट զ &04 #MPDL"MMPDBUPS ,XPO 8PPTVL FUBM&G fi DJFOUNFNPSZNBOBHFNFOUGPSMBSHFMBOHVBHFNPEFMTFSWJOHXJUIQBHFEBUUFOUJPO

ࢿמ࠺Ү ݽ؛015# (16/WJEJB"(# ӝઓदझమ W--. ୭؀୊ܻ۝ UPLFOTT
ড ড ୭؀ ߓ஖ࢎ੉ૉ ডѐ ডѐ IUUQT IVHHJOHGBDFDPGBDFCPPLPQUC] 'BTUFS5SBOTGPSNFS 0SDB] ,XPO 8PPTVL FUBM&G fi DJFOUNFNPSZNBOBHFNFOUGPSMBSHFMBOHVBHFNPEFMTFSWJOHXJUI QBHFEBUUFOUJPO

୭੸ച߽۳୊ܻ

--.୶ۿ੄ױ҅ "*ӝࣿ੉ࢎഥ৬҃ઁী޷஖חӛ੿੸ ࠗ੿੸ ৔ೱਸӐഋ੓ѱࢸݺೞҊ ੉ܳ؀࠺ೞӝਤ೧ пҴ੿ࠗ৬ӝস੉೧ঠೡ৉ೡী؀೧֤੄೧ ࠁইۄ "*ӝࣿ਷ബਯࢿೱ࢚җ࠺ਊ੺хীӝৈೞ חӛ੿੸ੋബҗо੓૑݅ زदীੌ੗ܻхࣗ
աࢎഥ੸ࠛಣ١ਸୡېೡਤ೷੉੓णפ׮п Ҵ੿ࠗח੉۞ೠ߸ചী؀਽ೞӝਤೠӏઁ৬ ૑ਗ੿଼ਸ݃۲೧ঠೞݴ ӝস਷ӝࣿਮܻ৬ ੋ۱੤Үਭী൨ॄঠ೤פ׮ 1SPNQU1SPDFTTJOHױ҅ 5PLFO(FOFSBUJPOױ҅

1SPNQU1SPDFTTJOHױ҅ 9 "UUFOUJPO-BZFS ⃗ E1 ⃗ E2 ⃗ E3 ⃗
E4 Q1 ⋅ K1 ⃗ V1 Q2 ⋅ K1 Q2 ⋅ K2 ⃗ V2 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 ⃗ V3 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K1 Q1 V1 K2 Q2 V2 K3 Q3 V3 K4 Q4 V4 ⃗ E1 ⃗ E2 ⃗ E3 ⃗ E4

5PLFO(FOFSBUJPOױ҅ ⃗ V5 ⃗ V6 9 "UUFOUJPO-BZFS ⃗ V1 ⃗
E5 Q5 ⋅ K4 Q5 ⋅ K5 K5 Q5 V5 ⃗ E5 ⃗ E6 Q6 ⋅ K4 Q6 ⋅ K5 Q6 ⋅ K6 K6 Q6 V6 ⃗ E6 ⃗ V1 ⃗ V5 ⃗ V4 Q5 ⋅ K1 j Q6 ⋅ K1 j

1SPNQU1SPDFTTJOH୊ܻ UPLFOHFOFSBUJPOߓ஖ݥ୶Ҋ୊ܻ W--. UPLFOHFOFSBUJPOߓ஖৬э੉୊ܻ 0SDB
UPLFOHFOFSBUJPOߓ஖оэ੉ו۰૗ QSPNQUQSPDFTTJOH࠙ೡ %FFQ4QFFE - 'BTU(FO UPLFOHFOFSBUJPOߓ஖৬࠺तೠ௼ӝ :V (ZFPOH - *O FUBM0SDB"EJTUSJCVUFETFSWJOHTZTUFNGPS\5SBOTGPSNFS - #BTFE^HFOFSBUJWFNPEFMT ] )PMNFT $POOPS FUBM%FFQTQFFE - GBTUHFO)JHI - UISPVHIQVUUFYUHFOFSBUJPO GPSMMNTWJBNJJBOEEFFQTQFFE - JOGFSFODF

ࢿמ࠺Ү UPLFOT୊ܻदр T ] )PMNFT $POOPS FUBM%FFQTQFFE - GBTUHFO)JHI
- UISPVHIQVUUFYUHFOFSBUJPOGPSMMNTWJBNJJBOEEFFQTQFFE - JOGFSFODF ݽ؛-MBNB# (16 "(# ೐܁೐౟ష௾ࣻ ࢤࢿష௾ࣻ ਃ୒۝RVFSJFTT W--. %FFQ4QFFE - 'BTU(FO ୊ܻ۝ UPLFOTT ড ড

୭੸ചQSF fi Yҕਬ

زੌೠQSF fi Y ೐܁೐౟ ࢎਊ੗оਃ୒ೠ੿ࠁীࢲࠛ௝ೞѢաਬ೧ೠղ ਊ਷ઁ৻ೞҊ ݺഛೞҊਬ੊ೠ੿ࠁܳઁҕೞࣁ ਃࠗ੸੺ೞѢաࠛ೙ਃೠࣁࠗࢎ೦਷Ѧ۞ղ Ҋ ೨ब੸ੋղਊ݅рѾೞҊ੉೧ೞӝऔѱࢸݺ
ೞࣁਃ಩۱੸ ର߹੸ ഇয়੸ ࠛߨ੸੉Ѣաࠛ ௝хਸ઴ࣻ੓חղਊ਷ݽفઁ৻ೞࣁਃ৘ܳ ٜয ੋઙ ࢿ߹ ઙҮী؀ೠର߹੸ੋj оա׮ۄ݃߄ࢎ ই੗ର஠ఋ౵ೞ ೐܁೐౟ ࢎਊ੗оਃ୒ೠ੿ࠁীࢲࠛ௝ೞѢաਬ೧ೠղ ਊ਷ઁ৻ೞҊ ݺഛೞҊਬ੊ೠ੿ࠁܳઁҕೞࣁ ਃࠗ੸੺ೞѢաࠛ೙ਃೠࣁࠗࢎ೦਷Ѧ۞ղ Ҋ ೨ब੸ੋղਊ݅рѾೞҊ੉೧ೞӝऔѱࢸݺ ೞࣁਃ಩۱੸ ର߹੸ ഇয়੸ ࠛߨ੸੉Ѣաࠛ ௝хਸ઴ࣻ੓חղਊ਷ݽفઁ৻ೞࣁਃ৘ܳ ٜয ੋઙ ࢿ߹ ઙҮী؀ೠର߹੸ੋj "#$%&'( )*+,-./01

.BTLFE"UUFOUJPO ⃗ V2 ⃗ V3 9 "UUFOUJPO-BZFS ⃗ V1 ⃗
E1 K1 Q1 V1 Q1 ⋅ K1 ⃗ E1 ⃗ E2 Q2 ⋅ K1 Q2 ⋅ K2 K2 Q2 V2 ⃗ E2 ⃗ E3 Q3 ⋅ K1 Q3 ⋅ K2 Q3 ⋅ K3 K3 Q3 V3 ⃗ E3 ⃗ E4 Q4 ⋅ K1 Q4 ⋅ K2 Q4 ⋅ K3 Q4 ⋅ K4 ⃗ V4 K4 Q4 V4 ⃗ E4 ⃗ V1 ⃗ V2 ⃗ V3 Q1 ⋅ K2 Q1 ⋅ K3 Q1 ⋅ K4 Q2 ⋅ K3 Q2 ⋅ K4 Q3 ⋅ K4

QSF fi Yҕਬ ࢲ۽׮ܲद௲झр,7நदҕਬ W--. $IVOL"UUFOUJPO
W--.֙ਘ (FNJOJ֙ਘ QSFWJFX $MBVEF֙ਘ CFUB 0QFO"*(15֙ਘ :F -V FUBM$IVOLBUUFOUJPO&G fi DJFOUTFMG - BUUFOUJPOXJUIQSF fi Y - BXBSFLWDBDIFBOEUXP - QIBTFQBSUJUJPO ] IUUQT HJUIVCDPNWMMN - QSPKFDUWMMNQVMM] IUUQT BJHPPHMFEFWHFNJOJ - BQJEPDTDBDIJOH] IUUQT XXXBOUISPQJDDPNOFXTQSPNQU - DBDIJOH] IUUQT PQFOBJDPNJOEFYBQJ - QSPNQU - DBDIJOH

QSF fi Yҕਬ from vllm import LLM llm = LLM(
..., enable_prefix_caching=True ) llm.generate(LONG_PROMPT + "AAA") llm.generate(LONG_PROMPT + “BBB") -PSFNJQTVNEPMPSTJU BNFU DPOTFDUFUVS BEJQJTDJOHFMJU6URVJT MPSFNNBTTjj """ -PSFNJQTVNEPMPSTJU BNFU DPOTFDUFUVS BEJQJTDJOHFMJU6URVJT MPSFNNBTTjj ###

ࢿמ࠺Ү ݽ؛$MBVEF4POOFU ஏ੿ӝળ 5JNFUP fi STUUPLFO
நदػ೐܁೐౟ٍ_ష௾ੋझ౟۟࣌ 1SF fi Yநद 9 1SF fi Yநद 0 UPLFODBDIFE QSPNQU T T UPLFODBDIFE QSPNQU T T IUUQT XXXBOUISPQJDDPNOFXTQSPNQU - DBDIJOH

݃ޖܻ

--. EFDPEFS - POMZUSBOTGPSNFS ҳઑ ੑ۱ ୹۱
ղࠗ 5SBOTGPSNFS#MPDL ୭੸ച 5PLFOJ[FS $MPTFE2VFTUJPO 4USVDUVSFE0VUQVU ,7$BDIF ߽۳୊ܻ ,7$BDIF౵ಞച৮ച ߽۳୊ܻ ߓ஖੹ۚ QSF fi Yҕਬ ߊ಴੿ܻ

ә߸ೞח"*࠙ঠ ࢜۽਍ӝࣿҗߑߨٜ੉॔ইઉա১ ҳઑ৬ਗܻܳঌইঠજ਷೒ۖಬ ۄ੉࠳۞ܻ بҳࢶఖоמ ҳઑ৬ਗܻܳঌইঠજ਷গ೒ܻா੉࣌ࢸ҅оמ
द௫झݽ؛݂࠙ঠীࢲ౟ےझನݠইఃఫ୊חೠزউ૑ߓ੸ੌѪ ౟ےझನݠইఃఫ୊ܳੜ౵ঈೞӝਤ೧֢۱઺ જ਷ࢸ҅ܳೞӝਤ೧֢۱઺ Ѿۿ

நܼఠಕܰࣗա

지연 시간 순삭! LLM 추론 구조와 효율적 애플리케이션 설계

지연 시간 순삭! LLM 추론 구조와 효율적 애플리케이션 설계

More Decks by kakao

Other Decks in Programming

Featured

Transcript