Upgrade to Pro — share decks privately, control downloads, hide ads and more …

밑바닥부터 시작하는 LLM 개발기

kakao
November 01, 2024

밑바닥부터 시작하는 LLM 개발기

#Kanana #LLM #pre-training #post-training

카카오의 자체 대규모 언어 모델(LLM) 개발 과정과 그에 따른 도전 과제, 해결 경험을 소개합니다.
데이터 수집과 전처리부터 시작해 LLM의 pre-training과 post-training을 거친 모델의 개발 과정 전반을 공유하고, 그 과정에서 마주한 주요 문제와 그 해결방안에 대해서도 살펴봅니다.

발표자 : mat.mul, kloud.off
mat.mul - Language Model Training팀의 mat이라고 합니다. 경쟁력 있는 LLM을 만들기위해 노력하고 있습니다.
kloud.off - Langauge Model Alignment팀의 kloud입니다. 말 잘듣는 LLM을 만들기위해 다양한 연구/개발을 수행하고 있습니다.

kakao

November 01, 2024
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. 1SF - USBJOJOHࢿמ QSF - USBJOJOH ױ҅о ՘դ ,BOBOB /BOP

     &TTFODF੄ ࢿמ਷ ࠺तೠ ௼ӝ੄ MMBNBC  MMBNBC؀࠺ೠҴয ࣻ೟ ௏٘ҙ۲߮஖݃௼ীࢲજ਷ࢿמਸഛࠁೣ
  2. 1PTU - USBJOJOHࢿמ QPTU - USBJOJOH ױ҅о ՘դ ,BOBOB /BOP

     &TTFODF੄ ࢿמ਷ ࠺तೠ ௼ӝ੄ MMBNBC  MMBNBC؀࠺ೠҴয৬௏٘ҙ۲߮஖݃௼੄ࢿמ੉਋ࣁೞݴ Ӓ৻ীח࠺Үೡ݅ೠࣻળ੐
  3. %BUBTUSBUFHZ 5о۝੄ష௾ਸ೟णীࢎਊೞחѪীޙઁоহחۄ੉ࣃझ FH BQBDIF DD - CZ١ ী೧׼ೞח ݈ޡ஖ٜ۽ࠗఠࣻ૘ೞҊ ੉ܳHPBM߹۽࠙ܨೣ

    (PBM ೟णীࢎਊоמ 
 ৘द ೟णীࢎਊࠛоמ 
 ৘द DPNNPOTFOTF HFOFSBMLOPXMFEHF ݽف੄݈ޡ஖ 
 Ҵ݀Ҵযਗ ੷੘ӂ੉੓חनޙӝࢎ TUFN ؀ӏݽҳݒبࢲӝ߈ೠҴয݈ޡ ஖ؘ੉ఠীࢲ45&.ҙ۲ղਊٜ 
 "*)VC LDJ֤ޙٜ DPEF QFSNJTTJWFMJDFOTFী೧׼ೞ ௏٘ޡ஖ٜ OPMJDFOTF੉Ѣա QFSNJTTJWFMJDFOTF оইצ௏٘ޡ஖ٜ XJLJQFEJB ೠҴযਤఃೖ٣ই աޖਤః
  4. %BUBTUSBUFHZ UXP - TUBHFEQSF - USBJOJOHਸҊ۰ೞৈ HPBM੉DPNNPOTFOTFHFOFSBMLOPXMFEHFੋೠҴয  
 ৔য݈ޡ஖ীRVBMJUZҙ۲੼ࣻ

    FH FEVDBUJPOBMTDPSF ஏ੿ਸदبೞҊ੗ೞ৓਺ Below is an extract from a web page. Evaluate whether the page has a high educational value and could be useful in an educational setting for teaching from primary school to grade school levels using the additive 5-point scoring system described below. Points are accumulated based on the satisfaction of each criterion: - Add 1 point if the extract provides some basic information relevant to educational topics, even if it includes some irrelevant or non-academic content like advertisements and promotional material. - Add another point if the extract addresses certain elements pertinent to education but does not align closely with educational standards. It might mix educational content with non-educational material, offering a superficial overview of potentially useful topics, or presenting information in a disorganized manner and incoherent writing style. - Award a third point if the extract is appropriate for educational use and introduces key concepts relevant to school curricula. It is coherent though it may not be comprehensive or could include some extraneous information. It may resemble an introductory section of a textbook or a basic tutorial that is suitable for learning but has notable limitations like treating concepts that are too complex for grade school students. - Grant a fourth point if the extract highly relevant and beneficial for educational purposes for a level not higher than grade school, exhibiting a clear and consistent writing style. It could be similar to a chapter from a textbook or a tutorial, offering substantial educational content, including exercises and solutions, with minimal irrelevant information, and the concepts aren't too advanced for grade school students. The content is coherent, focused, and valuable for structured learning. - Bestow a fifth point if the extract is outstanding in its educational value, perfectly suited for teaching either at primary school or grade school. It follows detailed reasoning, the writing style is easy to follow and offers profound and thorough insights into the subject matter, devoid of any non-educational or complex content. The extract: <EXAMPLE>. After examining the extract: - Briefly justify your total score, up to 100 words. - Conclude with the score using the format: "Educational score: <total points>"  5IF'JOF8FC%BUBTFUT%FDBOUJOHUIF8FCGPSUIF'JOFTU5FYU%BUBBU4DBMF 
  5. %BUBTUSBUFHZ ৔যী ؀೧ࢲח TUBHFҗ TUBHFী ೟ण ؘ੉ఠܳ ҳࢿೞӝਤ೧  %BUB$PNQ-.

    োҳীࢲ ࢎਊೞח 'BTU5FYUӝ߈੄RVBMJUZҙ۲ݽ؛ਸഝਊೣ  %BUB$PNQ - -.*OTFBSDIPGUIFOFYUHFOFSBUJPOPGUSBJOJOHTFUTGPSMBOHVBHFNPEFMT 'JMUFS $03& &95&/%&% 3F fi OFE8FCSFQSPEVDUJPO   5PQCZ1BHFSBOL   4FN%FEVQ   $MBTTJ fi FSPO#(&GFBUVSFT   "TL--.   1FSQMFYJUZ fi MUFSJOH   5PQ - LBWFSBHFMPHJUT   GBTU5FYU0) &-*   Table 4: Quality filtering comparison (1B-1x scale). We evaluate various choices for 
 model-based quality filters. Training a fastText classifier for filtering performs best. %BUBTFU 5ISFT IPME $03& ..-6 &95&/%&% 0) &-*     8JLJQFEJB     0QFO8FC5FYU     (15"QQSPY     0) &-*     0) &-*     Table 5: fastText ablations (7B-1x scale). We ablate choices for the positive data (top) and threshold (bottom). 'Dataset' is positive set for fastText, while the negatives are randomly sampled from RefinedWeb. 'Threshold' is the percentile used for filtering based on fastText scores. "GPT-3 Approx" refers to a uniform mix of Wikipedia, OpenWebText2, and RPJ Books, as in [30] 
  6. %BUBTUSBUFHZ HPBM੉XJLJQFEJBੋ݈ޡ஖ٜ੄҃਋TUBHFীࢲחೠҴয ৔যܳನೣೠѐ঱যী؀ೠXJLJQFEJBܳ 
 ࢎਊೞҊ TUBHF੄҃਋ীחೠҴয ৔যܳ઺बਵ۽ҳࢿೣ 3FHJPOT -BOHVBHFT 8FTUFSO&VSPQF

    (FSNBO 'SFODI 4QBOJTI 1PSUVHVFTF *UBMJBO %VUDI &BTUFSO$FOUSBM&VSPQF 3VTTJBO $[FDI 1PMJTI .JEEMF&BTU "SBCJD 1FSTJBO )FCSFX 5VSLJTI &BTUFSO"TJB +BQBOFTF $IJOFTF 4PVUI - &BTUFSO"TJB 7JFUOBNFTF 5IJB *OEPOFTJBO .BMBZ -BP #VSNFTF $FCVBOP ,INFS 5BHBMPH 4PVUIFSO"TJB )JOEJ #FOHBMJ 6SEV
  7. %BUBTUSBUFHZ ҕѐغয੓חਤఃೖ٣ই݈ޡ஖੄҃਋ࣻध୊ܻীޙઁо׮ࣻઓ੤ೣਸഛੋ TUBHFীࢲ઱۽૘઺ೞח 
 ೠҴয ৔যਤఃೖ٣ইীೠ೧ࢲ૒੽੹୊ܻܳࣻ೯ೞৈࢎਊೣ ߊࢤਗੋ ഥ੹ઝ಴҅ ഥ੹ઝ಴҅ ઝ಴҅Y

    Z [৬ઝ಴҅Y Z [ਸࠁ੗فઝ಴҅੄ਗ੼਷э׮пп੄҃਋ী؀ ೧߭ఠ਷فઝ಴҅ীࢲ׮਺җэ੉಴दػ׮  Y Z [ઝ಴  Y Z [ઝ಴҅ ߭ఠ੄ղ੸ਸ੉ਊ೧Y Z [ܳ      ਵ۽಴അೡࣻ੓׮ղ੸੄ߑߨ਷׮਺ җэ׮    ਵ۽಴അغחѪਸഛੋೡࣻ੓׮ ߊࢤਗੋ ഥ੹ઝ಴҅ ഥ੹ઝ಴҅ ઝ಴҅Y Z [৬ઝ಴҅Y Z [ਸࠁ੗فઝ಴҅੄ਗ੼਷э׮пп੄҃਋ী؀ ೧߭ఠ=NBUICG\S^਷فઝ಴҅ীࢲ׮਺җэ੉಴दػ׮ =NBUICG\S^Y=IBUY Z=IBUZ [=IBU[ Y Z [ઝ಴ =NBUICG\S^Y=IBUY Z=IBUZ [=IBU[ Y Z [ઝ಴҅  ߭ఠ੄ղ੸ਸ੉ਊ೧Y Z [ܳ =Y =IBUY =IBUY  =Z =IBUZ  =IBUZ  =[ =IBU[ =IBU[ ਵ۽಴അೡࣻ੓׮ղ੸੄ߑߨ਷׮਺ җэ׮ =NBUICG\S^\=IBUY^Y Y=IBUY Z=IBUZ [=IBU[ =IBUY  Y =IBUY=IBUY  Z =IBUZ=IBUY  [ =IBU[=IBUY  =NBUICG\S^\=IBUZ^Z Y=IBUY Z=IBUZ [=IBU[ =IBUZ  Y =IBUY=IBUZ  Z =IBUZ=IBUZ  [ =IBU[=IBUZ  =NBUICG\S^\=IBU[^[ Y=IBUY Z=IBUZ [=IBU[ =IBU[  Y =IBUY=IBU[  Z =IBUZ=IBU[  [ =IBU[=IBU[  ਵ۽಴അغחѪਸഛੋೡࣻ੓׮  ௏ܻৢܻബҗਤఃߔҗ ਋ܻݽف੄ߔҗࢎ੹  0VST IUUQT IVHHJOHGBDFDPEBUBTFUTXJLJNFEJBXJLJQFEJB
  8. ,BOBOB 
 &TTFODF 5SBJOJOHTUSBUFHZ ,BOBOB &TTFODFܳ ݢ੷ ٜ݅Ҋ  Ӕې

    ഝߊೞѱ োҳغҊ੓ח QSVOJOH  EJTUJMMBUJPO ӝߨਸ ੸ਊೞৈ ,BOBOB/BOPܳ೟णೞח੹ۚਸ଻ఖೣ ,BOBOB /BOP TUVEFOU 1SVOJOH %JTUJMMBUJPO 
 TUBHFEBUB SBOEPNJOJU 5XPTUBHFEQSF - USBJOJOH 
 TUBHF TUBHFEBUB
  9. 5SBJOJOHTUSBUFHZ ,BOBOB&TTFODF੄ҳઑחࢎप࢚಴ળੋMMBNB৬ਬࢎೠҳઑ۽ࢶ੿ ೟णߑध਷%FFL4FFL--.ীࢲ 
 ઁदೠߑߨۿ FH TDBMJOHMBX NVMUJ - TUFQMFBSOJOHSBUFTDIFEVMFS١

    ٜਸࢎਊೣ  %FFQ4FFL--.4DBMJOH0QFO - 4PVSDF-BOHVBHF.PEFMTXJUI-POHUFSNJTN  5JFFNCFEEJOH Y $POUFYUMFOHUI L )JEEFOMBZFST  "UUFOUJPOIFBET  ,FZWBMVFIFBET 
  10. 5SBJOJOHTUSBUFHZ TUBHF ೟ण੄ HPBM ߹ EJTUSJCVUJPO ఐ࢝ਸ ਤ೧ # о۝੄

    UPLFOਵ۽ BCMBUJPOਸ ૓೯  ೠҴয CFODINBSLੋLNNMV IBFSBF੄੼ࣻ৬৔যCFODINBSLੋNNMV੄੼ࣻ੄ಣӐਸ୭؀ചೞҊ੗ೣ  ,..-6.FBTVSJOH.BTTJWF.VMUJUBTL-BOHVBHF6OEFSTUBOEJOHJO,PSFBO  )"& - 3"&#FODI&WBMVBUJPOPG,PSFBO,OPXMFEHFJO-BOHVBHF.PEFMT )PXFWFS JUJTDSVDJBMUPOPUFUIBUXIJMFPG,..-6TDPOUFOUJTDVMUVSBMMZ TQFDJ fi D  UIF SFNBJOEFS CSPBEMZ BTTFTTFT B MBOHVBHF NPEFMT HFOFSBM LOPXMFEHF  JODMVEJOH BSFBT TVDI BT NBUIFNBUJDT  XIJDI BSF VOJWFSTBMMZ BQQMJDBCMFBOEOPUJFEUPBOZQBSUJDVMBSDPVOUSZTLOPXMFEHFCBTF 
  11. 5SBJOJOHTUSBUFHZ TUBHF੄BCMBUJPOप೷਷׮਺җэ੉૓೯ೣ  BCMBUJPOCBTFMJOFਵ۽HPBM߹ࣻ૘ػUPLFOনীӝ߈ೠEJTUSJCVUJPOਸࢎਊೞח҃਋  BCMBUJPOHPBM੉DPNNPOTFOTFHFOFSBMLOPXMFEHFੋೠҴযUPLFOٜਸ੺߈݅ 
 FQPDIT ࢎਊೞח҃਋ 

    BCMBUJPOHPBM੉DPNNPOTFOTFHFOFSBMLOPXMFEHFੋೠҴযUPLFOٜਸNVMUJ - FQPDI 
 FQPDIT ਵ۽ࢎਊೞח҃਋  BCMBUJPOHPBM੉TUFNੋ৔যUPLFOٜਸNVMUJ - FQPDI FQPDIT ਵ۽ࢎਊೞח҃਋
  12. 5SBJOJOHTUSBUFHZ ,BOBOB&TTFODF੄TUBHFQSF - USBJOJOH 5 ੉ઙܐػറ੄߮஖݃௼ࢿמ਷ইې৬э਺ 
 ೠҴযҙ۲߮஖݃௼ীࢲח࢚؀੸ਵ۽֫਷ࢿמਸӝ۾ ࣻ೟җ௏٘ҙ۲߮஖݃௼ীࢲחৌਤܳࠁ੐ NNMV

    
 BDD TIPUT LNNMV 
 FN TIPUT IBFSBF 
 BDD@OPSN TIPUT HTNL 
 FN@TUSJDU TIPUT IVNBOFWBM 
 QBTT! TIPUT NCCQ 
 QBTT! TIPUT TUBHF       MMBNBC      
  13. 5SBJOJOHTUSBUFHZ TUBHFBCMBUJPOਸ૓೯ೞחҗ੿ীࢲ ইې৬э਷ղਊਸഛੋೡࣻ੓঻਺  TUBHF੄HPBM߹UPLFOEJTUSJCVUJPOਸਬ૑ೞҊ EBUB੄RVBMJUZ݅ѐࢶ೧بࢿמೱ࢚ؽ  ੹୓੸ੋHPBM߹UPLFOEJTUSJCVUJPOਸઑ੿೧ঠೡ೙ਃо੓਺  TUFNؘ੉ఠ੄࠺઺ਸৢܻחѪ੉LNNMV੼࢚ࣻथীب਑੉ؽ

     ࢚؀੸ਵ۽IJHIRVBMJUZEBUBܳࢎਊೞח҃਋੉޲۽ MFBSOJOHSBUFܳ੸੺൤ࢎਊೞחѪ੉઺ਃೣ  ੌ੿੿ب੄DPEF NBUIҙ۲JOTUSVDUJPOEBUBܳ୶оೞחѪ ౠ൤NBUIҙ۲CFODINBSLࢿמ 
 ૐ؀ীب਑੉ؽ
  14. 5SBJOJOHTUSBUFHZ  BCMBUJPOTUBHF੄ؘ੉ఠ৬HPBM߹EJTUSJCVUJPOਸ TUBHFীࢲਬ૑ೠ଻۽೟ण  BCMBUJPOTUBHF੄HPBM߹EJTUSJCVUJPOਸਬ૑ ؘ੉ఠ ੄RVBMJUZܳѐࢶ  BCMBUJPOBCMBUJPOਸӝળਵ۽HPBM߹EJTUSJCVUJPOਸ

    Ӑ١ೞѱೣ  BCMBUJPOBCMBUJPOܳӝળীࢲTUFNҗDPEF੄࠺઺ਸ ৢܿ  BCMBUJPOBCMBUJPOࠁ׮ખ؊௾MFBSOJOHSBUFࢎਊ  BCMBUJPOBCMBUJPOীTGUؘ੉ఠ୶о
  15. 5SBJOJOHTUSBUFHZ UXP - TUBHFEQSFUSBJOJOHҗ࠺Үप೷ਸా೧ MMBNBC؀࠺޷݅੄೟णؘ੉ఠ݅ࢎਊೞҊب 
 ৈ۞߮஖݃௼ীࢲ࠺Үೡ݅ೞѢա ؊જ਷ࢿמਸQSF - USBJOJOHױ҅ীࢲ׳ࢿೣ

    %BUB$PNQBSFUPQSJPSWFSTJPOTPG-MBNB 5PVWSPOFUBM B C XFJNQSPWFE UIFRVBOUJUZBOERVBMJUZPGUIFEBUBXFVTFGPSQSF - USBJOJOHBOEQPTU - USBJOJOH 5IFTFJNQSPWFNFOUTJODMVEFUIFEFWFMPQNFOUPGNPSFDBSFGVMQSF - QSPDFTTJOH BOE DVSBUJPO QJQFMJOFT GPS QSF - USBJOJOH EBUB BOE UIF EFWFMPQNFOU PG NPSF SJHPSPVTRVBMJUZBTTVSBODFBOE fi MUFSJOHBQQSPBDIFTGPSQPTU - USBJOJOHEBUB8F QSF - USBJO-MBNBPODPSQVTPGBCPVU5NVMUJMJOHVBMUPLFOT DPNQBSFUP5 UPLFOTGPS-MBNB  5IF-MBNB)FSEPG.PEFMT 
  16. 5SBJOJOHTUSBUFHZ NNMV 
 BDD TIPUT LNNMV 
 FN TIPUT IBFSBF

    
 BDD@OPSN TIPUT HTNL 
 FN@TUSJDU TIPUT IVNBOFWBM 
 QBTT! TIPUT NCCQ 
 QBTT! TIPUT BWFSBHF ,BOBOB&TTFODF        MMBNBC        HFNNBC        RXFOC        NJTUSBMC - W        .JTUSBM#W VOUZJOH GVMM NPEFM JOQVU - FNCFE PVUQVU - FNCFE
  17. 5SBJOJOHTUSBUFHZ ,BOBOB/BOP੄҃਋MMBNB৬ਬࢎೠҳઑܳزੌೞѱ଻ఖೞҊ ,BOBOB&TTFODFী੸ਊೠزੌೠ 
 ۨदೖܳӝ߈ਵ۽GSPNTDSBUDI۽೟णਸ૓೯ ࠺तೠ௼ӝ੄MMBNBC࢚ܳഥೞחࢿמਸഛੋೣ 5JFFNCFEEJOH Y $POUFYUMFOHUI L

    )JEEFOMBZFST  "UUFOUJPOIFBET  ,FZWBMVFIFBET  NNMV 
 BDD TIPUT LNNMV@EJSFDU 
 FN TIPUT IBFSBF 
 BDD@OPSN TIPUT HTNL 
 FN@TUSJDU TIPUT IVNBOFWBM 
 QBTT! TIPUT NCCQ 
 QBTT! TIPUT BWFSBHF GSPNTDSBUDI        MMBNBC       
  18. 5SBJOJOHTUSBUFHZ ,BOBOB&TTFODF۽ࠗఠTUBHFীഝਊೠؘ੉ఠ 5 ܳഝਊೞৈQSVOJOHEJTUJMMBUJPOਸ 
 ੸ਊೠ҃਋ ݽٚCFODINBSLীࢲGSPNTDSBUDIࠁ׮ࢿמ੉જ਺ਸഛੋೣ NNMV 
 BDD

    TIPUT LNNMV 
 FN TIPUT IBFSBF 
 BDD@OPSN TIPUT HTNL 
 FN@TUSJDU TIPUT IVNBOFWBM 
 QBTT! TIPUT NCCQ 
 QBTT! TIPUT BWFSBHF GSPNTDSBUDI        QSVOJOHEJTUJMMBUJPO        MMBNBC       
  19. 5SBJOJOHTUSBUFHZ NNMV 
 BDD TIPUT LNNMV 
 FN TIPUT IBFSBF

    
 BDD@OPSN TIPUT HTNL 
 FN@TUSJDU TIPUT IVNBOFWBM 
 QBTT! TIPUT NCCQ 
 QBTT! TIPUT BWFSBHF ,BOBOB/BOP        MMBNBC        HFNNBC        RXFOC        NJOJUSPOC - CBTF       
  20. 'VSUIFSXPSL ,BOBOB&TTFODF৬/BOPܳ೟णೞݶࢲ ౠ൤ࣻ஖҅࢑җҙ۲ػTUFN݈ޡ஖ ؊ࠛয௏٘ҙ۲݈ޡ஖о 
 ࠗ઒ೣਸוՙҊ അ੤ҙ۲ؘ੉ఠܳѐࢶೞҊ੓਺  %FFQ4FFL -

    $PEFS8IFOUIF-BSHF-BOHVBHF.PEFM.FFUT1SPHSBNNJOH5IF3JTFPG$PEF*OUFMMJHFODF  %FFQ4FFL.BUI1VTIJOHUIF-JNJUTPG.BUIFNBUJDBM3FBTPOJOHJO0QFO-BOHVBHF.PEFMT  
  21. 1SFMJNJOBSZSFDJQF 4VQFSWJTFE'JOF - 5VOJOH  VTFS੄QSPNQU JOTUSVDUJPO ۽ࠗఠӒীݏ חSFTQPOTF BOTXFS

    ܳ૒੽ࢤࢿೞب۾೟ण 1SFGFSFODFCBTFE-FBSOJOH QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ղ੉ܴ਷஠աաঠ j ղ੉ܴ਷஠աաঠ %BUB --.
  22. 1SFMJNJOBSZSFDJQF 4VQFSWJTFE'JOF - 5VOJOH  VTFS੄QSPNQU JOTUSVDUJPO ۽ࠗఠӒীݏ חSFTQPOTF BOTXFS

    ܳ૒੽ࢤࢿೞب۾೟ण 1SFGFSFODFCBTFE-FBSOJOH  VTFS੄QSPNQU JOTUSVDUJPO җӒী࢚਽ೞ חৈ۞ѐ੄SFTQPOTFT BOTXFST ܳా೧؊ જ਷SFTQPOTFܳъചೞҊউજ਷SFTQPOTF ܳডചೞחߑೱਵ۽೟ण QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ղ੉ܴ਷஠աաঠ j ղ੉ܴ਷஠աաঠ %BUB --. QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ઁ੉ܴ਷஠աաੑפ׮👍 j %BUB SFTQPOTF Zs ա ஠աա👎 3FTQPOTF¬ --. 3FTQPOTF­ SFTQPOTF Zs ա ஠աա👎
  23. 0VS3FTVMUT &TTFODFNPEFM *OTUSVDUNPEFM -PHJD,PS 
 ೠҴযੌ߈ .BSV 
 ੗୓ೠҴয .5

    - #FODI ৔যੌ߈ "SFOB - IBSE 
 ৔যҊә *G&WBM 
 ઑѤ૑दࣻ೯ )VNBO&WBM ௏٘ (4.L ࣻ೟ MMBNBC        RXFOC        QIJTNBMM        NJTUSBMC - W        ,BOBOB&TTFODF        ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ
  24. 0VS3FTVMUT ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ -PHJD,PS .BSV .5 - #FODI "SFOB -

    IBSE JG&WBM )VNBO&WBM (4., LBOBOB - FTTFODF - JOTUSVDU MMBNBC - JOTUSVDU RXFOC - JOTUSVDU QIJTNBMML - JOTUSVDU NJTUSBMC - JOTUSVDU - W ೠҴযੌ߈ ੗୓ೠҴয ৔যੌ߈ ৔যҊә ઑѤ૑दࣻ೯ ௏٘ ࣻ೟
  25. 0VS3FTVMUT ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ ೠҴযੌ߈ ੗୓ೠҴয ৔যੌ߈ ৔যҊә ઑѤ૑दࣻ೯ ௏٘ ࣻ೟

    -PHJD,PS .BSV .5 - #FODI "SFOB - IBSE JG&WBM )VNBO&WBM (4., LBOBOB - FTTFODF - JOTUSVDU MMBNBC - JOTUSVDU RXFOC - JOTUSVDU QIJTNBMML - JOTUSVDU NJTUSBMC - JOTUSVDU - W
  26. 0VS3FTVMUT ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ ೠҴযੌ߈ ੗୓ೠҴয ৔যੌ߈ ৔যҊә ઑѤ૑दࣻ೯ ௏٘ ࣻ೟

    -PHJD,PS .BSV .5 - #FODI "SFOB - IBSE JG&WBM )VNBO&WBM (4., LBOBOB - FTTFODF - JOTUSVDU MMBNBC - JOTUSVDU RXFOC - JOTUSVDU QIJTNBMML - JOTUSVDU NJTUSBMC - JOTUSVDU - W
  27. 0VS3FTVMUT ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ ೠҴযੌ߈ ੗୓ೠҴয ৔যੌ߈ ৔যҊә ઑѤ૑दࣻ೯ ௏٘ ࣻ೟

    -PHJD,PS .BSV .5 - #FODI "SFOB - IBSE JG&WBM )VNBO&WBM (4., LBOBOB - FTTFODF - JOTUSVDU MMBNBC - JOTUSVDU RXFOC - JOTUSVDU QIJTNBMML - JOTUSVDU NJTUSBMC - JOTUSVDU - W
  28. 0VS3FTVMUT /BOPNPEFM *OTUSVDUNPEFM -PHJD,PS 
 ೠҴযੌ߈ .BSV 
 ੗୓ೠҴয .5

    - #FODI ৔যੌ߈ "SFOB - IBSE 
 ৔যҊә *G&WBM 
 ઑѤ૑दࣻ೯ )VNBO&WBM ௏٘ (4.L ࣻ೟ MMBNBC        RXFOC        QIJNJOJ        HFNNBC        ,BOBOB/BOP        ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ
  29. 0VS3FTVMUT /BOPNPEFM *OTUSVDUNPEFM -PHJD,PS 
 ೠҴযੌ߈ .BSV 
 ੗୓ೠҴয .5

    - #FODI ৔যੌ߈ "SFOB - IBSE 
 ৔যҊә *G&WBM 
 ઑѤ૑दࣻ೯ )VNBO&WBM ௏٘ (4.L ࣻ೟ MMBNBC        RXFOC        QIJNJOJ        HFNNBC        ,BOBOB/BOP        ഥࣻ೯റಣӐ ੗୓ಣо೐ۨ੐ਕ௼ࢎਊ
  30. ࢎۈ੉ ੋਤ੸ਵ۽ؘ݅ٚ੉ఠܳਤೠؘ੉ఠ ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF ੿ࠁ࠺؀ட߂ഥ҅੿ࠁ੄੿ࠁਤ೷ਸҊ۰ೞ ৈഥ҅хࢎ੄೙ਃࢿਸࢸݺೞदয় j хࢎੋ੉ࣻ೯ೞחഥ҅хࢎ੄Ҋਬೠ҅ܳ

    о૑ӝࣿೞदয় j хࢎੋ੉ഥ҅хࢎӝળ੉ਃҳೞחਃҳࢎ೦ ਵ۽ࠗఠ੉ఎೡࣻ੓ח࢚ടਸࢲࣿೞदয় j ఃਤ ߄աա ࢎҗ੄ࢿ࠙ী؀೧ঌ۰઻ j ੗ކী؀೧ࢲبঌ۰઻ j
  31. ࢎۈ੉पઁ۽ࢲ࠺झܳ੉ਊೞ٠ؘ݅ٚ੉ఠ ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF ৔য૕׹ਸ೧ঠغחؘ ੌױইېоղୡউ੉ঠ਋ࢶ ਷৔য಴അਸо׮ٞয઻<j઺ۚj> j ҡଳӟೠؘ

    ୐ߣ૩ޙ੢਷HSBEJFOU EJTUSJCVUJPOী৔ೱਸળ׮חஏݶীࢲց੄઱੢ ੉ݏ׮ۄחڷਵ۽ॳ۰Ҋ೮য j ইפইפHSBEJFOUEJTUSJCVUJPOী৔ೱਸળ׮ח ஏݶਸࢤпೠ׮ݶযו੿بݏח੉ঠӝ׮ۄחࣻળ JOUIBUਸݔ୊਺ীӒ۠੄޷۽ॳ۰Ҋ੄بೠѢ੐ j QZTQBSLীࢲQMPUMZۄ੉࠳۞ܻࢎਊ೧ࢲ IJTUPHSBNӒܻח౵੉ॆ௏٘ j KVQZUFSOPUFCPPLীࢲೡۄݶ j ੉ۧѱ೧بইޖѪبউࠁ੉חؘ j
  32. ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF MPX - RVBM IJHI - RVBM

    &OHMJTI ,PSFBO MPX - RVBM IJHI - RVBM MPX - RVBM IJHI - RVBM MPX - RVBM IJHI - RVBM $PEF .BUI SFTQPOTFBOTXFS੄௬ܻ౭ח઺ਃೞ׮ જ਷௬ܻ౭੄׹߸ਵ۽೟णೠݽ؛੉Ӓۧ૑ঋ਷ݽ؛ী࠺೧ࢿמ੉֫׮ 1FSGPSNBODF
  33. ࢲ۽׮ܲبݫੋ FH ௏٘ ࣻ೟ ઑѤഋध١ ؘ੉ఠࣇ਷пӝҳ୷੉೙ਃೞ׮ пبݫੋী೧׼ೞחؘ੉ఠࣇ੉੓যঠҙ۲מ۱੉ߊഅػ׮ пبݫੋрীחਬ੄޷ೠࢿמрࢼ੉ߊࢤೞ૑ঋח׮ DPOW DPOW

    DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM $PEF&WBM .BUI&WBM *'&WBM 1FSGPSNBODF ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF
  34. ࢲ۽׮ܲبݫੋ FH ௏٘ ࣻ೟ ઑѤഋध١ ؘ੉ఠࣇ਷пӝҳ୷੉೙ਃೞ׮ пبݫੋী೧׼ೞחؘ੉ఠࣇ੉੓যঠҙ۲מ۱੉ߊഅػ׮ пبݫੋрীחਬ੄޷ೠࢿמрࢼ੉ߊࢤೞ૑ঋח׮ DPOW DPOW

    DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM $PEF&WBM .BUI&WBM *'&WBM 1FSGPSNBODF ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF
  35. ࢲ۽׮ܲبݫੋ FH ௏٘ ࣻ೟ ઑѤഋध١ ؘ੉ఠࣇ਷пӝҳ୷੉೙ਃೞ׮ пبݫੋী೧׼ೞחؘ੉ఠࣇ੉੓যঠҙ۲מ۱੉ߊഅػ׮ пبݫੋрীחਬ੄޷ೠࢿמрࢼ੉ߊࢤೞ૑ঋח׮ DPOW DPOW

    DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM $PEF&WBM .BUI&WBM *'&WBM 1FSGPSNBODF ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF
  36. ࢲ۽׮ܲبݫੋ FH ௏٘ ࣻ೟ ઑѤഋध١ ؘ੉ఠࣇ਷пӝҳ୷੉೙ਃೞ׮ пبݫੋী೧׼ೞחؘ੉ఠࣇ੉੓যঠҙ۲מ۱੉ߊഅػ׮ пبݫੋрীחਬ੄޷ೠࢿמрࢼ੉ߊࢤೞ૑ঋח׮ DPOW DPOW

    DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM DPOW DPOW DPEF DPOW N BUI DPOW *' "MM $PEF&WBM .BUI&WBM *'&WBM 1FSGPSNBODF ,FZ-FTTPOTGSPN0VS3FDJQF JO4VQFSWJTFE'JOF - UVOF
  37. 1SFGFSFODF%BUBܳtੜu݅٘חѪ਷઺ਃೞ׮ بݫੋ߹۽જ਷QSFGFSFODFEBUBٜܳ݅ӝਤೠпӝ੹ۚ੉೙ਃೞ׮ ,FZ-FTTPOTGSPN0VS3FDJQF *O1SFGFSFODF-FBSOJOH അࣻחҗੌоѱܳ਍৔ೞҊ੓णפ׮оѱীחࢎҗ৬߄աաоппѐ৬ѐо੓णפ׮അࣻחоѱ੄ࢎҗ৬߄աաܳݽف౹ওਸٸ ੄୨ࣻੑਸ҅࢑ೞ۰Ҋ೤פ׮ࢎҗחѐ׼ਗ ߄աաחѐ׼ਗী౸ݒؾפ׮ ଵҊ׹਷ ਗ 8IJDIPOFJTCFUUFS

    അࣻооѱ੄ࢎҗ৬߄աաܳݽف౹ওਸٸ੄୨ࣻੑਸ҅ ࢑ೞ۰Ҋ೤פ׮ࢎҗחѐ׼ਗ ߄աաחѐ׼ਗ ী౸ݒؾפ׮ ࢎҗѐܳ౸ݒೞݶ Y ਗਸ঳ਸ ࣻ੓णפ׮ ߄աաѐܳ౸ݒೞݶ Y ਗਸ঳ ਸࣻ੓णפ׮ ٮۄࢲ ࢎҗ৬߄աաܳݽف౹ওਸٸ੄୨ࣻੑ਷  ਗ  ਗ ਗੑפ׮ അࣻоݽٚҗੌਸ౹ওਸٸ୨ࣻੑ਷ਗYѐ  ਗYѐ ਗੑפ׮
  38. 1SFGFSFODF%BUBܳtੜu݅٘חѪ਷઺ਃೞ׮ بݫੋ߹۽જ਷QSFGFSFODFEBUBٜܳ݅ӝਤೠпӝ੹ۚ੉೙ਃೞ׮ ,FZ-FTTPOTGSPN0VS3FDJQF *O1SFGFSFODF-FBSOJOH അࣻחҗੌоѱܳ਍৔ೞҊ੓णפ׮оѱীחࢎҗ৬߄աաоппѐ৬ѐо੓णפ׮അࣻחоѱ੄ࢎҗ৬߄աաܳݽف౹ওਸٸ ੄୨ࣻੑਸ҅࢑ೞ۰Ҋ೤פ׮ࢎҗחѐ׼ਗ ߄աաחѐ׼ਗী౸ݒؾפ׮ ଵҊ׹਷ ਗ 👎/FHBUJWFSFTQPOTF

    അࣻооѱ੄ࢎҗ৬߄աաܳݽف౹ওਸٸ੄୨ࣻੑਸ҅ ࢑ೞ۰Ҋ೤פ׮ࢎҗחѐ׼ਗ ߄աաחѐ׼ਗ ী౸ݒؾפ׮ ࢎҗѐܳ౸ݒೞݶ Y ਗਸ঳ਸ ࣻ੓णפ׮ ߄աաѐܳ౸ݒೞݶ Y ਗਸ঳ ਸࣻ੓णפ׮ ٮۄࢲ ࢎҗ৬߄աաܳݽف౹ওਸٸ੄୨ࣻੑ਷  ਗ  ਗ ਗੑפ׮ 👍1PTJUJWFSFTQPOTF അࣻоݽٚҗੌਸ౹ওਸٸ୨ࣻੑ਷ਗYѐ  ਗYѐ ਗੑפ׮
  39. <POMJOFOFTT>೟णद੼੄ݽ؛੄PVUQVUਸ૒੽੉ਊೞৈQSFGFSFODFܳ೟णೞחѪ਷઺ਃೞ׮ ,FZ-FTTPOTGSPN0VS3FDJQF *O1SFGFSFODF-FBSOJOH QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ઁ੉ܴ਷஠աաੑפ׮👍 j

    %BUB SFTQPOTF Zs ա ஠աա👎 3FTQPOTF¬ --. 3FTQPOTF­ SFTQPOTF Zs ա ஠աա👎 👍 👎 QSPNQU Y ց੉ܴ੉ޤפ j %BUB ઁ੉ܴ਷஠աաੑפ׮¬ --. ա ஠աա­ 3. ઁ੉ܴ਷஠աաੑפ׮ ա ஠աա PG fl JOFQSFGFSFODFMFBSOJOH POMJOFQSFGFSFODFMFBSOJOH
  40. पઁࢲ࠺झܳࢎਊೞחഋక৬ਬࢎೠؘ੉ఠ4ZOUIFUJDEBUBHFOFSBUJPO੽Ӕ ݽ؛җࣗా੉੗োझ۞਍NVMUJ - UVSOؘ੉ఠחӒۧ૑ঋ਷ؘ੉ఠী࠺೧ݽ؛ࢿמೱ࢚ী௼ѱӝৈೠ׮ فо૑ҙ੼ਸ઱ݾ    TJOHMF೟ ण

    উ જ ਷ N VMUJ೟ ण જ ਷ N VMUJ೟ ण TJOHMF೟ ण উ જ ਷ N VMUJ೟ ण જ ਷ N VMUJ೟ ण TJOHMF೟ ण উ જ ਷ N VMUJ೟ ण જ ਷ N VMUJ೟ ण 0WFSBMM TU - UVSO OE - UVSO
  41. 0OMJOF 4FMG - QMBZ QSFGFSFODFMFBSOJOH ੿׹ਸࠁݶࢲ೟ण 4'5  -> ׮ܲࢎۈ੄੿য়׹֢౟ܳా೧೟ण

    PG fl JOFQSFGFSFODFMFBSOJOH  فо૑ҙ੼ਸ઱ݾ QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ղ੉ܴ਷஠աաঠ j ղ੉ܴ਷஠աաঠ %BUB --. QSPNQU Y ց੉ܴ੉ޤפ SFTQPOTF Z ઁ੉ܴ਷஠աաੑפ׮👍 j %BUB SFTQPOTF Zs ա ஠աա👎 3FTQPOTF¬ --. 3FTQPOTF­ SFTQPOTF Zs ա ஠աա👎 4VQFSWJTFE'JOF - UVOF 0G fl JOF1SFGFSFODF-FBSOJOH
  42. 0OMJOF 4FMG - QMBZ QSFGFSFODFMFBSOJOH ੿׹ਸࠁݶࢲ೟ण 4'5  -> ׮ܲࢎۈ੄੿য়׹֢౟ܳా೧೟ण

    PG fl JOFQSFGFSFODFMFBSOJOH  -> ࠄੋ੉૒੽੿য়׹֢౟ٜܳ݅ݶࢲ೟ण POMJOFQSFGFSFODFMFBSOJOH  فо૑ҙ੼ਸ઱ݾ 👍 👎 QSPNQU Y ց੉ܴ੉ޤפ j %BUB ઁ੉ܴ਷஠աաੑפ׮¬ --. ա ஠աա­ 3. ઁ੉ܴ਷஠աաੑפ׮ ա ஠աա 0OMJOF1SFGFSFODF-FBSOJOH
  43. 5FBN-BOHVBHF.PEFM5SBJOJOH хࢎ೤פ׮ 5FBN-BOHVBHF.PEFM"MJHONFOU DIMPFQZ ೣ૑ো EBOJFMSM թਗక TFBOBJ ੿थ੤ LFWJOOMQ

    Ҋഅਔ LIFMI ӣഅഐ MPVJFN ੉ହ޹ KVMJFUCBL ߅ਮ઱ MBNCEBYQSJNF ੉ഐ૓ XBWZKVOH ੿ف೧ MBOBOZ ӣաো SZBOV ܨ޹ഐ
  44. 2"