Interrupt Affinityについて

Transcript

1. Interrupt Affinity
 ʹ͍ͭͯ @syuu1228
2. Interrupt Affinity? ࠓ࣌ͷී௨ͷ1$͸ϚϧνίΞͰ͋Δ
   σόΠε̍ɿ*32̍ɿ$16͍ͬͺ͍
 ͱ͍͏ؔ܎ʹͳ͍ͬͯΔ
   σόΠε͸Ͳ͜ʹׂΓࠐΜͰ͍Δͷʁ
 -JOVY͸Ͳͷ$16ʹׂΓࠐ·͍ͤͯΔͷʁ 
 ˠ
   *OUFSSVQU
   "⒏OJUZ
3. Interrupt Affinityͷ ϢʔβΠϯλϑΣʔε
4. $ cat /proc/interrupts CPU0 CPU1 0: 36843 61973 IO-APIC-edge timer 1: 518 591 IO-APIC-edge i8042 8: 1 0 IO-APIC-edge rtc0 9: 387 321 IO-APIC-fasteoi acpi 12: 11236 79 IO-APIC-edge i8042 16: 0 0 IO-APIC-fasteoi uhci_hcd:usb6 17: 12 11 IO-APIC-fasteoi uhci_hcd:usb7 18: 0 0 IO-APIC-fasteoi uhci_hcd:usb8 19: 6531 235 IO-APIC-fasteoi ehci_hcd:usb2 20: 0 0 IO-APIC-fasteoi uhci_hcd:usb3 21: 85 69 IO-APIC-fasteoi uhci_hcd:usb4 22: 0 0 IO-APIC-fasteoi uhci_hcd:usb5 23: 20 15 IO-APIC-fasteoi ehci_hcd:usb1 40: 0 0 PCI-MSI-edge PCIe PME, pciehp 41: 0 0 PCI-MSI-edge PCIe PME, pciehp 42: 0 0 PCI-MSI-edge PCIe PME, pciehp 43: 33199 31756 PCI-MSI-edge ahci 44: 19 1583 PCI-MSI-edge eth0 45: 10 11 PCI-MSI-edge mei 46: 41104 59303 PCI-MSI-edge i915 47: 170268 708 PCI-MSI-edge iwlwifi 48: 78 77 PCI-MSI-edge snd_hda_intel NMI: 1406 1430 Non-maskable interrupts $16ຖͷׂΓࠐΈ౷ܭΛදࣔ /proc/interrupts
5. /proc/irq/*/smp_affinity ׂΓࠐΈઌ$16܈ΛϏοτͰࢦఆ͢Δ ͱɺࢦఆ͞Εͨ$16܈ʹϥ΢ϯυϩϏ ϯͰׂΓࠐ·ΕΔʢϏοτ͕୯ҰͳΒ ৗʹͦͷ$16΁ׂΓࠐΈʣ # cat /proc/irq/43/smp_affinity 3 # echo 1 > /proc/irq/43/smp_affinity ͜ΕΛॻ͖׵͑ΔͱׂΓࠐΈઌ$16͕มΘΔ
6. LinuxͰͷσϑΥϧτ஋ શ$16ͷϏοτ͕༗ޮʹͳ͓ͬͯΓɺ ͭͷׂΓࠐΈ͕શ$16΁ϥ΢ϯυϩ Ϗϯ͢Δ
   ϥ΢ϯυϩϏϯׂΓࠐΈʹରԠ͠ͳ͍1$ Ͱ͸͋Δ$16΁ݻఆతʹׂΓࠐΉ ʢDQV ʣ
7. /proc/irq/*/smp_affinity_list ώϡʔϚϯϦʔμϒϧͳTNQ@B⒏OJUZ # echo ff > /proc/irq/43/smp_affinity # cat /proc/irq/43/smp_affinity_list
 0-7
8. /proc/irq/*/smp_affinity_hint σόΠευϥΠό͔ΒϢʔβϥϯυσʔ ϞϯʢJSRCBMBODFʣ΁͜ͷ*32ʹద੾ ͳ$16
   NBTLΛ௨஌
9. irqbalance ඵ͝ͱʹγεςϜෛՙঢ়گΛνΣο Ϋɺ֤*32ͷTNQ@B⒏OJUZΛॻ͖׵͑ ͯ$16ෛՙΛެฏʹ͢ΔσʔϞϯ
   POFTIPUϞʔυͰ͸ىಈ࣌ʹద੾ͱࢥ ΘΕΔ஋ΛTNQ@B⒏OJUZʹઃఆͯ͠ FYJU
10. MultiQueue NICͷׂΓࠐΈ $16ຖʹૹड৴ΩϡʔɺΩϡʔຖʹ .4*9ׂΓࠐΈΛ࣋ͭ
    ਖ਼͍͠TNQ@B⒏OJUZ͸ݻఆతʹܾ·ͬ ͍ͯΔ
    ෳ਺$16΁ࢄΒͯ͠͸ͳΒͳ͍ 47: 7602 0 0 0 0 3 23 0 PCI-MSI-edge p1p1-TxRx-0 48: 0 7602 0 0 0 0 13 12 PCI-MSI-edge p1p1-TxRx-1 49: 12 0 7605 0 0 0 10 0 PCI-MSI-edge p1p1-TxRx-2 50: 0 12 0 7602 3 0 10 0 PCI-MSI-edge p1p1-TxRx-3 51: 0 0 12 0 7602 3 10 0 PCI-MSI-edge p1p1-TxRx-4 52: 0 0 0 20 0 7602 13 0 PCI-MSI-edge p1p1-TxRx-5 53: 0 0 0 0 12 0 7612 3 PCI-MSI-edge p1p1-TxRx-6 54: 3 0 0 0 0 13 10 7602 PCI-MSI-edge p1p1-TxRx-7 55: 0 2 0 0 0 0 2 0 PCI-MSI-edge p1p1
11. ιʔείʔυ QSPDJOUFSSVQUT
 GTQSPDJOUFSSVQUTD
    QSPDJSR
 LFSOFMJSRQSPDD
    JSRCBMBODF
 IUUQTHJUIVCDPNJSRCBMBODF JSRCBMBODF
12. PCΞʔΩςΫνϟʹ͓͚Δ
 ׂΓࠐΈϧʔςΟϯά
13. PC্ͷׂΓࠐΈίϯϙʔωϯτ CPU Local APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip
14. Local APIC $16ຖͷׂΓࠐΈίϯτϩʔϥɹׂΓࠐΈΛڐՄɾϚεΫ͠ ͨΓɺ&0*ͨ͠Γ
    γεςϜશମͰҰҙͳ"1*$
    *%Λ࣋ͭ
    ଞͷ-"1*$΁ׂΓࠐΊΔʢ*1*
    *OUFSQSPDFTTPS
    *OUFSSVQU CPU Local APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip
15. LAPIC - Interrupt Command Register(ICR) 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸
    %FMJWFSZ
    .PEFɿׂΓࠐΈ഑ૹϞʔυ
    %FTUJOBUJPO
    .PEFɿѼઌϞʔυ
    %FMJWFS
    4UBUVTɿલճͷׂΓࠐΈͷ഑ૹঢ়گ
    5SJHHFS
    .PEFɿϨϕϧτϦΨʗΤοδτϦΨ
    %FTUJOBUJPO
    4IPSUIBOEɿѼઌͷϚεΫํࣜʢࣗ$16ΛϚεΫɾ ΞϯϚεΫͳͲʣ
    %FTUJOBUJPO
    *%ɿѼઌ
16. ιʔείʔυ *$3
 BSDIYJODMVEFBTNJQJI
17. σʔλγʔτ *OUFMš
    
    BOE
    *"
    "SDIJUFDUVSFT
    4PGUXBSF
    %FWFMPQFS
    .BOVBMT
 
    *OUFSSVQU
    $PNNBOE
    3FHJTUFS *$3
18. ิ଍ɿx2APIC -"1*$
    *%͸ैདྷCJU͕ͩͬͨɺίΞ਺ͷ૿ՃʹΑ ΓϏοτ਺͕଍Γͳ͘ͳͬͯ͠·ͬͨͷͰCJU΁ ϏοτΛ௥Ճ
    ͜Εʹ൐ͬͯɺҎ߱Ͱղઆ͢Δ֤ϨδελͰͷׂΓ ࠐΈઌ-"1*$ࢦఆϑΟʔϧυͷΞυϨε෯΋Ϗοτ ௥Ճ͞Εͨ
19. σʔλγʔτ *OUFMš
    
    "SDIJUFDUVSF
    Y"1*$
    4QFDJpDBUJPO
 
    (-044"3:
20. I/O APIC ݶΒΕͨຊ਺ͷ෺ཧׂΓࠐΈઢΛ֤1$*σόΠεͰڞ༗
 ʢ1$*Fʹ͸෺ཧׂΓࠐΈઢ͸ແ͘ɺϝοηʔδϯάʹΑͬͯ*0
    "1*$ܦ༝ׂΓࠐΈΛ ΤϛϡϨʔτʣ
    ֤*32ΛͲͷ-"1*$΁సૹ͢Δ͔Λఆٛ͢Δ3FEJSFDUJPO
    5BCMF
 ʢ*0
    "1*$্ͷϨδελ܈ʣΛ࣋ͭ
    1$*σόΠε͸*0
    "1*$Λ௨ͯ͡-"1*$΁ׂΓࠐΈϝοηʔδΛૹ৴ CPU Local APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip
21. I/O APIC - REDIR_TBL 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸
    %FMJWFSZ
    .PEFɿׂΓࠐΈ഑ૹϞʔυ
    %FTUJOBUJPO
    .PEFɿѼઌϞʔυ
    %FMJWFSZ
    4UBUVTɿલճͷׂΓࠐΈͷ഑ૹঢ়گ
    *OUFSSVQU
    *OQVU
    1JO
    1PMBSJUZɿׂΓࠐΈϐϯͷۃੑ
    3FNPUF
    *33ɿࣗಈతʹ&0*͢Δ͔Ͳ͏͔
    5SJHHFS
    .PEFɿϨϕϧτϦΨʗΤοδτϦΨ
    .BTLɿׂΓࠐΈϚεΫ
    %FTUJOBUJPO
    *%ɿѼઌ
22. ιʔείʔυ 3&%*3@5#-
 BSDIYLFSOFMBQJDJP@BQJDD
23. σʔλγʔτ *OUFMš
    *0
    $POUSPMMFS
    )VC
    
    *$)
    'BNJMZ
    %BUBTIFFU
 
    3&%*3@5#-3FEJSFDUJPO
    5BCMF
24. ϨΨγσόΠε ϨΨγσόΠε͔ΒͷׂΓࠐΈΛ1$*-1$
    CSJEHFͰ1$*΁ม׵
    1$*σόΠεಉ༷ʹ*0
    "1*$Λ௨ͯ͡-"1*$΁ׂΓࠐΈϝοηʔ δΛૹ৴
    ޓ׵ੑͷͨΊ"ʹ΋઀ଓ͞Ε͍ͯΔͱࢥΘΕΔ CPU Local APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip
25. 8259A PIC লུʢ·ͩ࢒ͬͯ·͕͢ɺ΋͏࢖Θͳ ͍Ͱ͠ΐ͏ʣ
26. MSI(-X) capable PCI devices ֤σόΠε͕೚ҙͷ਺ͷ*32Λ࣋ͯΔ
    ֤*32ͷׂΓࠐΈઌ-"1*$͸1$*σόΠεͷ1$*
    $POpHVSBUJPO
    4QBDFʹ࣋ͭ
    1$*σόΠε͸௚઀-"1*$΁ׂΓࠐΈϝοηʔδΛૹ৴ CPU Local APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip
27. MSI Capability on
 PCI configuration space Device ID Vendor ID … Reserved Cap. Pointer … Capability x CAP ID(x) Next Pointer … Capability y CAP ID(yy) Next Pointer … MSICTL CAP ID(D0h) Next Pointer MSIAR MSIDR
28. MSI Capability - MSICTL লུʢ.4*༗ޮɺCJUɺෳ਺ׂΓࠐ Έ༗ޮͳͲͷϑΟʔϧυʣ
29. MSI Capability - MSIAR %FTUJOBUJPO
    .PEFɿѼઌϞʔυ
    3FEJSFDUJPO
    )JOUɿMPHJDBM
    NPEFˍ -PX1SJͰͷSFEJSFDUJPOΛ༗ޮԽ
    %FTUJOBUJPO
    *%ɿѼઌ
30. MSI Capability - MSIDR 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸
    %FMJWFSZ
    .PEFɿׂΓࠐΈ഑ૹϞʔυ
    %FMJWFSZ
    4UBUVTɿׂΓࠐΈঢ়گ
    5SJHHFS
    .PEFɿϨϕϧτϦΨʗΤο δτϦΨ
31. MSI-Xͷ৔߹ লུʢϑΟʔϧυ͕֦ு͞Εͯ୔ࢁׂ ΓࠐΈ͕࣋ͯΔ͕ɺ࢓૊Έ͸ಉ͡ʣ
32. ιʔείʔυ .4* 9
    $BQBCJMJUZ
 BSDIYLFSOFMBQJDJP@BQJDD
33. σʔλγʔτ *OUFMš
    
    $IJQTFU
    BOE
    *OUFMš
    
    $IJQTFU
 
    .4*$5-
    .4*
    $POUSPM
    3FHJTUFS
 
    .4*"3
    .4*
    "EESFTT
    3FHJTUFS
 
    .4*%3
    .4*
    %BUB
    3FHJTUFS
34. ֤ํࣜͰڞ௨ͳཁૉ ҎԼͷࡾͭͷϑΟʔϧυͰѼઌ͕ࢦఆ ͞ΕΔ
    %FTUJOBUJPO
    .PEF
    %FMJWFSZ
    .PEF
    %FTUJOBUJPO
    *%
35. ׂΓࠐΈઌͷࢦఆํ๏ 1IZTJDBM
    %FTUJOBUJPO
    .PEF
    %FTUJOBUJPO
    'JFMEʹ"1*$
    *%Λࢦఆ
 ʮৗʹಉ͡$16΁ׂΓࠐΈʯΛ࣮ݱ
    -PHJDBM
    %FTUJOBUJPO
    .PEF 'MBU
    .PEFM
    %FTUJOBUJPO
    'JFMEͷCJUͰѼઌ-"1*$܈ͷൣғΛදݱ
    %FMJWFSZ
    .PEF
    'JYFE
 ࢦఆൣғͷશͯͷ-"1*$΁ׂΓࠐΈ
    -PXFTU
    1SJPSJUZ
 ࢦఆൣғͷதͰɺ࠷΋513ʢ5BTL
    1SJPSJUZ
    3FHJTUFSʣͷ஋͕ ௿͍$16΁ׂΓࠐΈ
36. Logical Destination Mode
 (Cluster Model) Y"1*$ͷ৔߹ɺ
 CJU
    
    $MVTUFS
    *%
 CJU
    
    -PHJDBM
    *%
    'MBU
    .PEFMʹൺ΂ͯଟ͘ͷ$16਺Λ αϙʔτͰ͖Δ
37. Lowest Priority Mode ࢦఆൣғͷதͰɺ࠷΋513ʢ5BTL
    1SJPSJUZ
    3FHJTUFSʣͷ஋͕௿͍-"1*$΁ ׂΓࠐΈ
    ୠ͠ɺ-JOVYͰ͸ಈ࡞தʹ513Λߋ৽͠ͳ ͍ˠશ-"1*$ͷ513͸શͯಉ͡஋
    ࠷খ஋ͷ513Λ࣋ͭ-"1*$͕ෳ਺͋Δ৔ ߹͸ϥ΢ϯυϩϏϯͰ̍ͭબ୒
38. σʔλγʔτ *OUFMš
    
    BOE
    *"
    "SDIJUFDUVSFT
    4PGUXBSF
    %FWFMPQFS
    .BOVBMT
 
    *OUFSSVQU
    $PNNBOE
    3FHJTUFS *$3 
 
    1IZTJDBM
    %FTUJOBUJPO
    .PEF
    
 
    -PHJDBM
    %FTUJOBUJPO
    .PEF
 
    -PXFTU
    1SJPSJUZ
    %FMJWFSZ
    .PEF
    *OUFMš
    
    "SDIJUFDUVSF
    Y"1*$
    4QFDJpDBUJPO
 
    -PHJDBM
    %FTUJOBUJPO
    3FHJTUFS
    *OUFMš
    *0
    $POUSPMMFS
    )VC
    
    *$)
    'BNJMZ
    %BUBTIFFU
 
    "VUPNBUJD
    3PUBUJPO
    .PEF
    &RVBM
    1SJPSJUZ
    %FWJDFT
    
 
    
    4QFDJpD
    3PUBUJPO
    .PEF
    4QFDJpD
    1SJPSJUZ
    
39. echo 1 > /proc/irq/<IRQ>/ smp_affinityͨ࣌͠ʹԿ͕ى͖Δͷ͔ .4*ͳ1$*FσόΠεͰ͔֬ΊͯΈΔ
    *OUFM
    1SP FF
    1$*
    $POpHVSBUJPO
    4QBDF͸SPPUͳΒ TZTCVTQDJ͔ΒಡΊΔ
 ˠϢʔβϥϯυ͔ΒಡΊΔ
 ˠ΋͔ͯ͠͠ɿMTQDJ

40. # lspci -vvvv -s 00:19.0 00:19.0 Ethernet controller: Intel Corporation 82567LM Gigabit Network Connection (rev 03) Subsystem: Lenovo Device 20ee Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 44 Region 0: Memory at f2600000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at f2625000 (32-bit, non-prefetchable) [size=4K] Region 2: I/O ports at 1840 [size=32] Capabilities: [c8] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee0300c Data: 41b9 Capabilities: [e0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: e1000e Kernel modules: e1000e ੺͘ృͬͨॴ͕.4*ͷϑΟʔϧυ͕ͩɺ͜Εͩ ͱಡΊͳ͍ͷͰMTQDJΛGPSLͯ͠.4*ϑΟʔϧ υΛදࣔ͢ΔπʔϧΛ࡞੒
 IUUQTHJTUHJUIVCDPN
41. # gcc -lpci msireg.c! # ./a.out 00:19.0! Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+! address_hi=0! address_lo=fee0300c dest_mode=logical redirection=lowpri dest_id=3! data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185 -PHJDBM
    NPEFͰ-PXQSJɺEFTUJEɺ WFDUPSʹͳ͍ͬͯΔ # echo 1 > /proc/irq/44/smp_affinity! # ./a.out 00:19.0! Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+! address_hi=0! address_lo=fee0100c dest_mode=logical redirection=lowpri dest_id=1! data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185 EFTU@JE͕ʹॻ͖׵Θͬͨ
42. Affinityηοτ࣌ͷΧʔωϧͷڍಈ γεςϜάϩʔόϧͳ%FTUJOBUJPO
    .PEFͱ %FMJWFSZ
    .PEFͷઃఆΛอ࣋
 ʢBQJDJSR@EFTU@NPEF 
 BQJDJSR@EFMJWFSZ@NPEFʣ
    "⒏OJUZηοτ࣌ʹ͜ͷ஋Λࢀরɺ%FTUJOBUJPO
    *%ͱซͤͯॻ͖ࠐΉ
    OBUJWF@DPNQPTF@NTJ@NTH
 BSDIYLFSOFMBQJDJP@BQJDD
43. OSॳظԽ࣌ͷׂΓࠐΈϞʔυઃఆ ؀ڥʹΑͬͯҟͳΔυϥΠόΛ࢖༻
 BSDIYLFSOFMBQJDBQJD@qBU@D
    YBQJD@DMVTUFSD
    YBQJD@QIZTDʜ
    BQJD@qBUʢඇYBQJDϞʔυʣ
 %FMJWFSZ
    .PEF
    
    -PX1SJ
 %FTUJOBUJPO
    .PEF
    
    -PHJDBM
    BQJD@QIZTqBUʢඇYBQJDϞʔυ
    $164
    
    ʣ
 %FMJWFSZ
    .PEF
    
    'JYFE
 %FTUJOBUJPO
    .PEF
    
    1IZTJDBM
    BQJD@YBQJD@QIZTʢYBQJDϞʔυ "$1*Ͱ1IZTJDBMϞʔυࢦఆʣ
 ಉ্
    ϥ΢ϯυϩϏϯׂΓࠐΈͰ͖Δ؀ڥ͸Ұఆ৚݅Λຬͨͨ͠৔߹ͷΈ
44. *OUFMš
    
    BOE
    *"
    "SDIJUFDUVSFT
    4PGUXBSF
    %FWFMPQFS
    .BOVBMT
    *OUFMš
    *0
    $POUSPMMFS
    )VC
    
    *$)
    'BNJMZ
    %BUBTIFFU
    *OUFMš
    
    "SDIJUFDUVSF
    Y"1*$
    4QFDJpDBUJPO
    *OUFMš
    
    $IJQTFU
    BOE
    *OUFMš
    
    $IJQTFU
    MJOVY%PDVNFOUBUJPO*32B⒏OJUZUYU
    6OEFSTUBOEJOH
    UIF
    -JOVY
    ,FSOFM
    SE
    &EJUJPO
    1$*
    -PDBM
    #VT
    4QFDJpDBUJPO
    3FWJTJPO
     ࢀߟࢿྉ
45. ࠓ·Ͱʹ·ͱΊͨࢿྉ ࠷ۙͷ1$ΞʔΩςΫνϟʹ͓͚ΔׂΓࠐΈϧʔςΟϯάͷ࢓૊Έ
    QSPDJSR*32TNQ@B⒏OJUZͷॻ͖׵͑Ͱ1$*ίϯϑΟάϨʔ γϣϯۭؒ͸ͲͷΑ͏ʹॻ͖׵ΘΔ͔
    QSPDJSR*32TNQ@B⒏OJUZ
    *OUFSSVQUT
    PO
    YW
    ϋΠύʔόΠβͷ࡞ΓํʙͪΌΜͱཧղ͢ΔԾ૝Խٕज़ʙ
    ୈ̐ճ
    * 0
    Ծ૝ԽʮׂΓࠐΈฤɾͦͷ̍ʯ