Interrupt Affinityについて

Interrupt Affinity  ʹ͍ͭͯ @syuu1228

Interrupt Affinity? ࠓ࣌ͷී௨ͷ1$͸ϚϧνίΞͰ͋Δ σόΠε̍ɿ*32̍ɿ$16͍ͬͺ͍  ͱ͍͏ؔ܎ʹͳ͍ͬͯΔ σόΠε͸Ͳ͜ʹׂΓࠐΜͰ͍Δͷʁ  -JOVY͸Ͳͷ$16ʹׂΓࠐ·͍ͤͯΔͷʁ   ˠ*OUFSSVQU"⒏OJUZ

Interrupt Affinityͷ ϢʔβΠϯλϑΣʔε

$ cat /proc/interrupts CPU0 CPU1 0: 36843 61973 IO-APIC-edge timer
1: 518 591 IO-APIC-edge i8042 8: 1 0 IO-APIC-edge rtc0 9: 387 321 IO-APIC-fasteoi acpi 12: 11236 79 IO-APIC-edge i8042 16: 0 0 IO-APIC-fasteoi uhci_hcd:usb6 17: 12 11 IO-APIC-fasteoi uhci_hcd:usb7 18: 0 0 IO-APIC-fasteoi uhci_hcd:usb8 19: 6531 235 IO-APIC-fasteoi ehci_hcd:usb2 20: 0 0 IO-APIC-fasteoi uhci_hcd:usb3 21: 85 69 IO-APIC-fasteoi uhci_hcd:usb4 22: 0 0 IO-APIC-fasteoi uhci_hcd:usb5 23: 20 15 IO-APIC-fasteoi ehci_hcd:usb1 40: 0 0 PCI-MSI-edge PCIe PME, pciehp 41: 0 0 PCI-MSI-edge PCIe PME, pciehp 42: 0 0 PCI-MSI-edge PCIe PME, pciehp 43: 33199 31756 PCI-MSI-edge ahci 44: 19 1583 PCI-MSI-edge eth0 45: 10 11 PCI-MSI-edge mei 46: 41104 59303 PCI-MSI-edge i915 47: 170268 708 PCI-MSI-edge iwlwifi 48: 78 77 PCI-MSI-edge snd_hda_intel NMI: 1406 1430 Non-maskable interrupts $16ຖͷׂΓࠐΈ౷ܭΛදࣔ /proc/interrupts

/proc/irq/*/smp_affinity ׂΓࠐΈઌ$16܈ΛϏοτͰࢦఆ͢Δ ͱɺࢦఆ͞Εͨ$16܈ʹϥ΢ϯυϩϏ ϯͰׂΓࠐ·ΕΔʢϏοτ͕୯ҰͳΒ ৗʹͦͷ$16΁ׂΓࠐΈʣ # cat /proc/irq/43/smp_affinity 3 #
echo 1 > /proc/irq/43/smp_affinity ͜ΕΛॻ͖׵͑ΔͱׂΓࠐΈઌ$16͕มΘΔ

LinuxͰͷσϑΥϧτ஋ શ$16ͷϏοτ͕༗ޮʹͳ͓ͬͯΓɺ ͭͷׂΓࠐΈ͕શ$16΁ϥ΢ϯυϩ Ϗϯ͢Δ ϥ΢ϯυϩϏϯׂΓࠐΈʹରԠ͠ͳ͍1$ Ͱ͸͋Δ$16΁ݻఆతʹׂΓࠐΉ ʢDQV ʣ

/proc/irq/*/smp_affinity_list ώϡʔϚϯϦʔμϒϧͳTNQ@B⒏OJUZ # echo ff > /proc/irq/43/smp_affinity # cat /proc/irq/43/smp_affinity_list 
0-7

/proc/irq/*/smp_affinity_hint σόΠευϥΠό͔ΒϢʔβϥϯυσʔ ϞϯʢJSRCBMBODFʣ΁͜ͷ*32ʹద੾ ͳ$16NBTLΛ௨஌

irqbalance ඵ͝ͱʹγεςϜෛՙঢ়گΛνΣο Ϋɺ֤*32ͷTNQ@B⒏OJUZΛॻ͖׵͑ ͯ$16ෛՙΛެฏʹ͢ΔσʔϞϯ POFTIPUϞʔυͰ͸ىಈ࣌ʹద੾ͱࢥ ΘΕΔ஋ΛTNQ@B⒏OJUZʹઃఆͯ͠ FYJU

MultiQueue NICͷׂΓࠐΈ $16ຖʹૹड৴ΩϡʔɺΩϡʔຖʹ .4*9ׂΓࠐΈΛ࣋ͭ ਖ਼͍͠TNQ@B⒏OJUZ͸ݻఆతʹܾ·ͬ ͍ͯΔ ෳ਺$16΁ࢄΒͯ͠͸ͳΒͳ͍ 47: 7602 0
0 0 0 3 23 0 PCI-MSI-edge p1p1-TxRx-0 48: 0 7602 0 0 0 0 13 12 PCI-MSI-edge p1p1-TxRx-1 49: 12 0 7605 0 0 0 10 0 PCI-MSI-edge p1p1-TxRx-2 50: 0 12 0 7602 3 0 10 0 PCI-MSI-edge p1p1-TxRx-3 51: 0 0 12 0 7602 3 10 0 PCI-MSI-edge p1p1-TxRx-4 52: 0 0 0 20 0 7602 13 0 PCI-MSI-edge p1p1-TxRx-5 53: 0 0 0 0 12 0 7612 3 PCI-MSI-edge p1p1-TxRx-6 54: 3 0 0 0 0 13 10 7602 PCI-MSI-edge p1p1-TxRx-7 55: 0 2 0 0 0 0 2 0 PCI-MSI-edge p1p1

ιʔείʔυ QSPDJOUFSSVQUT  GTQSPDJOUFSSVQUTD QSPDJSR  LFSOFMJSRQSPDD JSRCBMBODF  IUUQTHJUIVCDPNJSRCBMBODF JSRCBMBODF

PCΞʔΩςΫνϟʹ͓͚Δ  ׂΓࠐΈϧʔςΟϯά

PC্ͷׂΓࠐΈίϯϙʔωϯτ CPU Local APIC CPU Local APIC CPU Local APIC
ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip

Local APIC $16ຖͷׂΓࠐΈίϯτϩʔϥɹׂΓࠐΈΛڐՄɾϚεΫ͠ ͨΓɺ&0*ͨ͠Γ γεςϜશମͰҰҙͳ"1*$*%Λ࣋ͭ ଞͷ-"1*$΁ׂΓࠐΊΔʢ*1**OUFSQSPDFTTPS*OUFSSVQU CPU Local APIC CPU
Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip

LAPIC - Interrupt Command Register(ICR) 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸ %FMJWFSZ.PEFɿׂΓࠐΈ഑ૹϞʔυ %FTUJOBUJPO.PEFɿѼઌϞʔυ %FMJWFS4UBUVTɿલճͷׂΓࠐΈͷ഑ૹঢ়گ 5SJHHFS.PEFɿϨϕϧτϦΨʗΤοδτϦΨ
%FTUJOBUJPO4IPSUIBOEɿѼઌͷϚεΫํࣜʢࣗ$16ΛϚεΫɾ ΞϯϚεΫͳͲʣ %FTUJOBUJPO*%ɿѼઌ

ιʔείʔυ *$3  BSDIYJODMVEFBTNJQJI

σʔλγʔτ *OUFMBOE*" "SDIJUFDUVSFT4PGUXBSF %FWFMPQFS.BOVBMT  *OUFSSVQU$PNNBOE 3FHJTUFS *$3

ิ଍ɿx2APIC -"1*$*%͸ैདྷCJU͕ͩͬͨɺίΞ਺ͷ૿ՃʹΑ ΓϏοτ਺͕଍Γͳ͘ͳͬͯ͠·ͬͨͷͰCJU΁ ϏοτΛ௥Ճ ͜Εʹ൐ͬͯɺҎ߱Ͱղઆ͢Δ֤ϨδελͰͷׂΓ ࠐΈઌ-"1*$ࢦఆϑΟʔϧυͷΞυϨε෯΋Ϗοτ ௥Ճ͞Εͨ

σʔλγʔτ *OUFM"SDIJUFDUVSFY"1*$4QFDJpDBUJPO  (-044"3:

I/O APIC ݶΒΕͨຊ਺ͷ෺ཧׂΓࠐΈઢΛ֤1$*σόΠεͰڞ༗  ʢ1$*Fʹ͸෺ཧׂΓࠐΈઢ͸ແ͘ɺϝοηʔδϯάʹΑͬͯ*0"1*$ܦ༝ׂΓࠐΈΛ ΤϛϡϨʔτʣ ֤*32ΛͲͷ-"1*$΁సૹ͢Δ͔Λఆٛ͢Δ3FEJSFDUJPO5BCMF  ʢ*0"1*$্ͷϨδελ܈ʣΛ࣋ͭ 1$*σόΠε͸*0"1*$Λ௨ͯ͡-"1*$΁ׂΓࠐΈϝοηʔδΛૹ৴ CPU Local
APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip

I/O APIC - REDIR_TBL 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸ %FMJWFSZ.PEFɿׂΓࠐΈ഑ૹϞʔυ %FTUJOBUJPO.PEFɿѼઌϞʔυ %FMJWFSZ4UBUVTɿલճͷׂΓࠐΈͷ഑ૹঢ়گ *OUFSSVQU*OQVU1JO1PMBSJUZɿׂΓࠐΈϐϯͷۃੑ 3FNPUF*33ɿࣗಈతʹ&0*͢Δ͔Ͳ͏͔
5SJHHFS.PEFɿϨϕϧτϦΨʗΤοδτϦΨ .BTLɿׂΓࠐΈϚεΫ %FTUJOBUJPO*%ɿѼઌ

ιʔείʔυ 3&%*3@5#-  BSDIYLFSOFMBQJDJP@BQJDD

σʔλγʔτ *OUFM*0$POUSPMMFS)VC *$) 'BNJMZ%BUBTIFFU  3&%*3@5#-3FEJSFDUJPO 5BCMF

ϨΨγσόΠε ϨΨγσόΠε͔ΒͷׂΓࠐΈΛ1$*-1$CSJEHFͰ1$*΁ม׵ 1$*σόΠεಉ༷ʹ*0"1*$Λ௨ͯ͡-"1*$΁ׂΓࠐΈϝοηʔ δΛૹ৴ ޓ׵ੑͷͨΊ"ʹ΋઀ଓ͞Ε͍ͯΔͱࢥΘΕΔ CPU Local APIC CPU Local
APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip

8259A PIC লུʢ·ͩ࢒ͬͯ·͕͢ɺ΋͏࢖Θͳ ͍Ͱ͠ΐ͏ʣ

MSI(-X) capable PCI devices ֤σόΠε͕೚ҙͷ਺ͷ*32Λ࣋ͯΔ ֤*32ͷׂΓࠐΈઌ-"1*$͸1$*σόΠεͷ1$* $POpHVSBUJPO4QBDFʹ࣋ͭ 1$*σόΠε͸௚઀-"1*$΁ׂΓࠐΈϝοηʔδΛૹ৴ CPU Local
APIC CPU Local APIC CPU Local APIC ICH 8259A PIC Timer I/O APIC Legacy PCI Devices MSI(-X) Capable Devices IPI Legacy PCI 8259A Intr MSI(-X) PCI-LPC bridge SuperIO Chip

MSI Capability on  PCI configuration space Device ID Vendor ID
… Reserved Cap. Pointer … Capability x CAP ID(x) Next Pointer … Capability y CAP ID(yy) Next Pointer … MSICTL CAP ID(D0h) Next Pointer MSIAR MSIDR

MSI Capability - MSICTL লུʢ.4*༗ޮɺCJUɺෳ਺ׂΓࠐ Έ༗ޮͳͲͷϑΟʔϧυʣ

MSI Capability - MSIAR %FTUJOBUJPO.PEFɿѼઌϞʔυ 3FEJSFDUJPO)JOUɿMPHJDBMNPEFˍ -PX1SJͰͷSFEJSFDUJPOΛ༗ޮԽ %FTUJOBUJPO*%ɿѼઌ

MSI Capability - MSIDR 7FDUPSɿѼઌ-"1*$ͷϕΫλ൪߸ %FMJWFSZ.PEFɿׂΓࠐΈ഑ૹϞʔυ %FMJWFSZ4UBUVTɿׂΓࠐΈঢ়گ 5SJHHFS.PEFɿϨϕϧτϦΨʗΤο δτϦΨ

MSI-Xͷ৔߹ লུʢϑΟʔϧυ͕֦ு͞Εͯ୔ࢁׂ ΓࠐΈ͕࣋ͯΔ͕ɺ࢓૊Έ͸ಉ͡ʣ

ιʔείʔυ .4* 9 $BQBCJMJUZ  BSDIYLFSOFMBQJDJP@BQJDD

σʔλγʔτ *OUFM$IJQTFUBOE*OUFM $IJQTFU  .4*$5-.4*$POUSPM 3FHJTUFS  .4*"3.4*"EESFTT 3FHJTUFS  .4*%3.4*%BUB3FHJTUFS

֤ํࣜͰڞ௨ͳཁૉ ҎԼͷࡾͭͷϑΟʔϧυͰѼઌ͕ࢦఆ ͞ΕΔ %FTUJOBUJPO.PEF %FMJWFSZ.PEF %FTUJOBUJPO*%

ׂΓࠐΈઌͷࢦఆํ๏ 1IZTJDBM%FTUJOBUJPO.PEF %FTUJOBUJPO'JFMEʹ"1*$*%Λࢦఆ  ʮৗʹಉ͡$16΁ׂΓࠐΈʯΛ࣮ݱ -PHJDBM%FTUJOBUJPO.PEF 'MBU.PEFM %FTUJOBUJPO'JFMEͷCJUͰѼઌ-"1*$܈ͷൣғΛදݱ %FMJWFSZ.PEF 'JYFE 
ࢦఆൣғͷશͯͷ-"1*$΁ׂΓࠐΈ -PXFTU1SJPSJUZ  ࢦఆൣғͷதͰɺ࠷΋513ʢ5BTL1SJPSJUZ3FHJTUFSʣͷ஋͕ ௿͍$16΁ׂΓࠐΈ

Logical Destination Mode  (Cluster Model) Y"1*$ͷ৔߹ɺ  CJU$MVTUFS*%  CJU-PHJDBM*% 'MBU.PEFMʹൺ΂ͯଟ͘ͷ$16਺Λ αϙʔτͰ͖Δ

Lowest Priority Mode ࢦఆൣғͷதͰɺ࠷΋513ʢ5BTL 1SJPSJUZ3FHJTUFSʣͷ஋͕௿͍-"1*$΁ ׂΓࠐΈ ୠ͠ɺ-JOVYͰ͸ಈ࡞தʹ513Λߋ৽͠ͳ ͍ˠશ-"1*$ͷ513͸શͯಉ͡஋ ࠷খ஋ͷ513Λ࣋ͭ-"1*$͕ෳ਺͋Δ৔ ߹͸ϥ΢ϯυϩϏϯͰ̍ͭબ୒

σʔλγʔτ *OUFMBOE*""SDIJUFDUVSFT4PGUXBSF%FWFMPQFS .BOVBMT  *OUFSSVQU$PNNBOE3FHJTUFS *$3   1IZTJDBM%FTUJOBUJPO.PEF  -PHJDBM%FTUJOBUJPO.PEF  -PXFTU1SJPSJUZ%FMJWFSZ.PEF *OUFM"SDIJUFDUVSFY"1*$4QFDJpDBUJPO 
-PHJDBM%FTUJOBUJPO3FHJTUFS *OUFM*0$POUSPMMFS)VC *$) 'BNJMZ%BUBTIFFU  "VUPNBUJD3PUBUJPO.PEF &RVBM1SJPSJUZ%FWJDFT   4QFDJpD3PUBUJPO.PEF 4QFDJpD1SJPSJUZ

echo 1 > /proc/irq/<IRQ>/ smp_affinityͨ࣌͠ʹԿ͕ى͖Δͷ͔ .4*ͳ1$*FσόΠεͰ͔֬ΊͯΈΔ *OUFM1SP FF 1$*$POpHVSBUJPO4QBDF͸SPPUͳΒ
TZTCVTQDJ͔ΒಡΊΔ  ˠϢʔβϥϯυ͔ΒಡΊΔ  ˠ΋͔ͯ͠͠ɿMTQDJ 

# lspci -vvvv -s 00:19.0 00:19.0 Ethernet controller: Intel Corporation
82567LM Gigabit Network Connection (rev 03) Subsystem: Lenovo Device 20ee Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 44 Region 0: Memory at f2600000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at f2625000 (32-bit, non-prefetchable) [size=4K] Region 2: I/O ports at 1840 [size=32] Capabilities: [c8] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee0300c Data: 41b9 Capabilities: [e0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: e1000e Kernel modules: e1000e ੺͘ృͬͨॴ͕.4*ͷϑΟʔϧυ͕ͩɺ͜Εͩ ͱಡΊͳ͍ͷͰMTQDJΛGPSLͯ͠.4*ϑΟʔϧ υΛදࣔ͢ΔπʔϧΛ࡞੒  IUUQTHJTUHJUIVCDPN

# gcc -lpci msireg.c! # ./a.out 00:19.0! Message Signalled Interrupts:
64bit+ Queue=0/0 Enable+! address_hi=0! address_lo=fee0300c dest_mode=logical redirection=lowpri dest_id=3! data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185 -PHJDBMNPEFͰ-PXQSJɺEFTUJEɺ WFDUPSʹͳ͍ͬͯΔ # echo 1 > /proc/irq/44/smp_affinity! # ./a.out 00:19.0! Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+! address_hi=0! address_lo=fee0100c dest_mode=logical redirection=lowpri dest_id=1! data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185 EFTU@JE͕ʹॻ͖׵Θͬͨ

Affinityηοτ࣌ͷΧʔωϧͷڍಈ γεςϜάϩʔόϧͳ%FTUJOBUJPO.PEFͱ %FMJWFSZ.PEFͷઃఆΛอ࣋  ʢBQJDJSR@EFTU@NPEF   BQJDJSR@EFMJWFSZ@NPEFʣ "⒏OJUZηοτ࣌ʹ͜ͷ஋Λࢀরɺ%FTUJOBUJPO *%ͱซͤͯॻ͖ࠐΉ OBUJWF@DPNQPTF@NTJ@NTH  BSDIYLFSOFMBQJDJP@BQJDD

OSॳظԽ࣌ͷׂΓࠐΈϞʔυઃఆ ؀ڥʹΑͬͯҟͳΔυϥΠόΛ࢖༻  BSDIYLFSOFMBQJDBQJD@qBU@D YBQJD@DMVTUFSD YBQJD@QIZTDʜ BQJD@qBUʢඇYBQJDϞʔυʣ  %FMJWFSZ.PEF-PX1SJ  %FTUJOBUJPO.PEF-PHJDBM BQJD@QIZTqBUʢඇYBQJDϞʔυ
$164ʣ  %FMJWFSZ.PEF'JYFE  %FTUJOBUJPO.PEF1IZTJDBM BQJD@YBQJD@QIZTʢYBQJDϞʔυ "$1*Ͱ1IZTJDBMϞʔυࢦఆʣ  ಉ্ ϥ΢ϯυϩϏϯׂΓࠐΈͰ͖Δ؀ڥ͸Ұఆ৚݅Λຬͨͨ͠৔߹ͷΈ

*OUFMBOE*""SDIJUFDUVSFT4PGUXBSF %FWFMPQFS.BOVBMT *OUFM*0$POUSPMMFS)VC *$) 'BNJMZ %BUBTIFFU *OUFM"SDIJUFDUVSFY"1*$4QFDJpDBUJPO *OUFM$IJQTFUBOE*OUFM$IJQTFU MJOVY%PDVNFOUBUJPO*32B⒏OJUZUYU 6OEFSTUBOEJOHUIF-JOVY,FSOFM
SE&EJUJPO 1$*-PDBM#VT4QFDJpDBUJPO3FWJTJPO ࢀߟࢿྉ

ࠓ·Ͱʹ·ͱΊͨࢿྉ ࠷ۙͷ1$ΞʔΩςΫνϟʹ͓͚ΔׂΓࠐΈϧʔςΟϯάͷ࢓૊Έ QSPDJSR*32TNQ@B⒏OJUZͷॻ͖׵͑Ͱ1$*ίϯϑΟάϨʔ γϣϯۭؒ͸ͲͷΑ͏ʹॻ͖׵ΘΔ͔ QSPDJSR*32TNQ@B⒏OJUZ *OUFSSVQUTPOYW ϋΠύʔόΠβͷ࡞ΓํʙͪΌΜͱཧղ͢ΔԾ૝Խٕज़ʙୈ̐ճ* 0Ծ૝ԽʮׂΓࠐΈฤɾͦͷ̍ʯ

Interrupt Affinityについて

Interrupt Affinityについて

More Decks by Takuya ASADA

Other Decks in Technology

Featured

Transcript