/proc/irq/<IRQ>/ smp_affinity

Transcript

1. /proc/irq//
   smp_affinity
   @syuu1228
   12೥9݄22೔౔༵೔
   

   View Slide

2. cat /proc/interrupts
   12೥9݄22೔౔༵೔
   

   View Slide

3. $ cat /proc/interrupts
   CPU0 CPU1
   0: 36843 61973 IO-APIC-edge timer
   1: 518 591 IO-APIC-edge i8042
   8: 1 0 IO-APIC-edge rtc0
   9: 387 321 IO-APIC-fasteoi acpi
   12: 11236 79 IO-APIC-edge i8042
   16: 0 0 IO-APIC-fasteoi uhci_hcd:usb6
   17: 12 11 IO-APIC-fasteoi uhci_hcd:usb7
   18: 0 0 IO-APIC-fasteoi uhci_hcd:usb8
   19: 6531 235 IO-APIC-fasteoi ehci_hcd:usb2
   20: 0 0 IO-APIC-fasteoi uhci_hcd:usb3
   21: 85 69 IO-APIC-fasteoi uhci_hcd:usb4
   22: 0 0 IO-APIC-fasteoi uhci_hcd:usb5
   23: 20 15 IO-APIC-fasteoi ehci_hcd:usb1
   40: 0 0 PCI-MSI-edge PCIe PME, pciehp
   41: 0 0 PCI-MSI-edge PCIe PME, pciehp
   42: 0 0 PCI-MSI-edge PCIe PME, pciehp
   43: 33199 31756 PCI-MSI-edge ahci
   44: 19 1583 PCI-MSI-edge eth0
   45: 10 11 PCI-MSI-edge mei
   46: 41104 59303 PCI-MSI-edge i915
   47: 170268 708 PCI-MSI-edge iwlwifi
   48: 78 77 PCI-MSI-edge snd_hda_intel
   NMI: 1406 1430 Non-maskable interrupts
   LOC: 342351 327074 Local timer interrupts
   SPU: 0 0 Spurious interrupts
   PMI: 1406 1430 Performance monitoring interrupts
   IWI: 0 0 IRQ work interrupts
   RES: 83855 108903 Rescheduling interrupts
   CAL: 223 195 Function call interrupts
   TLB: 1486 1665 TLB shootdowns
   TRM: 0 6 Thermal event interrupts
   THR: 0 0 Threshold APIC interrupts
   MCE: 0 0 Machine check exceptions
   MCP: 11 11 Machine check polls
   ERR: 0
   MIS: 0
   ஸ౓ྑ͘ෳ਺ίΞ΁෼ࢄ͞ΕͯΔʁ
   12೥9݄22೔౔༵೔
   

   View Slide

4. smp_affinity
   ׂΓࠐΈઌCPU܈ΛϏοτͰࢦఆ͢Δ
   ͱɺࢦఆ͞ΕͨCPU܈ʹϥ΢ϯυϩϏ
   ϯͰׂΓࠐ·ΕΔ
   # cat /proc/irq/43/smp_affinity
   3
   # echo 1 > /proc/irq/43/smp_affinity
   ͜ΕΛॻ͖׵͑Ε͹͍͍ͷ͔ʂ
   12೥9݄22೔౔༵೔
   

   View Slide

5. ʔ ׬ ʔ
   12೥9݄22೔౔༵೔
   

   View Slide

6. ͦΕͰ͸#kernelvmΒ͘͠ͳ͍
   bitͷཱͬͯΔίΞ΁ׂΓࠐΈ෼ࢄ
   bit͕ҰͭͳΒ͍ͭ΋ͦ͜΁ׂΓࠐΈ
   ͜ΕɺͲ͏ͳͬͯΔΜͩΖ͏ʁ
   # cat /proc/irq/43/smp_affinity
   3
   # echo 1 > /proc/irq/43/smp_affinity
   12೥9݄22೔౔༵೔
   

   View Slide

7. ΠϚυΩͳx86ͷׂΓࠐΈ
   CPU
   Local APIC
   CPU
   Local APIC
   CPU
   Local APIC
   ICH(South bridge)
   8259A PIC
   Timer
   I/O APIC
   Legacy
   PCI
   Devices
   MSI
   Capable
   Devices
   IPI
   MSI
   Legacy PCI
   8259A Intr
   12೥9݄22೔౔༵೔
   

   View Slide

8. Local APIC
   CPUຖͷׂΓࠐΈίϯτϩʔϥɹׂΓࠐΈΛڐՄɾϚεΫͨ͠
   ΓɺEOIͨ͠Γ
   γεςϜશମͰҰҙͳAPIC IDΛ࣋ͭ
   ଞͷLAPIC΁ׂΓࠐΊΔʢIPI: Inter-processor Interrupt)
   CPU
   Local APIC
   CPU
   Local APIC
   CPU
   Local APIC
   ICH(South bridge)
   8259A PIC
   Timer
   I/O APIC
   Legacy
   PCI
   Devices
   MSI
   Capable
   Devices
   IPI
   MSI
   Legacy PCI
   8259A Intr
   12೥9݄22೔౔༵೔
   

   View Slide

9. I/O APIC
   ݶΒΕͨIRQΛෳ਺ͷPCIσόΠεͰڞ༗
   ֤IRQΛͲͷLAPIC΁సૹ͢Δ͔Λఆٛ͢ΔRedirection TableʢI/
   O APIC্ͷϨδελ܈ʣΛ࣋ͭ
   PCIσόΠε͸I/O APICΛ௨ͯ͡LAPIC΁ׂΓࠐΈϝοηʔδΛૹ৴
   CPU
   Local APIC
   CPU
   Local APIC
   CPU
   Local APIC
   ICH(South bridge)
   8259A PIC
   Timer
   I/O APIC
   Legacy
   PCI
   Devices
   MSI
   Capable
   Devices
   IPI
   MSI
   Legacy PCI
   8259A Intr
   12೥9݄22೔౔༵೔
   

   View Slide

10. MSI capable PCI devices
    ֤σόΠε͕೚ҙͷ਺ͷIRQΛ࣋ͯΔ
    ֤IRQͷׂΓࠐΈઌLAPIC͸PCIσόΠεͷPCI Configuration
    Spaceʹ࣋ͭ
    PCIσόΠε͸௚઀LAPIC΁ׂΓࠐΈϝοηʔδΛૹ৴
    CPU
    Local APIC
    CPU
    Local APIC
    CPU
    Local APIC
    ICH(South bridge)
    8259A PIC
    Timer
    I/O APIC
    Legacy
    PCI
    Devices
    MSI
    Capable
    Devices
    IPI
    MSI
    Legacy PCI
    8259A Intr
    12೥9݄22೔౔༵೔
    

    View Slide

11. ׂΓࠐΈઌͷࢦఆํ๏
    Physical Destination Mode
    Destination FieldʹAPIC IDΛࢦఆ
    ʮৗʹಉ͡CPU΁ׂΓࠐΈʯΛ࣮ݱ
    Logical Destination Mode(Flat Model)
    Destination FieldͷbitͰѼઌLAPIC܈ͷൣғΛදݱ
    Delivery Mode
    Fixed
    ࢦఆൣғͷશͯͷLAPIC΁ׂΓࠐΈ
    Lowest Priority
    ࢦఆൣғͷதͰɺ࠷΋TPRʢTask Priority Registerʣͷ
    ஋͕௿͍CPU΁ׂΓࠐΈ
    12೥9݄22೔౔༵೔
    

    View Slide

12. Lowest Priority Mode
    ࢦఆൣғͷதͰɺ࠷΋TPRʢTask
    Priority Registerʣͷ஋͕௿͍LAPIC
    ΁ׂΓࠐΈ
    ࠷খ஋ͷTPRΛ࣋ͭLAPIC͕ෳ਺͋Δ
    ৔߹͸ϥ΢ϯυϩϏϯͰ̍ͭબ୒
    12೥9݄22೔౔༵೔
    

    View Slide

13. echo 1 > /proc/irq//
    smp_affinityͨ࣌͠ʹԿ͕ى͖Δͷ͔
    MSIͳPCIeσόΠεͰ͔֬ΊͯΈΔ
    Intel Pro/1000(e1000e)
    PCI Configuration Space͸rootͳ
    Β/sys/bus/pci͔ΒಡΊΔ
    →Ϣʔβϥϯυ͔ΒಡΊΔ
    →΋͔ͯ͠͠ɿlspci
    12೥9݄22೔౔༵೔
    

    View Slide

14. ʊਓਓ ਓਓʊ
    ʼ lspci ʻ
    ʉY^Y^Y^Yʉ
    12೥9݄22೔౔༵೔
    

    View Slide

15. # lspci -vvvv -s 00:19.0
    00:19.0 Ethernet controller: Intel Corporation 82567LM Gigabit Network Connection (rev 03)
    Subsystem: Lenovo Device 20ee
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Latency: 0
    Interrupt: pin A routed to IRQ 44
    Region 0: Memory at f2600000 (32-bit, non-prefetchable) [size=128K]
    Region 1: Memory at f2625000 (32-bit, non-prefetchable) [size=4K]
    Region 2: I/O ports at 1840 [size=32]
    Capabilities: [c8] Power Management version 2
    Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
    Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
    Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Address: 00000000fee0300c Data: 41b9
    Capabilities: [e0] PCI Advanced Features
    AFCap: TP+ FLR+
    AFCtrl: FLR-
    AFStatus: TP-
    Kernel driver in use: e1000e
    Kernel modules: e1000e
    ੺͘ృͬͨॴ͕MSIͷϑΟʔϧυ͚ͩΕͲɺ͜
    Εͩͱྑ͘෼͔Βͳ͍ͷͰlspciΛforkͯ͠
    MSIϑΟʔϧυΛදࣔ͢ΔπʔϧΛ࡞੒
    https://gist.github.com/1568777
    12೥9݄22೔౔༵೔
    

    View Slide

16. # gcc -lpci msireg.c
    # ./a.out 00:19.0
    Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+
    address_hi=0
    address_lo=fee0300c dest_mode=logical redirection=lowpri dest_id=3
    data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185
    Logical modeͰLowpriɺdestid=3ɺ
    vector=185ʹͳͬͯΔ
    # echo 1 > /proc/irq/44/smp_affinity
    # ./a.out 00:19.0
    Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+
    address_hi=0
    address_lo=fee0100c dest_mode=logical redirection=lowpri dest_id=1
    data=41b9 trigger=edge level=assert delivery_mode=lowpri vector=185
    dest_id͕1ʹॻ͖׵Θͬͨʂ
    12೥9݄22೔౔༵೔
    

    View Slide

17. ༨ஊ
    ௐࢠʹ৐ͬͯϢʔβϥϯυ͔ΒϑΟʔ
    ϧυॻ͖׵͑ͨΒԿ͔͓͔͘͠ͳΓ·
    ͨ͠
    Χʔωϧ͞Μͱ݌՞ͯ͠Δ͔ɺϑΟʔ
    ϧυͷॻ͖ํؒҧ͔͑ͨ…
    12೥9݄22೔౔༵೔
    

    View Slide

18. Intel® 64 and IA-32 Architectures Software Developer
    Manuals
    Intel® I/O Controller Hub 10 (ICH10) Family Datasheet
    linux/Documentation/IRQ-affinity.txt
    Understanding the Linux Kernel, 3rd Edition
    PCI Local Bus Specification Revision 3.0
    ࠷ۙͷPCΞʔΩςΫνϟʹ͓͚ΔׂΓࠐΈϧʔςΟϯάͷ࢓૊Έ
    /proc/irq//smp_affinityͷॻ͖׵͑ͰPCIίϯϑΟάϨ
    ʔγϣϯۭؒ͸ͲͷΑ͏ʹॻ͖׵ΘΔ͔
    Interrupts on xv6
    ࢀߟࢿྉ
    12೥9݄22೔౔༵೔
    

    View Slide