No NMI? No Problem! – Implementing Arm64 Pseudo-NMI

As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread.

AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU.

This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it.

Julien Thierry, ARM

Transcript

1. ex-Arm Ltd., Kernel Team No NMI? No Problem! - Implemen ng Arm64 Pseudo-NMI Julien Thierry <jthierry@redhat.com> September 24, 2019 © 2019 Arm Limited
2. About Myself & the talk Joined Arm in June 2017 Linux Kernel team Cambridge, UK First kernel related job Le in August 2019 Work presented here done while employed by Arm 2 © 2019 Arm Limited
3. What are IRQs? Manifesta on of event in the system: device or program triggered Interrupts ac ve flow of execu on Managed with some primi ves: local_irq_disable/enable() local_irq_save/restore() 3 © 2019 Arm Limited
4. What are NMIs? Non-Maskable Interrupt Interrupts that can happen while interrupts are disabled Generally cannot be disabled locally Used for repor ng hardware status, mer expira on, profiling, ... x86: Dedicated excep on entry for NMIs SPARC: Interrupts with highest possible priority 4 © 2019 Arm Limited
5. NMI Handlers Dedicated context: nmi_enter() nmi_exit() Inform system wide features (e.g. printk, race, rcu, ...) Restric ons: No NMI nes ng No preemp on Keeps NMI handlers simpler 5 © 2019 Arm Limited
6. NMI Handlers and Locks void inc_counter ( void ) { unsigned long f l a g s ; f l a g s = r a w _ s p i n _ l o c k _ i r q s a v e (& count_lock ) ; gl o b a l _c o u n t e r += 1; r a w _ s p i n _ u n l o c k _ i r q r e s t o r e (& count_lock , f l a g s ) ; } 6 © 2019 Arm Limited
7. NMI Handlers and Locks void inc_counter ( void ) { unsigned long f l a g s ; f l a g s = r a w _ s p i n _ l o c k _ i r q s a v e (& count_lock ) ; gl o b a l _c o u n t e r += 1; r a w _ s p i n _ u n l o c k _ i r q r e s t o r e (& count_lock , f l a g s ) ; } Lock cannot protect against NMI Can protect from concurrency (SMP) → mutex 6 © 2019 Arm Limited
8. The Perf Case Demo 7 © 2019 Arm Limited
9. The Perf Case Perf event handler + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | | | | IRQ | | | | | | - - -| - - - - - - - - - - - - - - - - - - - - - - -| - - - - - - - - - - - - - - -| irq_disable() perf event irq_enable() (in random location) 8 © 2019 Arm Limited
10. Pseudo-NMI for Arm64 No architected NMI 9 © 2019 Arm Limited
11. Pseudo-NMI for Arm64 No architected NMI But has IRQ priori es and priority masking Emulate NMI’s behaviour using an IRQ Dis nguish NMI from normal IRQ: two dis nct priori es Block IRQs while allowing NMIs: mask priority of normal IRQs 9 © 2019 Arm Limited
12. Arm64 CPU Interrupt Control +---------------+ | | | CPU | | | | +--------+ | | | PSTATE | | IRQ | | | | - - - - - - - - - - - - - - - >| I | | | | | | | | | | | | | | | +--------+ | | | | | +---------------+ 10 © 2019 Arm Limited
13. Arm64 CPU Interrupt Control PSTATE.I bit Toggled by local_irq_enable/disable() Prevents CPU to jump to interrupt vector Single bit → only binary state 11 © 2019 Arm Limited
14. Arm64 CPU Interrupt Control +---------------+ | | | CPU | | | | +--------+ | | | PSTATE | | IRQ | | | | - - - - - - - - - - - - - - - >| I | | | | | | | | | | - - - - - - - - - - - - - - - >| F | | FIQ | +--------+ | | | | | +---------------+ 12 © 2019 Arm Limited
15. More control with the GIC +---------------+ | | | | +-----------------+ +---------------+ | | | | | | | | | GIC CPU | | CPU | | | | Interface | | | | | | | | +--------+ | | Generic | | +--------+ | | | PSTATE | | | | | | | | IRQ | | | | | Interrupt | - - - - - - - - - - - - - - - - - - - - - - >| | - - -| - - - - - - - - - - - - - - - - - - - - - - >| I | | | | | | P M R | | | | | | | Controller | | | | | | | | | | | | | | | | | | | | | | | | | | +--------+ | | | | +--------+ | | | | | | | | | | | +-----------------+ +---------------+ | | | | | | | | +---------------+ 13 © 2019 Arm Limited
16. More control with the GIC +---------------+ + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+ | | | | | | | +-----------------+ +---------------+ | | | | | | | | | | | | | GIC CPU | | CPU | | | | | | Interface | | | | | | | | | | +--------+ | | | Generic | | | +--------+ | | | PSTATE | | | | | | | | | | IRQ | | | | | | Interrupt | - - - - - - - - - - - - - - - - - - - - - - >| | - - -| - - - - - - - - - - - - - - - - - - - - - - >| I | | | | | | | | P M R | | | | | | | | Controller | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +--------+ | | | | | | +--------+ | | | | | | | | | | | | | | | +-----------------+ +---------------+ | | | | | | | + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+ | | Per CPU | | +---------------+ 14 © 2019 Arm Limited
17. Fi ng priori es in linux Stop touching PSTATE.I bit Mask IRQs using PMR in local_irq_*() Configure NMIs with high enough priority in GIC 15 © 2019 Arm Limited
18. Fi ng priori es in linux Stop touching PSTATE.I bit Mask IRQs using PMR in local_irq_*() Configure NMIs with high enough priority in GIC But: Upon IRQ, interrupts are automa cally disabled (PSTATE.I) Some contexts (idle, KVM guests, ...) cannot use PMR for IRQ disabling Need to take care to have consistent state when reaching generic code 15 © 2019 Arm Limited
19. Using the NMI Requester needs to set up target IRQ priority through irqchip Ini al proposal: introduce new NMI state in enum irqchip_irq_state 16 © 2019 Arm Limited
20. Using the NMI Requester needs to set up target IRQ priority through irqchip Ini al proposal: introduce new NMI state in enum irqchip_irq_state Not so popular: Adding NMI d e l i v e r y support at low l e v e l a r c h i t e c t u r e i r q chip l e v e l i s p e r f e c t l y fine , but the exposure of that needs to be r e s t r i c t e d very much . Adding i t to the g e n e r i c i n t e r r u p t c o n t r o l i n t e r f a c e s i s not going to happen . That ’ s doomed to begin with and a complete abuse of the i n t e r f a c e as the handler can not ever be used f o r that . Thanks , t g l x 16 © 2019 Arm Limited
21. NMI API include/linux/interrupt.h Provide NMI API similar to the IRQ one: request_nmi() / free_nmi() enable_nmi() / disable_nmi() irq_chip changes Flag to adver se NMI support chip->irq_nmi_setup() / chip->irq_nmi_teardown() Not exported to modules (Per-cpu NMI variants available) 17 © 2019 Arm Limited
22. Results on Perf Demo 18 © 2019 Arm Limited
23. Timeline Work ini ated as by Daniel Thompson from Linaro: 19 © 2019 Arm Limited
24. Timeline Work ini ated as by Daniel Thompson from Linaro: 2015-03-18: RFC: Pseudo-NMI for arm64 using ICC_PMR_EL1 (GICv3) 2016-08-19: RFC v3, v4.8-rc2 7 patches Uses PMR for IRQ enabling/disabling Hardcoded NMI priority for backtrace IPI ”The code works-for-me (tm) and is more ”real” than the last me I shared these patches.” 19 © 2019 Arm Limited
25. Timeline Work ini ated as by Daniel Thompson from Linaro: 2015-03-18: RFC: Pseudo-NMI for arm64 using ICC_PMR_EL1 (GICv3) 2016-08-19: RFC v3, v4.8-rc2 7 patches Uses PMR for IRQ enabling/disabling Hardcoded NMI priority for backtrace IPI ”The code works-for-me (tm) and is more ”real” than the last me I shared these patches.” 2017-07 or 2017-08: I start working on it 19 © 2019 Arm Limited
26. Timeline Work ini ated as by Daniel Thompson from Linaro: 2015-03-18: RFC: Pseudo-NMI for arm64 using ICC_PMR_EL1 (GICv3) 2016-08-19: RFC v3, v4.8-rc2 7 patches Uses PMR for IRQ enabling/disabling Hardcoded NMI priority for backtrace IPI ”The code works-for-me (tm) and is more ”real” than the last me I shared these patches.” 2017-07 or 2017-08: I start working on it 2017-10-11: RFC: arm64: provide pseudo NMI with GICv3, v4.15-rc2 7 patches Tes ng with perf interrupt dropped backtrace IPI 19 © 2019 Arm Limited
27. Timeline 2018-05-21: V3, v4.17-rc6 6 patches 20 © 2019 Arm Limited
28. Timeline 2018-05-21: V3, v4.17-rc6 6 patches Review As i t is , t h i s patch i s almost i mp oss i b l e to review . I t turns the i n t e r r u p t masking upside down , messes with the GIC , hacks KVM . . . Too many t h i n g s change at once , and I f i n d i t very hard to b u i l d a mental p i c t u r e of the changes j u s t by s t a r i n g at i t . [ . . . ] Thanks , M. 20 © 2019 Arm Limited
29. Timeline 2018-05-25: V4 26 patches 21 © 2019 Arm Limited
30. Timeline 2018-05-25: V4 26 patches 2018-08-28: V5, v4.19-rc1 Depends on separate API for NMIs series (4 patches) 21 © 2019 Arm Limited
31. Timeline 2018-05-25: V4 26 patches 2018-08-28: V5, v4.19-rc1 Depends on separate API for NMIs series (4 patches) 2019-01-31: V10 25 patches 21 © 2019 Arm Limited
32. Timeline 2018-05-25: V4 26 patches 2018-08-28: V5, v4.19-rc1 Depends on separate API for NMIs series (4 patches) 2019-01-31: V10 25 patches 2019-02-06: Merge of v10 + v6 of NMI API in 5.1 21 © 2019 Arm Limited
33. Timeline 22 © 2019 Arm Limited
34. Timeline 2019-03-29: Bug: System hangs when using Ftrace graph tracer 22 © 2019 Arm Limited
35. Timeline 2019-03-29: Bug: System hangs when using Ftrace graph tracer 2019-06-21: Fixed in 5.3 (V4 of 8 patches series) 22 © 2019 Arm Limited
36. Try it, buy it Get Aarch64 capable pla orm with GICv3 Get Linux v5.3+ sources Kernel Features → Support for NMI-like interrupts (CONFIG_ARM64_PSEUDO_NMI) Boot op on irqchip.gicv3_pseudo_nmi=1 arm_pmu patches using NMI (V4) S ll needs an update 23 © 2019 Arm Limited
37. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver 24 © 2019 Arm Limited
38. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver Look into using NMI API for SPARC NMIs 24 © 2019 Arm Limited
39. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver Look into using NMI API for SPARC NMIs No plan for GICv2 support Prior to GICv3, CPU interface is memory mapped 24 © 2019 Arm Limited
40. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver Look into using NMI API for SPARC NMIs No plan for GICv2 support Prior to GICv3, CPU interface is memory mapped Slower accesses than for system register 24 © 2019 Arm Limited
41. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver Look into using NMI API for SPARC NMIs No plan for GICv2 support Prior to GICv3, CPU interface is memory mapped Slower accesses than for system register Need access to per-cpu base address from local_irq_*() 24 © 2019 Arm Limited
42. Next steps Use NMI for more features on Arm64 Backtrace, CPU stop IPI: requires moving IPIs to use IRQ framework Hard lockup detector: need some rework of arm_pmu driver Look into using NMI API for SPARC NMIs No plan for GICv2 support Prior to GICv3, CPU interface is memory mapped Slower accesses than for system register Need access to per-cpu base address from local_irq_*() Technically do-able, but not necessarily clean nor interes ng/usable 24 © 2019 Arm Limited
43. Thanks! The Arm trademarks featured in this presenta on are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respec ve owners. www.arm.com/company/policies/trademarks © 2019 Arm Limited
44. Ques ons 26 © 2019 Arm Limited