OpenRISC Toolchains

In this talk I give an update on the state of the OpenRISC GNU and LLVM toolchains, joint-presented with Jeremy Bennett with information on how to test the various toolchains.

Transcript

1. OpenRISC Toolchains Simon Cook Jeremy Bennett Ed Jones
2. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license GNU Toolchain • GCC – 4.9 on GitHub – In need of review for upstreaming • Binutils/sourceware – Up-to-date version on GitHub – Upstreamed • GDB – Pre-git version • Various libcs – Newlib, uClibc, musl, ...
3. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license LLVM Toolchain • LLVM/Clang – 3.5 on GitHub • LLDB – Possibly starting port • Compiler-RT – No longer need libgcc for C runtime components • Use of GNU Tools – Can use clang instead of gas – Use GNU linker, binutils, debugger
4. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Regression Testing Test: GCC Testsuite as of OpenRISC master Platform: or1ksim, GCC 4.x === gcc Summary === # of expected passes 81236 # of unexpected failures 508 # of unexpected successes 1 # of expected failures 94 # of untested testcases 0 # of unresolved testcases 420 # of unsupported tests 1584
5. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Regression Testing Test: GCC Testsuite as of OpenRISC master Platform: or1ksim, LLVM 3.5 === gcc Summary === # of expected passes 39883 # of unexpected failures* 18464 # of unexpected successes 15 # of expected failures 28 # of untested testcases 14581 # of unresolved testcases 9114 # of unsupported tests 1758 * currently in process of LLVMifying GCC regression, which will remove most failures
6. Regression Testing the OpenRISC Tool Chain Jeremy Bennett
7. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Agenda • About regression testing – OR1K test configurations • OR1K regression testing in the past – GCC 4.5.1 tool chain – parallel testing – verifying the Verilog • OR1K regression testing in the future – GCC 4.9 testing for upstreaming – LLVM testing
8. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license What is DejaGnu • A POSIX 1003.3 compliant test framework – Written in expect, an extension of TCL • Typical directory structure – config: expect config for each board to be tested – lib: expect config for each tool to be tested – toolname-suffix.tests • test files: testname.exp • Top level command is runtest • More info: – www.embecosm.com/resources/appnotes/#EAN8
9. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license GNU Tool Chain Regression Testing • Tests to ensure past bugs never recur – approx 75,000 C tests, 50,000 C++ tests – futher tests for libstdc++-v3, binutils and GDB • Running tests – all C/C++ tests make check-gcc – just the C torture execution tests • make RUNTESTFLAGS="execute.exp" check-gcc – just one execution test • make RUNTESTFLAGS="execute.exp=vrp-1.c" check-gcc • Can use -j flag to run in parallel
10. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Testing with Or1ksim Or1ksim as standalone simulator DejaGnu load_generic_config "sim" load_generic_config "sim"
11. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Testing via GDB GDB RSP Server I/F TCP/IP RSP USB JTAG DejaGnu GDB RSP Server TCP/IP RSP Custom DejaGnu Verilator Model GDB Or1ksim with RSP Server TCP/IP RSP DejaGnu load_generic_config "gdb_stub" load_generic_config "gdb_stub"
12. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Testing Linux Application Tool Chains GDB gdbserver on Linux TCP/IP RSP DejaGnu load_generic_config "gdb_stub" load_generic_config "gdb_stub" Linux TCP/IP telnet & FTP DejaGnu load_generic_config "unix" load_generic_config "unix"
13. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license GCC 4.5.1 • Developed for NASA's TechEdSat – Requirement for robustness – Goal: no regression failures for binutils or GCC • Results – goals met for binutils and GCC • both ELF and Linux toolchains – not met for GDB – key success was robust C++ support • Challenge – verifying the Linux tool chain while kernel port in development –
14. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Parallel Testing for Speed and Robustness Linux on Or1ksim TCP/IP telnet & FTP DejaGnu Linux on Or1ksim Linux on Or1ksim • Each target has its own port – selected in board description file • Script calls runtest explicitly – break into 128 threads • Restart thread if target is dead. • Total 2 hours for full regression test – 16 Or1ksim instances on 3 machines – 3-4 alive at the end
15. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Verifying the Verilog GDB RSP Server I/F TCP/IP RSP USB JTAG DejaGnu GDB RSP Server TCP/IP RSP Custom DejaGnu Verilator Model GDB Or1ksim with RSP Server TCP/IP RSP DejaGnu
16. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license The Future • GCC upstreaming – clean regression is a major requirement • LLVM testing – relies on GCC 4.2 regression tests – many are not suitable – Embecosm are working on solving this • way to specify irrelevant tests expected to fail • goal to use any GCC regression tests – issue of LLVM upstream requirements
17. Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Copyright © 2014 Embecosm. Freely available under a Creative Commons license Toolchain TODO • 64-bit variant • More testing/bug fixing • Testing on verilog implementations • Binaries for various platforms
18. Toolchain BoF