Using the GCC regression test suite for LLVM (and other compilers) Simon Cook Copyright © 2018 Embecosm. Freely available under a Creative Commons license. 

● First spoke about this at GNU Tools Cauldron 2015 – Jeremy Bennett: Keeping other compilers honest ● Aim to use relevant tests from GCC test suite to validate ports of LLVM (and other compilers) – Mainly gcc.c-torture and gcc.dg suites Background 

● Given a list of known incompatible tests, either: – Move incompatible tests from the source tree; and/or – Provide list to DejaGNU to mark tests as XFAIL ● List generated from running complete test run against golden reference of target compiler ● Pros: Quick path to 0 unexpected results, fast test execution time ● Cons: Relies on correctness of golden reference In 2015 - Blacklisting 

Annotate all tests with tests to check for GCC features other compilers may not support Example __attribute__((leaf)) proc check_effective_target_leaf {} { return [check_no_compiler_messages leaf assembly { void __attribute__((leaf)) foo (void) {} }] } Annotate test with /* { dg-require-effective-target leaf } */ Feature Tests 

Python script to manage execution of tests without build directory Similar to what make check-gcc does, but also: 1. Load balances each thread based on test count 2. Sets up any gdbservers / external tools needed for each test thread Currently configured for a given target, but plan is to load that configuration in at test time Test Runner 

Embecosm Build Server https://buildbot.embecosm.com/view/LLVM-GCC/ 

Test suite available at https://github.com/embecosm/gcc-for-llvm-testing Latest test results: Ongoing work Plan is to finish annotating tests, then fix bugs test suite exposes Eventually, plan to merge changes upstream Current Status RISC-V # of expected passes 27750 # of unexpected failures 4083 # of unexpected successes 2 # of expected failures 14 # of unresolved testcases 1525 # of unsupported tests 3272 x86_64 # of expected passes 30882 # of unexpected failures 3016 # of unexpected successes 2 # of expected failures 15 # of unresolved testcases 183 # of unsupported tests 2956 

Thank you Copyright © 2018 Embecosm. Freely available under a Creative Commons license.