In 2017, I wrote an article titled Keeping XCTest in sync on Linux. In it, I proposed to add some code to your test classes that would alert you when you forget to update the
allTests array after adding a test method.
Swift 5.1 ships with a much better solution: automatic test discovery on non-Apple platforms, such as Linux and Windows.
This feature is not yet enabled by default. You have to pass the flag explicitly for now:
swift test --enable-test-discovery
While XCTest on Apple platforms uses the Objective-C runtime to find test methods, this command looks them up in an index created by the compiler during build time. (This index also powers SourceKit-LSP.)
How to test your SwiftPM package on Linux from macOS
Check out Testing Swift packages on Linux using Docker.
Adopting automatic test discovery
I found two open bugs (SR-11951 and SR-12008) that suggest that the test discovery doesn’t work correctly with certain test case setups. I suppose this is (part of) the reason why it’s not yet enabled by default, even in the latest Swift 5.2 nightly builds.
You should confirm that the automatic test discovery finds all test methods in your package. Diffing the outputs of
swift test --list-tests and
swift test --enable-test-discovery --list-tests should not show any differences. If you find a problem that isn’t covered by an existing bug, file one.
If everything works and if your package can drop support for (being tested on) older Swift versions, switching to automatic test discovery allows you to delete a bunch of boilerplate from your test classes:
You can delete all
allTestsproperties in your test classes.
You can delete all
XCTestManifests.swiftfiles (there should be one per test target).
Do not delete
Tests/LinuxMain.swift. It must continue to exist for now, otherwise
swift buildon Linux will fail1. I suggest you replace the contents with this line:
// LinuxMain.swift fatalError("Run the tests with `swift test --enable-test-discovery`.")
When someone now runs
swift teston Linux without passing the
--enable-test-discoveryflag, they will see an immediate crash with a meaningful error message.