In 2017, I wrote an article titled Keeping XCTest in sync on Linux. In it, I proposed to add some code to your test classes that would alert you when you forget to update the allTests array after adding a test method.

Swift 5.1 ships with a much better solution: automatic test discovery on non-Apple platforms, such as Linux and Windows.

This feature is not yet enabled by default. You have to pass the flag explicitly for now:

swift test --enable-test-discovery

While XCTest on Apple platforms uses the Objective-C runtime to find test methods, this command looks them up in an index created by the compiler during build time. (This index also powers SourceKit-LSP.)

How to test your SwiftPM package on Linux from macOS

Check out Testing Swift packages on Linux using Docker.

Adopting automatic test discovery

I found two open bugs (SR-11951 and SR-12008) that suggest that the test discovery doesn’t work correctly with certain test case setups. I suppose this is (part of) the reason why it’s not yet enabled by default, even in the latest Swift 5.2 nightly builds.

You should confirm that the automatic test discovery finds all test methods in your package. Diffing the outputs of swift test --list-tests and swift test --enable-test-discovery --list-tests should not show any differences. If you find a problem that isn’t covered by an existing bug, file one.

Deleting allTests

If everything works and if your package can drop support for (being tested on) older Swift versions, switching to automatic test discovery allows you to delete a bunch of boilerplate from your test classes:

1. You can delete all allTests properties in your test classes.

2. You can delete all XCTestManifests.swift files (there should be one per test target).

3. Do not delete Tests/LinuxMain.swift. It must continue to exist for now, otherwise swift build on Linux will fail¹. I suggest you replace the contents with this line:

   // LinuxMain.swift
   fatalError("Run the tests with `swift test --enable-test-discovery`.")
   

   When someone now runs swift test on Linux without passing the --enable-test-discovery flag, they will see an immediate crash with a meaningful error message.