UI testing improvements in Xcode 16
09 July 2024
While the new Swift Testing framework announced this year at WWDC24 is getting a lot of attention, there are some notable improvements coming to UI testing in XCTest in Xcode 16.
turing complete with a stack of 0xdeadbeef
While the new Swift Testing framework announced this year at WWDC24 is getting a lot of attention, there are some notable improvements coming to UI testing in XCTest in Xcode 16.
I’m happy to share that Nimble 10.0 has been released. It includes a number of fixes, improvements, and new features. It joins the latest release of Quick. For both of these libraries, these are the first releases in almost a year.
If you develop for Apple platforms and use a third-party testing framework, you are very likely using Quick and Nimble. Otherwise, you have probably at least heard of them. I don’t use any third-party testing frameworks in my personal projects, but I have worked on teams that do. And I currently work on a team that uses Quick and Nimble, which is why it was important for me to get a recent critical bug fixed in Quick.
I discovered a bug in Xcode 13 where tests crash for framework projects, preventing unit tests from successfully running on CI. The issue is due to some obscure code signing or debugger error that did not occur on Xcode 12. Fortunately, I have found a workaround.
As I continue to pursue Mac app development more seriously, I can build on and borrow from my many years of iOS experience. While many aspects of writing Mac apps are very similar to iOS, or at least somewhat familiar, other aspects are quite different. One of the big differences is testing, and deciding how many versions of macOS to support.
Xcode’s UI testing framework has had its ups and downs over the years. Most recently, it has been much more robust and reliable in my experience. However, tests still tend to flake sometimes. Here are some ways that I have been able to reduce flakiness in UI tests.
I recently discovered that unit tests and UI tests for a macOS Xcode project will fail with obscure error messages if the hardened runtime is enabled. It took me awhile to realize what the actual source of the problem was, because the error messages led me in the wrong direction. Hopefully this will save you some time.
OCMock is a powerful mock object unit testing library for Objective-C. Even if you are using Swift, as long as your classes inherit from NSObject, you can use some of its features. But what if you are writing pure Swift code which does not have access to the dynamic Objective-C runtime? Or, what if you don’t want your Swift code to be hampered by NSObject subclasses and @objc annotations? Perhaps, you merely want to avoid dependencies and use ‘plain old’ XCTest with Objective-C. It’s relatively easy and lightweight to achieve the same effect in some testing scenarios without using OCMock.
Core Data is probably loved as much as it is shunned by iOS developers. It is a framework of great power that often comes with great frustration. But it remains a popular tool among developers despite its pitfalls — likely because Apple continues to invest in it and encourages its adoption, as well as the availability of the many open-source libraries that make Core Data easier to use. Consider unit testing, and Core Data gets a bit more cumbersome. Luckily, there are established techniques to facilitate testing your models. Add Swift to this equation, and the learning curve gets slightly steeper.