UI testing improvements in Xcode 16
09 July 2024
While the new Swift Testing framework announced this year at WWDC24 is getting a lot of attention, there are some notable improvements coming to UI testing in XCTest in Xcode 16.
turing complete with a stack of 0xdeadbeef
While the new Swift Testing framework announced this year at WWDC24 is getting a lot of attention, there are some notable improvements coming to UI testing in XCTest in Xcode 16.
More and more Apple Platform developers are migrating away from CocoaPods in favor the Swift Package Manager, which is Apple’s first-party tool for managing and integrating dependencies. While it is still not quite a complete replacement for CocoaPods, it is getting closer. Unfortunately, SwiftPM’s integration with Xcode still has a number of shortcomings, even though it was introduced with Xcode 11 — 4 years ago. The worst bug is that Xcode frequently and randomly deletes the Package.resolved, which in turn produces dozens or hundreds of 'missing package product' errors. Here’s how I’ve worked around this bug on a team I work on.
At last year’s WWDC, Apple introduced privacy manifests. They recently sent out a reminder that the deadline for complying with these new requirements is May 1. Privacy manifests expand on the previously introduced privacy “nutrition labels” that are self-reported by developers and displayed on the App Store. Developers must start including a privacy manifest in their apps by the aforementioned deadline, but what’s more interesting is that Apple is, for the first time, imposing these new privacy rules on third-party SDKs as well. Even more interesting is the list of SDKs that Apple has published, which, upon inspection is quite bizarre.
While working on updating iOS screenshots for the App Store recently, I discovered that simctl status_bar is still broken. And unfortunately, I do not expect it to be fixed any time soon.
I recently discovered, while setting up my first Apple Silicon Mac, that Xcode does not have access to your shell environment. But there’s one caveat to that. (Thanks to Boris for confirming!) This post will hopefully be a reminder to my future self when I encounter this issue again.
After setting up my new M3 MacBook Pro, I decided to do some quick performance comparisons with my old Intel machine. Anecdotally, I would have told you that it is insanely faster but seeing the data made my jaw drop.
Xcode 15 introduces a new “bookmarks” feature, which lets you bookmark lines or entire files. It is a welcome change that has sherlocked my hack for using breakpoints as bookmarks.
Whether you are starting a new job or joining a new project, getting oriented in a new iOS codebase can be difficult and overwhelming. It is particularly hard if the codebase is very large, and especially challenging if you are early in your career and do not yet have much experience to draw from.
Large Xcode projects can be difficult to navigate, especially when you are making a large change across a large number of files. Depending on how your project is configured, modified files will be spread across multiple nested directories and multiple targets.
When debugging a large project in Xcode that a large team works on, the console can get quite busy. Logs are everywhere! It can be difficult to sift through the noise, particularly when you have a number of breakpoints configured to log messages, execute debugger commands, and continue after evaluating rather than pause.
In my previous post, I explained how to use symbolic breakpoints to discover when view controllers load their views into memory. Often breakpoints are specific to a project. You’ll create one for a specific class that only exists for that particular app. However, what I discussed in that post would be useful in any project. Unlike regular breakpoints, symbolic breakpoints (at least when set on system frameworks) are more or less universal.
While working on a very large iOS client project, I was investigating the causes for our slow app launch time. We had a hypothesis that part of the problem was that too many view controllers were getting loaded in memory, in particular, ones that were not even being presented to the user during app startup. What could cause view controllers to load too early? How might you discover this happening? And how do you fix it? Let’s find out.
On iOS the operating system employs a watchdog that monitors for and terminates unresponsive apps. If your app is blocking the main thread for too long, the system will kill it. In crash reports, you can identify watchdog terminations via the termination reason code 0x8badf00d (“ate bad food”).
If you are using fastlane to automate your release process, you might be using the increment_version_number and increment_build_number actions to bump your version and build numbers, respectively. However, if your Xcode project is configured to use xcconfig files, then you are out of luck. Shockingly, fastlane does not seem to support projects that use xcconfig files and there is a surprising dearth of information online about how to make fastlane work with Xcode build configuration files.
I previously wrote about writing a custom shell command to quickly switch between Xcodes. But recently, I needed to determine the version of Swift that is bundled with Xcode — specifically the version of Swift that is shipping with the current Xcode 13.3 beta. I was pretty sure that it is Swift 5.6, but I wanted to know for certain.
A few weeks ago, I wrote about this bizarre Xcode 13 crash when running tests. I just discovered the root cause for one of the issues I mentioned in that post — I think. At the very least, I have a “fix”. The issue happens when running unit tests. Sometimes the full test suite will complete, sometimes not, and then LLDB will crash. This occurs with all of my projects. It doesn’t seem to matter what I do, the crash always happens. It has been driving me crazy.
This post started out as a “how to” for SwiftUI, but as I started testing and verifying I realized it is just an Xcode 13 bug. Historically, if you wanted to restrict your iOS app to specific device orientations, you would check or uncheck the various “Device Orientation” options in your project settings. You can find these by selecting your Xcode Project > App Target > “General” tab.
I discovered a bug in Xcode 13 where tests crash for framework projects, preventing unit tests from successfully running on CI. The issue is due to some obscure code signing or debugger error that did not occur on Xcode 12. Fortunately, I have found a workaround.
I came across a situation today where I needed to run an iOS test suite for a Swift Package. Previously, this required you to have an Xcode project but it no longer does.
In last week’s issue of iOS Dev Weekly, Dave linked to this tweet from Mohammad Azam, which linked to this StackOverflow post on resetting your app between UI tests by completely deleting it. It’s a very clever idea! This post offers an improved version of the code and some thoughts on when to use this.
Lately, it feels like every few days someone is sharing a new Xcode tip on Twitter or on their blog. They range from hidden settings to features I simply never knew about. I started saving links and planned to add a new “Xcode tips” section to my TIL repo on GitHub to reference later. But as I started, I realized that the resulting markdown file would not be easily discoverable or shareable. I thought, wouldn’t it be nice if the iOS and macOS developer community had a single place to find and share Xcode tips?
As I continue to pursue Mac app development more seriously, I can build on and borrow from my many years of iOS experience. While many aspects of writing Mac apps are very similar to iOS, or at least somewhat familiar, other aspects are quite different. One of the big differences is testing, and deciding how many versions of macOS to support.
Xcode’s UI testing framework has had its ups and downs over the years. Most recently, it has been much more robust and reliable in my experience. However, tests still tend to flake sometimes. Here are some ways that I have been able to reduce flakiness in UI tests.
Xcode 12 was released and it includes a change to how tabs and navigation work. In Xcode 12, the tabs have their own tabs. It makes no sense to me. I know we are supposed to be nice to each other about software, but this new UI/UX is beyond incomprehensible. What made it worse is that this new “tabs within tabs” was the default setting (overriding preferences I had previously set) and I could not figure out how to restore the previous (desired) behavior.
The release notes for Xcode 12 beta state that the release “supports on-device debugging for iOS 9 and later, tvOS 9 and later, and watchOS 2 and later.” I am not sure if that means support for building and deploying for iOS 8 is completely removed, but it sounds like it. Who is still deploying to iOS 8, anyway?
I try to have only one Xcode installed at a time for simplicity and tidiness. But such a setup is rare as we often must manage stable releases and beta versions simultaneously.
I few months ago I wrote a script to override status bar display settings in the iOS simulator using the new simctl status_bar feature in Xcode 11. This was great, but it still required that you manually run the script after launching simulators. This was not ideal, as Dave pointed out in iOS Dev Weekly when he challenged me to automate this anytime a simulator launches.
For an iOS project that I am currently working on, I am implementing Dark Mode. The codebase is approaching 7 years old, it is mostly Swift with some legacy Objective-C, and it currently supports iOS 11 and above. Aside from the tedium of ensuring the updated colors are being used throughout the codebase, I expected this task to be straight-forward. However, there were some unanticipated issues.
I recently wrote about using SwiftPM instead of CocoaPods, which included a list of pros and cons. While working on one of my projects that is using SwiftPM, I realized another issue with how SwiftPM currently integrates with Xcode.
JetBrains recently released a new typeface for developers and I wanted to give it a try. I switched to JetBrains Mono in my two primary editors, Xcode and Sublime Text. Much to my surprise, I really enjoyed it. I think it is a great typeface. But I quickly discovered that I hate ligatures.
Last year Xcode 11 was released with integrated support for the Swift Package Manager. For a couple of small projects of mine, I decided to try using it to manage dependencies instead of CocoaPods. Overall, using SwiftPM was a great user experience, but (as expected) it has clear shortcomings due to its lack of maturity.
I recently discovered that unit tests and UI tests for a macOS Xcode project will fail with obscure error messages if the hardened runtime is enabled. It took me awhile to realize what the actual source of the problem was, because the error messages led me in the wrong direction. Hopefully this will save you some time.
Xcode has a great UI for setting and editing breakpoints. I use breakpoints all the time while working and debugging, but I want to share another, unconventional way that I use them.
I have started using GitHub Actions for CI on a new project as a replacement for my usual setup on Travis CI. It generally seems to be much faster and more reliable so far. It also has an equivalent feature set, as far as I can tell. But one issue that I have run into is selecting a specific Xcode version, which is a bit cumbersome and not fully documented.
I recently wrote about overriding iOS simulator status bar display settings using simctl status_bar. In that post I provided some ways we can improve the tool, but I realized we can do even better.
With version 11 of Xcode, the IDE ships with a new feature in the simctl tool that can override status bar values for iOS simulators. This allows you to take better screenshots for the App Store without having to worrying about the time, battery level, etc. It is a great improvement, but there are some significant shortcomings. I’ve written a script to fix at least some of those.
The other day I was debugging a crash in a UI test for an open pull request at work. The bug turned out to be extremely subtle and difficult to notice. I spent way too much time staring at the changes, trying to understand what was wrong. Let’s see if you can spot the error.
I recently discovered a preference in Xcode’s Navigation settings that makes the ‘Assistant Editor’ much more useful, especially when writing Swift.
When you file a radar for a bug on one of Apple’s platforms, you should (usually) always attach a sysdiagnose. A sysdiagnose provides a lot of helpful information for the person who is trying to understand how the bug happened. Amongst other things, it contains logs from various parts of the OS, and all recent crash logs. Without it, the person on the other end of your report inside Apple may not be of much help. On macOS running sysdiagnose is somewhat common, but what about iOS?
The Swift type-checker remains a performance bottleneck for compile times, though it has improved tremendously over the past two years. You could even say the type-checker has gone from being drunk to sober. To help users debug these issues, awhile back Jordan Rose added a frontend Swift compiler flag that would emit warnings in Xcode for functions that took too long to compile, or rather took too long to type-check. In Xcode 9, there’s a new, similar flag for checking expressions.
OCMock is a powerful mock object unit testing library for Objective-C. Even if you are using Swift, as long as your classes inherit from NSObject, you can use some of its features. But what if you are writing pure Swift code which does not have access to the dynamic Objective-C runtime? Or, what if you don’t want your Swift code to be hampered by NSObject subclasses and @objc annotations? Perhaps, you merely want to avoid dependencies and use ‘plain old’ XCTest with Objective-C. It’s relatively easy and lightweight to achieve the same effect in some testing scenarios without using OCMock.
I spent most of my free time last weekend and a few days of last week on migrating my Swift code to Swift 3.0 — I migrated my open source projects as well as my private side projects. Overall, I would say my experience was “OK”. It definitely could have been better, but I think the largest problem was overcoming the cognitive hurdle of seeing all the changes and errors from Xcode’s migration tool at once. The best thing to do is wipe away the tears, put your headphones on, and start hacking. 🤓
In iOS development, the core of nearly every app rests on the foundations provided by UICollectionView and UITableView. These APIs make it simple to build interfaces that display the data in our app, and allow us to easily interact with those data. Because they are so frequently used, it makes sense to optimize and refine how we use them — to reduce the boilerplate involved in setting them up, to make them testable, and more. With Swift, we have new ways with which we can approach these APIs and reimagine how we use them to build apps.
Surprisingly, I have not seen anyone talking about what I just discovered in the iOS 9.0 API Diffs. (Well, actually what Max von Webel discovered.) There’s a hidden gem in the UIKit diffs. We no longer have to suffer through tracking down obscure bugs due to non-zeroing weak references.
As I continue my work with Core Data and Swift, I have been trying to find ways to make Core Data better. Among my goals are clarity and safety, specifically regarding types. Luckily, we can harness Swift’s optionals, enums, and other features to make managed objects more robust and more clear. But even with the improvements that Swift brings, there are still some drawbacks and limitations with Xcode’s current toolset.
Core Data is probably loved as much as it is shunned by iOS developers. It is a framework of great power that often comes with great frustration. But it remains a popular tool among developers despite its pitfalls — likely because Apple continues to invest in it and encourages its adoption, as well as the availability of the many open-source libraries that make Core Data easier to use. Consider unit testing, and Core Data gets a bit more cumbersome. Luckily, there are established techniques to facilitate testing your models. Add Swift to this equation, and the learning curve gets slightly steeper.
When the App Store launched, there was one iPhone with one screen size and one pixel density. Designing your user interfaces was relatively simple and the technical debt of hard-coding them was cheap. Today, developers and designers face many challenges in creating apps that must work on dozens of different devices. Long gone are the days of 480x320. We can no longer depend on physical screen sizes and must always be prepared for the next generation of devices.
When I find my code is slow or troubled, friends and colleagues comfort me. Speaking words of wisdom, write in C. It is understood that foregoing the features and abstractions of high-level programming languages in favor of their low-level counterparts can yield faster, more efficient code. If you abandon your favorite runtime, forget about garbage collection, eschew dynamic typing, and leave message passing behind; then you will be left with scalar operations, manual memory management, and raw pointers. However, the closer we get to the hardware, the further we get from readability, safety, and maintainability.
As Apples to apples, Part II made its way around the web, it was praised as well as critiqued. The latter largely consisted of questions regarding the real-world applications of these benchmarks. In general, benchmarks should be taken with a grain of salt. I want to take a minute to clarify my thoughts on benchmarks and how I think they can be valuable.
If at first you don’t succeed, try, try again. Practice makes perfect. These proverbs have encouraged us all in many different contexts. But in software development, they tug at our heartstrings uniquely. Programmers persevere through countless nights of fixing bugs. Companies march vigilantly toward an MVP. But after 1.0 there is no finish line, there is no bottom of the 9th inning. There are more bugs to be fixed. There are new releases ahead. The march continues, because software is not a product, it is a process.
When Craig Federighi arrived at his presentation slide about Objective-C during this year’s WWDC keynote everyone in the room seemed puzzled, curious, and maybe even a bit uneasy. What was happening? As he continued, he considered what Objective-C would be like without the C, and the room abruptly filled with rumblings and whispers [1] as developers in the audience confided in those around them. If you had been following the discussions in our community about the state of Objective-C (and why we need to replace it) during the previous months, you could only have imagined one thing: Objective-C was no more — at least not as we knew it.