Apples to apples, Part II

If at first you don’t succeed, try, try again. Practice makes perfect. These proverbs have encouraged us all in many different contexts. But in software development, they tug at our heartstrings uniquely. Programmers persevere through countless nights of fixing bugs. Companies march vigilantly toward an MVP. But after 1.0 there is no finish line, there is no bottom of the 9th inning. There are more bugs to be fixed. There are new releases ahead. The march continues, because software is not a product, it is a process.

This week, Apple has reminded us of the value of this iterative process and its rewards with the fifth beta release of Xcode 6, iOS 8, OS X Yosemite, and most importantly — Swift. This update includes a number of improvements, but perhaps the most interesting are those not listed. Swift was rough around the edges during its launch at WWDC, but it is definitely beginning to live up to its name.

If you missed my first post, Apples to Apples, you should head over there now to catch up.

Setup

The setup is the same as before, but I’ll reiterate for clarity.

• Code: swift-sorts, objc-sorts, and std lib sort implementations
• Software: OS X Mavericks 10.9.4, Xcode 6 beta 5
• Hardware: 2008 unibody MacBook Pro, 2.4 Ghz Intel Core 2 Duo, 8 GB 1067 MHz DDR3 memory

The benchmarks consist of T trials, which are averaged at the end to obtain the average execution time for each algorithm. Each trial begins by generating an array of N random integers in the range [0, UINT32_MAX). Then, each sorting algorithm is passed a copy of this initial array to sort. The current time is logged before and after each sort and the difference between the two yields the execution time for the algorithm for the current trial.

These two programs were carefully crafted to be a true apples-to-apples comparison. All of the algorithms, as well as main.swift and main.m, are implemented as similarly as possible, bounded only by the confines and paradigms of the languages themselves. In Objective-C, NSArray and NSNumber are used intentionally as the counterparts to Swift's Array and Int. The APIs are language-specific too, for example exchangeObjectAtIndex: withObjectAtIndex: versus swap().

Results

Below are the results of running each program over 10 trials with 10,000 integers. The build configuration settings are noted for each run and the average execution times are displayed in seconds. The average case runtime complexity for each algorithm is also noted.

The final row in each table is the difference in speed of Swift compared to Objective-C. A positive value indicates that Swift is faster, while a negative value indicates that Swift is slower. For example, if Objective-C runs in 3.6 seconds and Swift runs in 1.2 seconds, then Swift is 3 times (3x) faster.

Table 1

When not optimized, Swift is nothing to write home about. You can see that Objective-C performs substantially better. But this is ok — since you will be shipping with optimized builds. With no optimizations, this is what we should expect to see. The swiftness of Swift is in the compiler — LLVM and Clang. Mike Ash’s Friday Q&A last month on the Secrets of Swift’s Speed provides a good overview of how better performance can be achieved with Swift versus Objective-C with compiler optimizations. We can see these optimizations in action below.

Table 2

If you recall the results from the previous post, then this should be quite shocking (in a good way). Take a deep breath. Yes, yes this is real life. The tables have completely turned (no pun intended). I’ve been running these trials since the first beta, and this is the first time that Swift has performed better than Objective-C for every single algorithm, with standard optimizations. And not only is Swift faster, but it is faster by significant margins.

Table 3

As of beta 5, the Swift optimization level of -Ofast has been deprecated in favor -Ounchecked. This renaming is a delightful change. Previously it read as, "Ooooh fast!", begging for misuse and misunderstanding. But now it's more like "Oh... unchecked?", which better reflects its unsafe nature.

This should come as no surprise. Swift performance at this optimization level was always better than Objective-C, with the exception of the std lib sort — which no longer the case.

The benchmarks above were gathered with N = 10,000 to be consistent with the previous post. However, this is no longer a challenge for Swift. Let’s see what happens when N = 100,000. Given that most, if not all, developers will be compliling their modules and apps at the standard optimization level, the trials below were only run with -O and release. As expected, Swift comes out on top.

Table 4

Full speed ahead

When Apple introduced Swift, we were assured safety, clarity, modernity, and speed. It is clear to me that they have delivered and are continuing to deliver on these promises. The refinements and enhancements made over the past few months have been absolutely great. Some highlights for me include array value semantics, array and dictionary syntactic sugar, the ..< operator replacing the .. operator, and the performance improvements seen here. I think Swift is coming along quite nicely and I am more excited than ever for the next beta.

Swift is a breath of fresh air that makes reading and writing Objective-C feel archaic. I cannot wait for 1.0 and the moment when I can say goodbye to Objective-C.

The Apples to apples series will continue as betas are released. Stay tuned for updates and new posts!