There has been a ton of debate on the swift-evolution mailing lists about access control in Swift. A couple of days ago, the proposal SE-0159: Fix Private Access Levels was rejected. I want to share my thoughts on this, as well as thoughts on the larger story for access control in general. But first, let’s begin with a brief history of access control in Swift.
⚠ Warning: some opinions are forthcoming. 😄
A brief history of access control
In the early days of Swift — pre-1.0 — there were no access controls. These were the golden days of Swift. Everything was public and globally accessible from anywhere. No one had to think about proper encapsulation. There were no month-long email debates (because swift-evolution didn’t exist yet). No access controls were the simplest access controls, and no evolution was the best evolution. 😉
Access control arrives
In Xcode 6 beta 4, Swift added support for access control. It was easy to understand, easy to use, and quite elegant. Shortly after, Swift 1.0 was officially released with this access control model. Swift 1.0 was bundled with Xcode 6. There was no “protected” access level, which conflates access with inheritance. There was no “friend” access level, because that’s just gross. There were only three access levels:
publicentities were accessible from any file that imports the module (a framework or library)internal(the default) entities were only accessible within their current module (an app or framework target, also think “current directory” à la Swift Package Manager)privateentities were only accessible from within the source file where they were defined
As a library author and app developer, these access levels provide all the tools necessary for me to express anything I want. Aside from the obvious use cases, one can achieve a “traditional” notion of “private” access by simply defining a type or other entities in their own files. For example, if I define class A and want all of its (private) properties to be inaccessible by other classes, then I can define class A in its own file with nothing else. This is what I love about Swift access control — it encourages best practices (reducing file bloat and module bloat) by providing access levels that basically reflect the “physical” structure of files and directories on disk. Designing proper encapsulation means you have to move files into clearly defined modules (directories) and you have to define related types in a single file or completely avoid multiple type definitions in a single file. Code organization and access control are nicely coupled and encourage developers to keep code well-organized. (Coupling is typically bad in software design, but in this case the coupling is desirable.)
Introducing fileprivate
The next phase began when Swift was open sourced and the Swift Evolution Process was introduced. The proposal SE-0025: Scoped Access Level was reviewed, revised, and finally accepted for Swift 3. This proposal changed the meaning of private to restrict access to an entity to within the current scope or declaration, and preserved the former meaning with a new keyword, fileprivate. The hypothesis at the time was that the new (and somewhat intentionally ugly) fileprivate keyword would rarely be used, thus abiding by Swift’s design philosophy of progressive disclosure. Little did we know that this would not be true in practice. Another side effect was that cognitive load increased, due to the overloaded term “private” and the overlapping functionality of fileprivate and private.
Out in the open
A mere three months later, the initial discussions began for another change to access control. After three controversial review periods and revisions, proposal SE-0117: Allow distinguishing between public access and public overridability was accepted for Swift 3. The proposal introduced a new access level called open and changed the definition of public in some contexts. The meaning of public narrowed regarding subclassing and overriding. A public class can no longer be subclassed outside of the module in which it is defined and any public members of a class can no longer be overridden by a subclass outside of the class’s module. Thus, open classes are public and also subclassable and any open property or function on an open class is overridable by subclasses. Did you follow that? The rules regarding public and open are a labyrinth and are further complicated by final, which prevents overriding and subclassing. Thus, final public and public classes have different semantics depending on whether or not a client is within the same module or outside of it. Again, the cognitive load increased when thinking about access control. This time, the overlapping functionality of open and public was even more pronounced and harder to discern.
Despite the complexity of open and its jungle of rules and edge cases, it abides by the progressive disclosure philosophy extremely well. Many Swift users, especially beginners, may never need to use or know about open. Only library authors and more advanced users will likely encounter uses for open. The progressive disclosure of open is further manifested in Swift’s design affinity for value types, for which open does not apply. I think this is why we haven’t seen a proposal to revert SE-0117, or further modify open.
Even though open can be successfully progressively disclosed, I would still argue that it lacks merit, at least in my experience. I almost always declare classes as final, especially if part of a public API. I have rarely encountered a situation where I would want to subclass a class within a module, but not outside of the module. Especially in a language as expressive and rich as Swift, there are plenty of other ways to design classes and modules to share behavior internally, but avoid exposing that behavior externally. A feature like open is definitely one that won’t be used the majority of the time, which makes its impact on the semantics of public even more regretful.
Furthermore, it’s worth pointing out that even though SE-0117 was discussed and reviewed after SE-0025 was accepted and implemented, it was essentially proposed in isolation from SE-0025. Sure, the community knew about SE-0025 at the time, but no one had actually used Swift 3 and the new private and fileprivate access modifiers. (One could have played with this in a snapshot, but very few developers are doing that.) We were still completely in the dark about the implications and reality of SE-0025. While drunk on augmenting access control, the community pushed through yet another proposal to change it. To be clear, no one is to blame. We simply didn’t realize.
Access control in Swift 3.0
This brings us to the current state of access control in Swift. Paraphrasing from The Swift Programming Language eBook:
openaccess enables entities to be used within any source file from their defining module or from another module that imports the defining module.openonly applies to classes and allows them to be subclassed from where they are accessible. Anyopenclass can also declare its members asopen, which allows them to be overridden.publicaccess is similar toopen, except that subclassing and overriding are only allowed from within the defining module.internalaccess (the default) enables entities to be used within any source file from their defining module, but not outside of that module.fileprivateaccess restricts the use of an entity to its own defining source file.privateaccess restricts the use of an entity to the enclosing declaration or scope.
In a very short time, Swift nearly doubled its number of access levels from three to five and altered the semantics of two previous keywords. I’ve seen experienced programmers struggle to explain the difference between them or articulate their appropriate usage. You know something is wrong when it’s easier to explain monads to a beginner than it is to explain access control levels. 😄
Returning to the philosophy of progressive disclosure, which of these access levels do we regularly need to consider? We can omit open for the reasons mentioned above. We can also omit internal since it is the default and does not need to be typed explicitly. This leaves public, fileprivate, and private for common, daily usage — one more keyword than before, with more complex behavior than before.
Saving fileprivate, or the great compromise
Most recently, proposal SE-0159: Fix Private Access Levels was put forward to simply revert the changes of SE-0025. That is, remove fileprivate and restore the original (Swift 1 and 2) semantics of private. Why? As I alluded to earlier, fileprivate turned out to be used quite often, breaking progressive disclosure. The new private essentially broke Swift’s extension-oriented style, as private members of a type were no longer accessible from an extension on that type, even if the extension was declared in the same file. The proposal was reviewed with as much controversy and ferver as SE-0025 itself. Ironically (or serendipitously?), nearly one year to the day after the final decision for SE-0025 was announced, SE-0159 was rejected, leaving the state of access control unaltered. The proposal could not be accepted because the impact on source stability for Swift 4 would be too great. I agree, this is problematic.
However, there’s clearly an issue with access control — SE-0025 did not turn out as expected — and there is disagreement in the Swift community on how to address it. The Core Team is well aware. Doug Gregor started a new discussion to hopefully find a compromise and settle Swift’s access control story for now, possibly for good:
The design, specifically, is that a “private” member declared within a type “X” or an extension thereof would be accessible from:
- An extension of “X” in the same file
- The definition of “X”, if it occurs in the same file
- A nested type (or extension thereof) of one of the above that occurs in the same file
This design has a number of apparent benefits:
privatebecomes the right default for “less than whole module” visibility, and aligns well with Swift coding style that divides a type’s definition into a number of extensions.fileprivateremains for existing use cases, but now its use is more rare, which has several advantages:
- It fits well with the “progressive disclosure” philosophy behind Swift: you can use
public/internal/privatefor a while before encountering and having to learn aboutfileprivate(note: we thought this was going to be true of SE-0025, but we were clearly wrong)- When
fileprivateoccurs, it means there’s some interesting coupling between different types in the same file. That makesfileprivatea useful alert to the reader rather than, potentially, something that we routinely use and overlook so that we can separate implementations into extensions.privateis more closely aligned with other programming languages that use type-based access control, which can help programmers just coming to Swift. When they reach forprivate, they’re likely to get something similar to what they expect—with a little Swift twist due to Swift’s heavy use of extensions.- Loosening the access restrictions on
privateis unlikely to break existing code.There are likely some drawbacks:
- Developers using patterns that depend on the existing lexically-scoped access control of
privatemay find this new interpretation ofprivateto be insufficiently strict- Swift’s access control would go from “entirely lexical” to “partly lexical and partly type-based”, which can be viewed as being more complicated
Ultimately, I regret the changes that brought the fileprivate and open access levels to Swift. I wish we could revert both of these changes and instead consider any modifications to access control cohesively as part of a Swift theme. As Doug notes, the hypothesis that fileprivate would rarely be used was incredibly wrong. This was primarily the result of extensions, which break the lexical scoping of private.
The requested revisions to SE-0025 hinted at the potential scoping issues with extensions and the new behavior of private, but these implications were not widely discussed on the mailing lists, nor fully realized until developers actually started using Swift 3. Looking back, this was a major oversight. It certainly took me by surprise when I first realized I’d have to use fileprivate everywhere, due to how I had designed my types and extensions. In my experience the new private/fileprivate is a burden rather than a solution to any tangible problem. The strict lexical scoping of private feels broken in what has become idiomatic and conventional Swift, where extensions on types are heavily used to organize functionality. Protocol extensions amplify these symptoms of brokenness.
The Core Team’s proposal that Doug outlines above is a good compromise. Before I could publish this post, David Hart opened a pull request with a draft proposal titled Type-based Private Access Level for this. I hope it gets accepted and implemented. Although it introduces even more complexity into Swift’s access control system, I think most of the complexities are behind-the-scenes implementation details — from a user’s perspective the private and fileprivate modifiers sound much easier to explain and reason about. Prior to actually using private as defined in SE-0025, I think most Swift users expected private members to be accessible from extensions. The benefits of a “partly lexical and partly type-based” access control that Doug explains are clear, and I think they outweigh the drawbacks.
We are obviously not in an optimal position. This is far from ideal, but it solves a real problem. If the suggestions above are implemented, we can escape from the corner we have painted ourselves into, albeit leaving a trail of paint-soaked footprints behind us. We will return to the state where only having to know about public and private will be necessary for most users most of the time. (Remember, internal is still the default and doesn’t need to be typed.) The usage of fileprivate would become rare and conspicuous, perhaps eventually considered bad practice.
Opportunity costs
I think it’s fair to say that the Swift community has learned a lot from the Swift 3.0 release — not only the debates and churn around access control, but around the Swift Evolution Process in general. We should keep this in mind moving forward and continue to reflect on proposals, their outcomes, where we’ve been, and how we arrived at where we are today. What do we want from this programming language? What should be prioritized and what should be deferred for the betterment of the language? Does your proposal fit with the theme of the current Swift release?
Swift evolution is anything but cheap. Some consider it actively harmful. Every change has a cost, as does every deferment. Some changes are clearly expensive while others are more subtle. Swift 3 arrived with a very real cost — a completely different set of goals (see this diff) and an enormously painful migration. But these were just the actual costs, the results of changes made.
What is perhaps more important to consider are the changes that were not made. The opportunity cost of each Swift release is the value of the changes we decide to forgo — that is, the value of everything that was not implemented. This includes major features, as well as tasks like fixing bugs, addressing compile time issues, improving runtime performance, increasing overall stability, and more. That’s not to say that these things didn’t happen, they certainly did — but a lot of time was spent on Swift Evolution Proposals, some of which should definitely have been deferred in hindsight.
None of this means the Swift community did something wrong, this is just how it is. We are all learning, even the Core Team. Fortunately, the Core Team is definitely being more strict and thoughtful for Swift 4 proposals, so I doubt we will find ourselves in a situation like this again. But, this is software development. There are always trade-offs.