macOS 10.15: Slow by Design

In episode 379 of ATP both Marco Arment and John Siracusa described noticeable delays and stalls after upgrading to macOS 10.15.

I have been struggling with this issue myself and have found several system operations that can cause these delays, which I will detail below.

One way to solve the delays is to disable your internet connection. This is tough medicine, but if you notice these delays, try it for an hour just to verify that indeed the issue is resolved by disabling internet connectivity.

Another way to reduce the delays is by disabling System Integrity Protection. I say reduce, because I still do get some delays even with SIP disabled, but the system does overall feel much faster, and I would strongly recommend anyone who thinks their system is sluggish to do the same.

Spawning a new Process

Apple has introduced notarization, setting aside the inconvenience this brings to us developers, it also results in a degraded user experience, as the first time a user runs a new executable, Apple delays execution while waiting for a reply from their server. This check for me takes close to a second.

This is not just for files downloaded from the internet, nor is it only when you launch them via Finder, this is everything. So even if you write a one line shell script and run it in a terminal, you will get a delay!

You can test this by running the following two lines in a terminal:

echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh
time /tmp/test.sh && time /tmp/test.sh

Update 2020-05-23: Some users have a Developer Tools category in the Security & Privacy preferences pane. If your terminal is added to this category, you will not be able to reproduce this delay. Though there have been enough confirmations to establish that the delay is real. One user in China reports a delay of 5.7 seconds when using their VPN.

Honestly, this is downright baffling. Are Apple sending the source of all my custom scripts to their server? With their stance on privacy, I wouldn’t think so, so they are likely just sending a checksum, but what are they doing with that checksum that the system couldn’t do locally?

As for the notarization check, the result is cached, so second invocation should be fast, but if you are a developer, you may update your scripts and binaries regularly, which trigger new checks (it appears caching is based on inode, so an update-in-place save may avoid triggering a new check), or you may have workflows that involve dynamically creating and executing scripts, which performance now hinges upon the responsiveness of Apple’s servers.

The worst delay I have seen for this particular issue is around 7 seconds, and I have had a few episodes where it seemed to not cache the result, so repeated launches would still have the delay.

This issue has been reported to Apple and assigned FB7674490. Apple has however responded that it is “by design” (hence the title of this post).

Privileged File System Locations

Apple has a few file system locations that require user permission to access them, for example ~/Desktop, ~/Documents, and ~/Downloads.

Surprisingly though, just obtaining the display name or icon for one of these folders will trigger Apple’s code to verify that the client is allowed to access the location.

This is done by sending a message to the sandboxd process which sends a message to tccd which calls SecCodeCheckValidityWithErrors and seems to communicate with yet another process, but I can’t find which, and this takes around 150 ms per location.

This delay is for each of the privileged locations accessed, and while the result is cached, the cache only last until next relaunch of the application.

To reproduce this, I opened Transmission and went to Preferences → Transfers. Here you can configure 3 different folders, which are shown in the pop-up button with name and icon. I configured them as ~/Desktop, ~/Documents, and ~/Downloads. Relaunching Transmission and opening Preferences on my system resulted in a 0.62 second delay (quite noticeable).

The relevant parts of the stack (during this delay) was -[NSWorkspace iconForFile:] → getattrlist → __WAITING_ON_APPROVAL_FROM_SANDBOXD__.

This issue has been reported to Apple and assigned FB7674470.

Keychain Access

Like with privileged folders, keychain items also require permission for applications to access them.

But again, something is wrong. Specifically calling SecKeychainFindGenericPassword can cause noticeable delays, on a bad internet day I had this call stall for 3.3 seconds and this was with System Integrity Protection disabled!

However, disabling internet removed the delay. With internet enabled, it was reproducible by relaunching the application and triggering the code that called SecKeychainFindGenericPassword.

This issue has been reported to Apple and assigned FB7679198. Apple has responded that applications should not use this function, though the documentation for SecKeychainFindGenericPassword does not state that it is deprecated, and even if it was, it should still not cause a multi-second delay when an application that has already been granted access to a keychain item requests this item.

Contacts

Creating an instance of ABAddressBook takes on average 0.3 seconds on my system, but I have had this take over a second.

Like with keychain access the problem is entirely removed if disabling internet but not when disabling SIP.

This issue has been reported to Apple and assigned FB7679193.

Application Launch Delays

This is the worst issue, sometimes, things will stall for 5-30 seconds.

Mostly though it is when launching applications. Sampling the application during launch shows stalls in ImageLoaderMachO::loadCodeSignature, SLSMainConnectionID, and many references to Skylight and CGS in the stack trace.

The delays are removed when disabling internet, but unfortunately not fixed by disabling SIP.

This issue has been reported to Apple and assigned FB7679285.

Final Words

All of the above issues were experienced on a new Mac mini with the OS intentionally kept as close as possibly to “factory default”, I got confirmation on some of the above from a friend (who ran tests on his machine), although his timings were different than mine, but they all showed significant delays on the operations identified above.

I am writing this post to call attention to what I consider a serious design problem with Apple’s most recent OS where it appears that low-level system API such as exec and getxattr now do synchronous network activity before returning to the caller.

With SIP enabled and on a bad internet day I can have the entire machine freeze for 1-2 seconds every 10th minute, not to mention everything just being sluggish.

Update 2020-05-24: About the system freezing, I posted a comment on Michael Tsai’s blog with a hypothesis as to why some of us suffer from system freezes. In addition to internet connectivity, it also depends on how many new processes are spawned, with people running make (or similar tools) probably getting close to the system’s limit on how many capability requests it can process (assuming each new process require capability checks).