Is multicore ARC scaling in scope for libobjc2? some retain/release and weak-reference contention findings

[DTW-Thalion]

We run a number of GNUstep-based services at Thalion on a high-throughput server farm, and we have been trying to squeeze more performance out of the runtime under that kind of load (many cores, heavily multithreaded, long-lived processes). Before we take this any further or propose anything concrete, I wanted to check with you whether this is even a direction the project cares about, because how much of it is worth doing depends a lot on what a typical libobjc2 workload actually looks like.

My impression is that a large share of GNUstep use is GUI or otherwise single-threaded, where the runtime already performs very well. The things we are running into only really show up under sustained multicore contention. So the first question is: do you consider the many-core, heavily-threaded case to be in scope for libobjc2, or are most users single-threaded enough that improving multicore scaling would be effort spent on a case few people would benefit from? If the answer is "most workloads are single-threaded", so be it, what follows may not be worht inclusion in the master repo.

To make the question concrete, here is what we found: We profiled the hot paths under 1/2/4-thread scaling. The first thing worth saying is that dispatch is already excellent and we do not think it should be touched: it is lock-free, it scales flat across cores, and polymorphic dispatch measures the same as monomorphic (about 3.0 vs 3.2 ns), so the trie is genuinely O(1) with no cache-thrash penalty. Swapping in objc4-style per-class hash caches would be a lateral move at best and would give up the never-miss property.

The operations that scale badly are all in ARC and the metadata locks, not dispatch.

The one finding that helps even single-threaded callers is the retain/release atomics. A retain/release pair is about 31 ns, and an isolated micro-test (no Objective-C, just the atomic pattern) shows that nearly all of that is the atomic pattern itself:

libobjc2-style pair  (fetch_add(p,0) seed + CAS, seq_cst)   27.6 ns
relaxed-load seed    + acq/rel CAS                          14.4 ns
plain fetch_add/fetch_sub (relaxed)                          8.1 ns

The __sync_fetch_and_add(refCount, 0) used to seed the CAS loop is a full read-modify-write barrier where a plain relaxed load would do, and the whole loop runs under sequential consistency. Seeding with a relaxed load, using release ordering on the decrement, and taking an acquire fence before -dealloc took the pair from about 31 ns to about 17 ns on our host (a bit under 2x), and as far as we can tell it preserves the ordering the code needs (the CAS loop already tolerates a stale seed). This one is attractive precisely because it is not a multicore-only concern: it is the single most frequent ARC operation and the win shows up on one thread.

The bigger multicore item is the weak-reference table. Every weak load/store/destroy takes one process-global mutex, and because it is a plain mutex rather than a reader-writer lock, concurrent weak loads serialise against each other too. In our measurements weak store goes from about 87 ns/op on one thread to about 714 ns/op on four:

storeWeak pairs, distinct slots    1T 87    2T 478    4T 714 ns/op
loadWeakRetained, distinct slots   1T 62    2T 220    4T 444 ns/op

We prototyped sharding the weak table by object address, along the lines of objc4's SideTable model, which took storeWeak at four threads from about 714 ns down to about 36 ns and made the scaling flat (roughly 36 ns at one, two and four threads). I want to be honest that this is the least trivial of the changes: our first cut had a use-after-free (a reader picked its shard from a control block that another thread was concurrently freeing), which we fixed by making the control blocks type-stable per shard so the shard index is immutable and safe to read without a lock. That experience is part of why I am asking rather than just sending a patch. The global lock may well be a deliberate simplicity/robustness tradeoff on your part, and I would rather hear your view on whether the added complexity is worth it before writing it up properly.

There is also a false-sharing effect on the refcount word that is invisible in single-threaded profiling. Because the count sits immediately before the object body, two small objects allocated close together put their refcount words on the same cache line, so independent threads retaining independent objects ping-pong that line:

distinct objects, unpadded     1T 31   2T 83   4T 127 ns/op
distinct objects, 128B padded  1T 31   2T 31   4T  31 ns/op

Cache-line-isolating the refcount word removes the cliff (it took the distinct-object case from about 89 ns back to 16 ns at four threads for us), but it roughly doubles the memory footprint of the smallest objects (we measured 32 to 64 bytes for a 16-byte instance), so we would only ever see that as an opt-in build flag, never a default.

Separately, while reading the dtable code we noticed two ordering things that look like they could bite on weakly-ordered targets (aarch64 in particular): a few dtable publication points use plain stores where a release store looks intended, and there is a narrow window in objc_resize_dtables where a class can be seen as installed against the new uninstalled_dtable pointer before it has been rewritten, which could route a message to forwarding without running +initialize. These are independent of the performance work and I would rather raise them on their own if they are of interest, so I am only mentioning them here for completeness.

So the questions, in order:

Is the multicore/many-core case something you want libobjc2 to be good at, or is single-threaded use enough the norm that this is low priority for the project?

If nothing else, the relaxed-atomics change looks like a clean, low-risk, single-threaded win. Would you welcome a PR for just that on its own?

For the weak-table sharding, would you want that as a PR, or is the single global lock a tradeoff you would rather keep? If you would take it, do you have a preference on approach (reuse the existing lock_for_pointer address striping, versus a dedicated shard array)?

All the numbers above are from a development machine 32-core x86-64 host with clang 18.1.3, measured against a fresh master build with and without the changes; absolute values vary by machine, but the scaling shapes (flat versus collapsing) are the part we are relying on and those are microarchitecture-independent. All 194 upstream tests pass with the changes applied, and the retain/release and weak-reference stress tests we wrote pass too. Happy to share the benchmark harnesses.

Thanks!

cc @davidchisnall @rfm @gcasa

[davidchisnall]

Is the multicore/many-core case something you want libobjc2 to be good at, or is single-threaded use enough the norm that this is low priority for the project?

It is. We have at least one known user doing high-throughout processing on 24-core systems.

For the weak-table sharding, would you want that as a PR, or is the single global lock a tradeoff you would rather keep? If you would take it, do you have a preference on approach (reuse the existing lock_for_pointer address striping, versus a dedicated shard array)?

The refactoring to abstract over the table was intended to make it easy for that table to be a set of tables that we stride over, with independent locks. I would like to make the number of tables a template parameter so that we can expose it as a build-time parameter with a sensible default, allowing specialised use cases to burn a bit more memory in exchange for more performance.

Part of the reason that I have not done this is that I do not have any realistic test workloads that are bottlenecked on this (and no one replied to prior requests for testing). Adding the flag to check whether there are any weak references was driven by profiling from a workload that doesn’t use weak references and hit contention on this, but that was a long time ago.

Note that we do not accept LLM-generated code.

If nothing else, the relaxed-atomics change looks like a clean, low-risk, single-threaded win. Would you welcome a PR for just that on its own?

Most of the code that does atomics predates C++11. It is now a build requirement and I am slowly replacing the old sync builtins with proper atomics. Patches to do this are very welcome.

The other thing I am planning at some point is to factor out the object header (isa and refcount) so that we can do the Apple trick of keeping small refcounts in the isa pointer and use a sharded table for refcounts for overflow. This refactoring would make your false sharing worse, but would also enable adding more padding at the start with inline refcounts.

[DTW-Thalion]

While looking at the dispatch path for the multicore work I did notice something about how the dtable sparse arrays are published, and wanted to get your take before treating it as a bug, since this is the hot path and I may be missing a barrier that already covers it.

New interior nodes are published into the parent with a plain store:

// sarray2.c, SparseArrayInsert
init_pointers(newsarray);
sarray->data[i] = newsarray;          // plain store of a freshly built node

and the fast path reads the same array with plain loads, without a lock:

// sarray2.h, SparseArrayLookup  (reached from objc_msgSend)
uint32_t i = index;
... sarray->data[...] ...

On a strongly-ordered target this is fine. On a weakly-ordered one (aarch64) a reader on another thread could in principle observe the data[i] pointer before it observes init_pointers(newsarray), i.e. see the node before its contents. The same shape shows up where uninstalled_dtable is replaced during a resize (dtable.c:256 / :627): a class could be seen as "installed" against the new uninstalled dtable in the window before it has been rewritten, and skip +initialize.

So, this looks mostly benign in practice. The nodes are calloced, so a reader that races the publication sees zeroed children (a miss) and falls through to the slow path under the lock, rather than dereferencing garbage — so the sparse-array case seems self-healing. The +initialize skip is narrower, only at a depth crossing while classes are being loaded concurrently. I have not reproduced an actual failure: x86-64 is TSO so it can't show it, and I do not have an aarch64 machine to try it on. ThreadSanitizer does report a data race at dtable.c:824 in objc_send_initialize against the dispatch read in sendmsg2.c, which is consistent with the publication not being ordered, but that is not the same as an observed miscompile.

So the question: is there a barrier somewhere on the send path that already orders these publications, and I have just missed it? Or is the intent that the lock-free readers tolerate a transient stale read and recover via the slow path — in which case, is the +initialize-skip window considered acceptable? If neither, would release stores on the publication points (and acquire loads on the corresponding fast-path reads) be a change you'd want, or is the cost on the dispatch path not worth it for a window this narrow?

Thoughts @davidchisnall ?

cc @rfm @gcasa

[davidchisnall]

There is no defined ordering in the Objective-C abstract machine between setting a method in one thread and sending a message to the object with a matching selector in another thread, so it is fine for another thread to see either the old or new value.

In theory, this should be a relaxed consistency store, but there's currently a paper in review with WG21 that makes all primitive-sized loads and stores relaxed-consistency atomic because it turns out that's what all C/C++ implementations do.