Measured findings — LuckyCore (peer-to-peer Redis) vs NeoCore (master-slave gRPC)

2026-08-07. Both systems driven through the harnesses in harness/. Absolute latency is not reported: the host was at load ~13–17 on 12 cores with swap exhausted, which dominates any sub-millisecond measurement. Everything below is either deterministic (byte counts), relative (same host, same run conditions, trend across a swept parameter), or structural (ordering), so it survives that noise. Re-run on a quiet machine before publication.

1. Fan-out cost is the headline result

How much does it cost the origin to publish one change as the fleet grows? 5000 messages, back to back, send-side wall time measured from the sender's own CSV.

(LuckyCore figures are the mean of three repetitions per level; NeoCore one repetition per level.)

NeoCore's origin cost grows with the receiver count. LuckyCore's does not. From 0 to 4 receivers NeoCore's origin throughput falls from ~44.6k to ~4.1k messages per second, close to an order of magnitude, and the increments are roughly linear at ~277 ms per receiver per 5000 messages. LuckyCore stays flat within noise across the same sweep.

This is a topology consequence, not a transport one, and it is visible directly in the code. PubSubManager.broadcast walks the observer list and calls onNext once per connected slave, so the master performs N sends per change. A Redis publisher issues a single PUBLISH regardless of subscriber count; the broker absorbs the fan-out.

Why it matters for the paper: the master is not only a single point of failure, it is a throughput bottleneck that tightens as the fleet grows — exactly the regime a large network operates in. This is the strongest argument the peer-to-peer side has, and it is measured rather than asserted.

2. Per-message bytes favour gRPC, total bytes favour the broker

Deterministic, computed from MasterPubSubPacket.getSerializedSize() and the RESP framing LuckyCore's publish produces, for an equivalent payload.

Per message NeoCore is about 24% leaner. But the origin sends N copies while the Redis publisher sends one:

The crossover is at two receivers. Above that, the more efficient encoding is entirely erased by the fan-out, and NeoCore's origin pushes strictly more bytes. A per-message byte comparison alone would have reported the opposite conclusion — worth stating explicitly in the evaluation, since it is the kind of measurement that is easy to get wrong.

3. Ordering: the master buys a guarantee the broker does not

Arrival order of message ids at a receiver, counted as inversions over all pairs.

NeoCore arrived in perfect order. LuckyCore reordered — [7, 6, 4, 3, 1, 5, 8, 2, 9, 10, 11, ...] — with every inversion confined to the first eight messages, then clean.

The cause is in RedisHandler.sendRequest, which hands each publish to a cached thread pool. Sends race while the pool is still growing threads; once it is warm they serialise naturally. So the reordering is a cold-start effect, not a steady-state one, but the guarantee is absent either way: nothing in the design prevents it under load or on a busy origin. NeoCore serialises every change through the master, which is what makes its ordering a property rather than an accident.

Nothing was lost in either system in these runs. Loss behaviour under an induced disconnect is still to be measured, and that is where the two should genuinely diverge.

4. What this adds up to

Three independent measurements point the same way, and they are the trade-off the paper is about:

• The master-slave design buys ordering and an acknowledged path.
• It pays with origin-side fan-out cost that scales with the fleet, in both time and bytes.
• The broker design pays with no ordering guarantee and no acknowledgement, and buys an origin cost that is flat in the number of receivers.

That is a clean latency–robustness–cost triangle, sitting on evidence rather than on architecture diagrams.

Still to measure

1. Propagation latency distributions on a quiet host (p50/p95/p99) — the one number everyone will look for.
2. Update loss and recovery under an induced receiver disconnect, which is where fire-and-forget should visibly break.
3. Database cost per propagation, which only becomes interesting once LuckPerms is in the comparison (it reads the store on every receiver; both systems here read ~0).
4. Higher receiver counts (8, 16, 32) to confirm the fan-out slope holds and to find where the master saturates.
5. Repetitions with variance at every level — the fan-out sweep above has one repetition per level for NeoCore.

Round 2 — fan-out saturation and a stalled receiver

Added 2026-08-07, same noisy host, same caveats.

5. The fan-out sweep, extended

Two receiver-simulation methods were used, and they are not interchangeable, so both are reported.

Method A — one JVM per receiver (each receiver is a separate process with its own connection):

Close to linear, about 280 ms of extra send time per receiver per 5000 messages.

Method B — one JVM, one gRPC channel per simulated receiver (needed to reach higher counts without
running 32 JVMs on a shared host):

A methodology warning worth keeping. A first attempt at N=16 put all 16 streams on a shared
channel and came out faster than N=8, which is impossible if the master truly pays per receiver.
The cause was HTTP/2 multiplexing: 16 streams over one connection is not 16 servers. Giving each
simulated receiver its own channel restored the expected ordering. Any harness that sweeps fan-out
has to give each receiver its own connection or it measures the wrong thing.

Where the two methods agree, and where they do not. Both show origin throughput falling
monotonically as receivers are added, and there is no knee up to 30 receivers — the master
degrades smoothly rather than hitting a wall. Method A's marginal cost is roughly constant
(~280 ms/receiver); Method B's falls with N (199, 159, 114 ms/receiver at 8, 14, 30), i.e. sublinear.
The likely reason is that Method B's receivers share one client JVM, so receive-side work and netty
write batching are amortised in a way a real fleet would not be. Method A is the one that
resembles a real deployment; Method B should be read as a lower bound on the cost.

The robust statement, true under both methods and unaffected by host noise: NeoCore's origin
throughput fell from ~44.6k msg/s with no receivers to ~1.4k msg/s at 30 receivers, roughly a 30x
collapse, while LuckyCore's publisher stayed flat across its whole sweep.

Note also that 16 and 32 requested receivers registered as 14 and 30. The rest missed the
five-second keepalive window during startup and were dropped. Effective counts are reported.

6. A stalled receiver does not slow the master. It should.

2000 messages to one receiver, before and after freezing that receiver with SIGSTOP (socket stays
open, nothing is read):

The master got faster when the receiver stopped reading — about 2.6x. That is the opposite of
what a flow-controlled sender does, and it is the interesting part: PubSubManager.broadcast calls
observer.onNext(packet) on every observer with no readiness check. grep -rn "isReady" over
rpc-master returns nothing, so the master never consults gRPC's flow-control signal. Messages for
a stalled receiver are handed to netty and queued; the send call returns immediately, which is why
it looks quicker.

Consequence: a slow or hung slave causes unbounded buffering at the master, with no
backpressure and no shedding. On a real fleet that is a memory-growth path on the single most
critical node — the same node that is already the throughput bottleneck from §5 and the single
point of failure.

The peer-to-peer side does not have this failure mode. A Redis publisher hands one message to the
broker and is done; a subscriber that stops reading is the broker's problem, and Redis handles it
by disconnecting a client whose output buffer exceeds its limit. The failure is bounded and local
to the slow subscriber.

This belongs in the paper as a robustness finding, and it strengthens the SPOF argument
considerably: the master is not just a node that can fail, it is a node that a misbehaving peer
can degrade.

7. Revised summary

The master-slave design buys ordering and acknowledgement. It pays with origin fan-out cost in both
time and bytes, and with an unbounded-buffering exposure to any slave that stops reading.

Still to measure

1. Latency distributions on a quiet host.
2. Loss under an induced disconnect (as opposed to a stall), and recovery behaviour on reconnect.
3. Master memory growth under a sustained stall, to put a number on the buffering exposure.
4. Method A repeated at 16 and 32 receivers on a machine that can host them, to settle the linear
   vs sublinear question.
5. Repetitions with variance at every level.

Round 3 — a correction, targeted sends, and shedding

2026-08-07.

8. CORRECTION to §6: the master sheds, it does not buffer

§6 reported that a stalled receiver made the master faster and concluded there was no
backpressure and therefore unbounded buffering. That conclusion was wrong. The memory test
shows why.

Master RSS while 200,000 messages were broadcast at a SIGSTOPed receiver, sampled every 20,000:

Not a single kilobyte. The reason is in the send log:

8 x  Sent 20000 message(s) to 0 client
2 x  Sent 20000 message(s) to 1 client

A SIGSTOPed client also stops sending keepalives, so ObserverData.isExpired() fired after five
seconds and PubSubManager's cleaner removed the observer. From the third batch on, the master
was broadcasting to nobody. The "2.6x faster when stalled" number in §6 was not fast buffering, it
was the master sending to an empty observer list.

The corrected finding is better for NeoCore than the wrong one was. The master has crude but
real load shedding: a receiver that stops responding is evicted on a five-second liveness timeout
and costs the master nothing thereafter. Memory is bounded.

What is still untested. The eviction is driven by the client's keepalive, not by write
readiness. A receiver that keeps pinging but drains slowly would not be evicted, and
PubSubManager.broadcast still calls onNext with no readiness check (grep -rn "isReady" over
rpc-master returns nothing). That is the real backpressure question, and SIGSTOP cannot answer
it because it freezes the keepalive too. A client that pings on one thread while never reading the
stream is needed to settle it.

9. Loss during a disconnect: both lose everything

From the same run: 160,000 messages were broadcast while no receiver was registered. None were
buffered, none were replayed on reconnect. Loss during the gap is total.

Redis pub/sub behaves identically: a disconnected subscriber misses every message published while
it was away, and there is no replay.

So on this axis the two topologies are the same, which is worth saying plainly because the
architecture diagrams suggest otherwise. What differs is what happens next:

NeoCore's advantage is detection and a state refetch, not message replay. Neither system is a
durable log.

10. Targeted sends: the fan-out penalty disappears

§1 and §5 measured broadcast. NeoCore also has a targeted path,
PubSubManager.sendMessage(message, receiverIdentifier), which sends to one observer. 5000
messages addressed to a single client, with the rest of the fleet connected:

Flat in fleet size, as expected for a single onNext. Compare the same fleet size on the
broadcast path: 2358.0 ms and 2,120 msg/s at 8 receivers. Targeted delivery is about 5.6x
cheaper at N=8, and the gap widens with N.

Now the comparison inverts. LuckyCore has no addressing at all — every packet goes to the one
LuckyCore channel and every subscriber receives it, filtering client-side (that is exactly what
NeoCore's own PRIVATE-CHAT handler does with its RECEIVER field). For a message intended for
one server:

At 8 receivers that is 106 B to one node versus 1120 B to eight, seven of which discard it. The
master-slave design wins targeted traffic as decisively as it loses broadcast traffic.

And NeoCore does not use its own targeted path where it should. PubSubHelper calls
broadcast for global chat, private chat, and the benchmark; only the RELAY handler in
listener/PubSubListener uses sendMessage. Private chat is addressed to one player on one
server, so it is the textbook case for targeted delivery, and it currently costs the master a
full fleet-wide fan-out. That is an implementation gap, not a topology limit — worth stating as
such, since it is a one-line fix with a 5.6x effect at N=8.

11. Revised bottom line

The earlier "master-slave is expensive" summary was too broad. It depends on the traffic mix:

• Broadcast traffic (rank change everyone caches, global chat): peer-to-peer wins. Origin cost
  flat vs a ~30x collapse to 30 receivers.
• Targeted traffic (private chat, one player's data to one server): master-slave wins. Cost
  flat in N, and no wasted delivery to N-1 uninterested servers.
• Failure: the same. Both drop everything sent during a disconnect. Only NeoCore notices, and
  it refetches state rather than replaying messages.
• A real fleet runs both kinds of traffic, so the honest conclusion is a mix-dependent
  crossover rather than a winner.

12. Broadcast vs targeted, measured back to back — and why private chat broadcasts

Both paths in one master process, 8 receivers connected, 5000 messages each:

18.6x. Larger than the 5.6x seen across separate runs, because both paths ran in the same warm
JVM here; this is the cleaner number.

The correction to §10. §10 called private chat's use of broadcast an implementation gap. It is
not. The master can route — PlayerListManager.getServerOfPlayer(UUID) exists — but the player
location map is an ExpiringObjectMap with a 10 second TTL, swept every 5 seconds. A player who
just switched servers may be absent from it or recorded on their previous server. A targeted send
against stale routing data is silently lost, because nothing replays it (§9). Broadcasting is
correct regardless of how stale the map is: every server receives it and the one hosting the player
acts on it.

So the design is deliberately trading throughput for delivery correctness under stale routing
state, and the price of that choice is now measured: 18.6x at 8 receivers, widening with fleet
size.

This is the most interesting thing the targeted path shows, and it generalises past the game
domain. Directed delivery in a master-slave topology is only as reliable as the routing table, and
a soft-state routing table with a TTL pushes a system back toward broadcasting. Peer-to-peer
pub/sub sidesteps the question entirely by never routing — which is why LuckyCore has no addressing
at all, and why its receivers filter client-side.

A hybrid would recover most of the difference: consult getServerOfPlayer, send targeted on a hit,
fall back to broadcast on a miss. That keeps the correctness guarantee for the stale case and pays
the fan-out only when the routing state is actually unknown. Worth stating as future work rather
than as a defect.

13. Hybrid routing: measured, not guessed

§12 suggested a hybrid — consult PlayerListManager.getServerOfPlayer, address the send on a hit,
fall back to broadcast on a miss. Implemented on branch bench/hybrid-routing (commit e0ddc71)
and swept by routing hit rate. Eight slaves connected, 5000 messages per phase, all in one warm
master process:

Read the hybrid rows against the 0% row, not the control: by the time the control had run the
JVM was still warming, so the control and the 0% case measure the same code path at different warmth
(both fall back to a full broadcast on every message). The 0% row is the honest baseline.

Two things worth stating:

The fallback is free. At 0% hits the hybrid does a failed map lookup and then exactly what the
original code did, and it costs nothing measurable. There is no regression for the pathological
case, which is what makes the change safe to take.

The gain arrives long before perfect routing. Half the messages resolving is enough for 3.8x.
A real player list will resolve the large majority of lookups, because players change servers rarely
compared to how often they are messaged, so the operating point sits much closer to the 100% row
than the 0% one.

The limitation this does not remove

The hybrid converts unknown routing state into a safe broadcast. It does not detect stale
routing state. If the map says a player is on server A when they have already moved to B, the
addressed send goes to A and is lost, because nothing replays it (§9). The implementation guards
against a target that has disconnected — it checks getObserverData(target) != null — but a
still-connected server holding an out-of-date entry is indistinguishable from a correct one.

So the hybrid trades a small, bounded correctness risk for a large throughput gain, where the
original broadcast had no such risk. Closing that gap needs either a shorter TTL (more player-list
traffic), an ack from the target with a broadcast retry on failure (more round trips), or a
sequence-numbered player list so the master can tell fresh entries from stale ones. Those are the
real design options and none is free — which is itself the point the paper is making about
master-slave topologies: directed delivery is only as good as the routing table, and keeping a
routing table fresh is its own distributed-systems problem.

Round 4 — propagation latency on a quiet host

2026-08-07, seven-server: 8 cores, Intel Xeon Silver 4216 @ 2.10 GHz, load average 1.04-1.87
during the runs (13-23% subscription). This is the first host quiet enough for the absolute numbers
to mean anything. Both systems run in eclipse-temurin:17-jdk containers on --network host;
Redis 7 for LuckyCore. 2000-message warmup at 200 msg/s discarded, then 5000 measured messages at
500 msg/s, one receiver.

14. The headline metric

At the median the two are indistinguishable. 0.37 ms against 0.41 ms is a 40 microsecond
difference on a loopback path; nobody should draw an architectural conclusion from it, and the
peer-to-peer side is nominally ahead.

The tail is where they separate. NeoCore's p99 is 2.3 ms against 3.7 ms, and its worst case is
6.6 ms against 82.2 ms — a factor of twelve on the maximum. The mean follows: 0.48 ms against
0.75 ms, pulled up entirely by LuckyCore's outliers rather than by its typical case.

This is the same underlying cause as the ordering result in §3. LuckyCore hands each publish to a
cached thread pool and lets sends race; the master serialises every change through one path. Racing
is free most of the time and occasionally very expensive. Serialising costs a little at the median
and removes the expensive case.

What to say in the paper: for state propagation at this scale, the master-slave topology does
not buy lower latency — it buys predictable latency. If the requirement is "a rank change is
visible within X milliseconds, always", the tail is the number that matters and the master-slave
design is materially better on it. If the requirement is average throughput, the two are equivalent
and the fan-out cost from §1 dominates the decision instead.

15. Why the earlier runs were unusable, quantified

The same measurement on the loaded host gave a p95/p50 ratio of about 19x. Here it is 2.1x.

The loaded host inflated p50 by roughly 7.5x and p95 by roughly 68x. Any conclusion drawn from it
about architecture would have been measuring the scheduler. Recording core count and load average
alongside every run is not bookkeeping, it is what makes the numbers admissible.

16. Where the results now stand

Neither topology wins outright, and that is the paper's result. The choice turns on the traffic
mix: broadcast-heavy fleets favour the broker, targeted-heavy fleets and latency-SLO workloads
favour the master.

17. The fan-out sweep, redone properly — and it changes the story

Same quiet host as §14. Three repetitions per level with standard deviations, 2000-message
warmup discarded, 5000 measured messages per repetition, saturation rate (no pacing).

NeoCore is faster than LuckyCore at every level measured. That is the opposite of what §1
implied, and it is the more trustworthy result: §1 came off a host at 122% CPU subscription with one
repetition per level.

What survives from §1 is the shape, not the ranking. Normalised to each system's own zero-receiver
throughput:

NeoCore's origin cost starts climbing at the very first receiver and keeps climbing. LuckyCore's
is flat to four receivers. That is the topology difference, and it is unambiguous in the region
where the measurement is clean. NeoCore starts so far ahead in absolute terms (143k vs 14k with no
receivers, because an idle broadcast loop does nothing while a Redis publish still round-trips to
the broker) that it stays ahead even after losing 94% of its throughput.

Where this measurement stops being clean

Above four receivers both systems fall sharply, and that is the harness rather than the
architecture. Every simulated receiver runs in the same eight-core host as the sender, so at eight
and sixteen receivers the receive-side threads are competing with the sender for CPU. The
LuckyCore column shows it plainly: flat at 0.98 and 0.85 through four receivers, then 0.16 at
eight. A Redis publisher does not care how many subscribers exist; that drop is the box saturating.

So the honest reading is:

• N <= 4 is the clean region. NeoCore declines 7.5x while LuckyCore declines 1.18x. Real, and
  attributable to topology.
• N >= 8 is confounded by co-located receivers on an eight-core host. Report the numbers,
  attribute the shared decline to the test bed, and note that separating it needs receivers on
  their own machines.

A harness bug worth recording

The first attempt at eight subscribers hung indefinitely. JedisPoolConfig defaults to
maxTotal = 8, so eight subscriber connections exhausted the pool and the publisher blocked
forever waiting for a connection that would never be returned. Raising the cap to subs + 8 fixed
it.

This is a property of the harness, not of LuckyCore: in a real fleet each server has its own pool,
so the limit is per server rather than shared. Worth stating because it is exactly the kind of
artefact that would otherwise be reported as "Redis does not scale past eight subscribers".

What this does to the paper's argument

The claim is no longer "peer-to-peer wins broadcast throughput". It is narrower and better
supported:

The master's per-change cost grows with fleet size while the broker's does not. Whether that
matters depends on where each starts: in this test bed the master began roughly ten times ahead
and remained ahead through sixteen receivers, so the scaling penalty is real but had not yet
overturned the ranking at the fleet sizes measured.

That is a more honest and more interesting statement than the earlier one, and it makes the
extrapolation explicit — the curves converge, and finding where they actually cross needs receivers
on separate hosts.

Round 5 — LuckPerms, the third system

2026-08-07, seven-server. LuckPerms 5.5.71 (bukkit-legacy) on two Paper 1.12.2 servers, MySQL 8
storage, Redis 7 messenger, sync-minutes: 0 so nothing propagates on a timer. Both servers plus
both stores on one host. Commands issued over RCON; LuckPerms runs them asynchronously, so the RCON
response returns before the work completes and the effect has to be read from the database.

18. Cost per permission change

Twenty lp user <uuid> permission set <node> true commands, counters read from
SHOW GLOBAL STATUS LIKE 'Questions' and redis-cli INFO stats before and after. All twenty rows
confirmed written to luckperms_user_permissions.

7.2 MySQL queries per permission change, against roughly zero for NeoCore and LuckyCore, which
carry the changed data in the message itself. That is the cost of the notify-then-read design and it
is the number the fan-out and byte comparisons in §2 and §17 were missing.

19. The "+1 database read per receiver" assumption was wrong

§2 and the harness design both assumed LuckPerms costs one extra database read per receiving server,
and that this was the fairness-critical correction to the byte comparison. The measurement does
not support it. Stopping the second server changed the MySQL query count not at all: 144 either
way. Redis moved by two commands, which is the notification being delivered to one fewer subscriber.

The likely explanation is that LuckPerms only reloads data it is holding. The benchmark users
were offline, so the receiving server had nothing cached for them and the push cost it nothing. The
reload is proportional to affected users who are online on that server, not to the size of the
fleet.

That is a better design than the earlier assumption gave it credit for, and it changes the argument:

• The wire-versus-database trade-off in §2 is not LuckPerms paying N reads against the push
  systems' zero. For state nobody is currently using, LuckPerms pays nothing on the receiving side
  at all.
• Its database cost is concentrated on the origin (7.2 queries per change), and is a property
  of writing through storage rather than of the topology.
• The receiver-side read only appears for online, affected users, which in a real fleet is at
  most a handful of servers regardless of how many exist.

This needs confirmation with an actually-online player before it goes in the paper. The runs
above had zero players connected, so the "affected user is online here" path was never exercised.
The honest claim today is the measured one — receiver count did not affect database load for offline
users — plus the stated hypothesis for why.

20. Where LuckPerms sits in the comparison

The row that matters is the last one. LuckPerms is the only one of the three where a missed
notification is not a lost update: the data is in storage, so a receiver that misses the signal is
merely stale until something makes it read again. Both push designs lose the update outright (§9).

That is the real trade LuckPerms makes, and it is not the one the earlier sections described. It
pays roughly seven database queries per change at the origin, and in exchange a dropped message
costs staleness rather than data. The push designs pay almost nothing per change and lose updates
when a receiver is away.

Limitations of this round

• Two servers only; the receiver-side sweep that would settle §19 needs more, and an online player.
• No propagation-latency figure for LuckPerms yet, so it is absent from the §14 table. Measuring it
  needs a hook on the receiving side — the LuckPerms API or a small plugin — because a command
  round-trip does not tell you when the data arrived.
• Everything co-located on one eight-core host, same caveat as §17.

Round 6 — LuckPerms propagation latency

2026-08-07, seven-server, load 1.42. Two Paper 1.12.2 servers, LuckPerms 5.5.71, MySQL 8 + Redis 7.
Change issued on s1 over RCON; arrival observed on s2 by a purpose-built plugin
(harness/luckperms/LatencyProbe.java) subscribed to UserDataRecalculateEvent.

21. Why a plugin and not a console client

A console client can only see chat and command output. It cannot see when the receiving server's
data changed, which is the thing being measured. UserDataRecalculateEvent fires at the point
where the receiver has finished applying the new permissions, and that is strictly later than the
messenger delivering its notification — LuckPerms sends a signal and the receiver then reloads from
storage. Timing the notification would understate what a player experiences.

22. Holding users, and why it was necessary

The first two attempts recorded zero events on the receiver for 30 changes. So did a third,
after loading the same users with lp user <uuid> info on the receiver first.

The reason is the §19 finding, now confirmed directly: a LuckPerms receiver only reloads users it
is currently holding, and it holds a user while that player is online. With nobody connected,
nothing is held, and an incoming push costs the receiver nothing at all — no database read, no
recalculation, no event.

The probe therefore loads thirty users through UserManager.loadUser on startup and keeps the
references, standing in for those players being online on that server. With that in place, 30
changes on s1 produced exactly 30 recalculations on s2.

This is worth stating plainly in the paper because it is a real architectural property, not a test
artefact: LuckPerms' receiver-side cost is proportional to affected users who are online there,
not to fleet size.

23. The measurement

All figures in milliseconds. For comparison, from §14 on the same host:

Roughly 165x the median of either push design.

The baselines are not identical, and that matters

This must be said explicitly or the comparison is unfair. The push measurements time
message sent -> message applied. The LuckPerms measurement times command issued -> data applied
elsewhere, and therefore also contains:

• RCON dispatch and LuckPerms' asynchronous command scheduling,
• the MySQL write on the origin,
• the Redis notification,
• the MySQL re-read on the receiver.

So the 67 ms is not 67 ms of network. It is the cost of a design that routes data through storage
rather than through the message, plus command overhead the other two never paid.

What survives the caveat is the shape of the result. Two database round trips and a scheduling hop
put the answer in the tens of milliseconds, while carrying the data in the message puts it in
the sub-millisecond range. That is three orders of magnitude of design decision, and no amount
of baseline correction moves it into the same bracket.

How to close the gap honestly

To make it strictly comparable, the origin timestamp should be taken inside LuckPerms at the point
the change is committed rather than at RCON dispatch. That needs a second probe on the sending side
using the same event bus. Worth doing before publication; the conclusion is unlikely to change, but
the number would then be defensible as a like-for-like latency rather than an end-to-end one.

24. All three systems, one table

The three systems are not on a single axis. LuckPerms is two orders of magnitude slower and is the
only one that does not lose updates when a receiver is away. The two push designs are
indistinguishable at the median and separate on the tail (§14), on fan-out cost (§17), and on
ordering (§3).

Round 7 — LuckPerms latency, like for like

2026-08-07, seven-server, load 1.25. The §23 figure timed command-issued to data-applied, which
included RCON dispatch and LuckPerms' asynchronous command scheduling. This round removes both.

25. Timestamping inside LuckPerms at both ends

The probe now runs on both servers and subscribes to two events:

• NodeAddEvent on the originating server, which fires as the node is committed — after the
  command has been parsed and scheduled, so none of that is counted.
• UserDataRecalculateEvent on the receiving server, which fires once the new data is applied.

The difference is the propagation itself: the storage write, the messenger notification, and the
receiver's reload. Pairs are joined by user UUID; both JVMs are on one host so System.nanoTime()
is comparable.

26. The corrected number

30 changes, all matched:

The command machinery was about 50 ms of the original 67. That is RCON dispatch plus
LuckPerms' asynchronous command scheduling, and none of it is propagation. Reporting the 67 ms
against the push systems' sub-millisecond figures would have overstated the gap by roughly a factor
of four.

27. The three systems on one basis

All measured on the same quiet host, all timing change committed -> change applied elsewhere:

About 45x, not 165x. The direction is unchanged and the reason is unchanged: routing the data
through storage costs a write and a read that carrying it in the message does not. But the honest
multiplier is a factor of forty-odd, and that is the number the paper should use.

Two further things the corrected measurement shows:

• LuckPerms' tail is disproportionate. p99 of 61.33 ms against a p50 of 17.38 is a ratio of 3.5,
  worse than either push design (5.6 and 10.2 in absolute terms but 2.1x and 9.2x relatively — see
  §14). The database round trip is not just slower on average, it is less predictable.
• The floor is 10.86 ms. Even the fastest observed propagation took ten milliseconds, so this is
  structural rather than incidental. No amount of tuning brings a write-then-read design into the
  same range as a push.

28. What this does not change

The §19/§22 finding stands: a receiver only reloads users it is holding, so for state nobody is
using the propagation costs the receiver nothing at all. And §20 stands: LuckPerms is the only one
of the three where a missed notification is staleness rather than a lost update.

The trade is now precisely quantified. LuckPerms pays roughly forty times the propagation latency
and seven database queries per change, and in exchange a dropped message costs staleness rather
than data. Both push designs pay almost nothing per change and lose updates outright when a
receiver is away.

Round 8 — correcting the loss claim

29. "Total loss" was wrong for both push systems

§9 and §20 said a disconnected receiver loses updates outright while LuckPerms merely goes stale.
That overstated the difference. Checked in the source:

LuckyCore reloads from storage on join. PlayerListener handles AsyncPlayerPreLoginEvent and
does new CachedData(plugin, uuid, name) -> cachedData.getData() -> playerData.load(data). The
pub/sub packet is an optimisation that avoids waiting for the next join; MySQL remains the source
of truth. A missed packet leaves that server stale until the player next connects to it, at which
point the correct data is read.

NeoCore's pub/sub does not carry player permission data at all. Its dataloader has no join
listener; it pushes the online-player list every 2000 ms (SpigotPlayerListUpdater) and the channel
otherwise carries chat, private chat and mute traffic. Permission data comes from Atreus on a
separate path (§7). So the messages that can be lost here are either inherently transient (a chat
line) or self-healing on the next 2-second tick (the player list).

Corrected picture — all three recover, by different means and on different triggers:

The honest statement is that none of the three treats the message as the durable record, and
none of them is a replayable log. What differs is the recovery trigger: LuckPerms and LuckyCore both
fall back to shared storage, LuckPerms on any reload and LuckyCore specifically on join; NeoCore's
player list is re-pushed on a timer while its chat traffic is genuinely fire-and-forget, as chat
usually is.

What this does to the argument. It removes what looked like LuckPerms' decisive advantage. The
durability difference is much smaller than §20 claimed: LuckyCore has the same storage backstop, it
just waits for a join rather than a reload. The remaining honest distinctions are the ones already
measured — latency, fan-out cost, ordering, and credential distribution — not durability.

This is worth flagging in the paper's threats-to-validity as a case where reading the propagation
path in isolation produced a wrong conclusion; the recovery path lives elsewhere in the codebase
(a join listener) and had to be read separately.

30. Correction to §29 — permission recovery in Atreus/NeoCore is a TTL, not a join

§29 said NeoCore's losable traffic was "transient chat, or self-healing on the next 2-second player
list tick". The player-list tick is real but irrelevant: it carries who is online, not permission
data. The permission path was described wrongly.

Checked properly. There is no join listener anywhere in Atreus's client, client-api or
client-api-standalone — a grep for PreLogin, PlayerJoin, LoginEvent and onJoin matches no
files. Permission data is pulled on demand:

• Datastore.get(uuid) returns from cache on a hit, otherwise calls rpcDatastore.getPermissionData,
  an RPC to the master.
• The cache is cache2k with expireAfterWrite(permissionLifespan). The shipped default is
  5000 ms (config.yml: cache.lifespan.permissionData: 5000).
• A received update calls cacheDatastore.remove(uuid) and fires PermissionUpdateEvent.
• cache.type is ALL, HALF or NONE. With NONE every read goes to the master and nothing can
  be stale at all.

So a missed pub/sub message costs at most five seconds of staleness, after which the entry
expires and the next read refetches from the authority.

Corrected recovery table — three genuinely different mechanisms:

This reverses the ranking §29 implied. The master-slave design has the tightest recovery bound
of the three, and it is the only one that is time-based rather than event-based: it needs no player
action and no external trigger. LuckyCore's window is unbounded in wall-clock terms, because it
closes only when that player next connects to that specific server.

That is a genuine advantage of pulling from an authority rather than pushing into a cache: the
authority is always reachable, so a bounded TTL is sufficient to guarantee convergence. A
peer-to-peer design with no authority has to wait for a natural read, which in this case is a login.

Two corrections in two rounds on this one axis, both because the recovery path lives outside the
propagation code. Worth saying in threats-to-validity: for each system the message path and the
convergence path had to be read separately, and the convergence path is what determines the actual
consistency guarantee.