20260913T115758Z-f79672d9-tap-compare-scaling
Same-hardware TAP-server resource-scaling comparison: the parity
protocol's corpus, gates, query stream, formats and statistics, with
every server's resource pins raised tier by tier. Where the host
cannot hold every pinned stack of a tier at once, the servers are
measured one at a time under that tier's pins, so repetitions do not
interleave across them; the gates run per tier with the stacks up.
The pins actually applied, as docker inspect and SHOW saw them, are
under pins/. See benchmarks/tap-compare/scaling/PROTOCOL.md for
the pre-registered design. Where resources.jsonl covers a rung, the
resource tables give each server's CPU cores used, CPU time per request
and memory over the rung's window (resources.csv has every rung);
rungs measured before the sampler started show —.
Tier 24
Gates
| target |
TAP |
taplint |
maxrec default |
| argus-scaling3 |
1.1 |
PASS (2 errors) |
134217728 |
| dachs-scaling3 |
1.1 |
PASS (0 errors) |
20000 |
| egernia-scaling3 |
1.1 |
PASS (0 errors) |
10000 |
Agreement gate: 11 classes agree, none disagree.
csv
| class |
c |
argus-scaling3 rps |
argus-scaling3 p95 (s) |
dachs-scaling3 rps |
dachs-scaling3 p95 (s) |
egernia-scaling3 rps |
egernia-scaling3 p95 (s) |
verdict |
| Q01 |
8 |
541.4 ±79.3 |
0.017 ±0.000 |
19.1 ±0.1 |
0.511 ±0.004 |
591.9 ±15.0 |
0.024 ±0.010 |
tie |
| Q01 |
32 |
852.8 ±22.0 |
0.044 ±0.002 |
17.2 ±0.1 |
2.001 ±0.020 |
718.9 ±10.2 |
0.064 ±0.004 |
argus-scaling3 |
| Q11 |
8 |
29.9 ±1.3 |
0.305 ±0.013 |
4.2 ±0.1 |
2.433 ±0.170 |
46.9 ±22.4 |
0.292 ±0.161 |
tie |
| Q11 |
32 |
31.1 ±0.1 |
1.093 ±0.011 |
4.4 ±0.0 |
12.189 ±0.810 |
65.2 ±4.7 |
0.887 ±0.136 |
egernia-scaling3 |
votable
| class |
c |
argus-scaling3 rps |
argus-scaling3 p95 (s) |
dachs-scaling3 rps |
dachs-scaling3 p95 (s) |
egernia-scaling3 rps |
egernia-scaling3 p95 (s) |
verdict |
| Q01 |
8 |
555.3 ±37.8 |
0.016 ±0.000 |
19.0 ±0.4 |
0.516 ±0.002 |
572.1 ±62.8 |
0.030 ±0.009 |
tie |
| Q01 |
32 |
861.0 ±9.0 |
0.043 ±0.000 |
16.7 ±0.1 |
2.062 ±0.002 |
718.1 ±5.9 |
0.067 ±0.009 |
argus-scaling3 |
| Q11 |
8 |
23.5 ±0.4 |
0.395 ±0.003 |
4.5 ±0.3 |
2.275 ±0.708 |
44.7 ±9.7 |
0.286 ±0.055 |
egernia-scaling3 |
| Q11 |
32 |
23.7 ±0.1 |
1.434 ±0.021 |
4.8 ±0.2 |
9.990 ±1.475 |
53.3 ±6.5 |
1.258 ±0.372 |
egernia-scaling3 |
resources, csv
| class |
c |
argus-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
dachs-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
egernia-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
| Q01 |
8 |
4.43 |
8.2 ms |
2.40 |
2.54 |
2.08 |
109.5 ms |
0.97 |
0.98 |
3.56 |
6.0 ms |
1.08 |
1.10 |
| Q01 |
32 |
7.81 |
9.2 ms |
2.88 |
3.14 |
2.04 |
120.1 ms |
0.97 |
0.98 |
5.65 |
7.9 ms |
1.09 |
1.09 |
| Q11 |
8 |
7.86 |
264.1 ms |
3.16 |
3.17 |
1.51 |
361.2 ms |
1.01 |
1.03 |
4.68 |
99.6 ms |
1.12 |
1.13 |
| Q11 |
32 |
8.24 |
266.6 ms |
3.20 |
3.22 |
1.62 |
376.1 ms |
1.04 |
1.07 |
7.14 |
109.7 ms |
1.21 |
1.22 |
resources, votable
| class |
c |
argus-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
dachs-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
egernia-scaling3 cores |
CPU s/req |
mem mean GiB |
mem peak GiB |
| Q01 |
8 |
4.60 |
8.3 ms |
1.36 |
1.50 |
2.15 |
114.0 ms |
0.78 |
0.79 |
3.37 |
5.9 ms |
0.87 |
0.87 |
| Q01 |
32 |
8.00 |
9.3 ms |
1.87 |
2.16 |
2.07 |
126.4 ms |
0.80 |
0.81 |
5.67 |
7.9 ms |
0.91 |
0.92 |
| Q11 |
8 |
7.82 |
333.8 ms |
2.17 |
2.19 |
1.51 |
338.5 ms |
0.97 |
1.01 |
5.74 |
128.9 ms |
1.02 |
1.04 |
| Q11 |
32 |
7.99 |
340.1 ms |
2.21 |
2.23 |
1.67 |
359.9 ms |
1.03 |
1.09 |
7.36 |
138.3 ms |
1.18 |
1.20 |
throughput vs CPU cores used (mix)
| format |
c |
target |
API workers |
rps |
cores |
rps per core |
CPU ms/req |
|
|
|
|
|
|
|
|
Claims
This run may claim the relative behaviour of these versions, on this
hardware, on this corpus, as deployed by their own documentation,
under the recorded resource pins — nothing else. A tie verdict is
pre-registered: overlapping 95% intervals, or under 10% apart
in throughput. Classes a gate excluded are absent, not hidden.
Threats to validity
- The corpus is generated by egernia's own seeder; its distributions
may flatter egernia's index choices.
- The query classes descend from egernia's own performance history.
- The team operates egernia expertly and the other servers from their
documentation.
- Single hardware, single run window; versions frozen at the recorded
digests.
- Within a tier the servers were measured one after the other, not
interleaved: host drift over a tier's hours lands on one server.
The order alternates between tiers (recorded in
pins/).
Reproduce with
scripts/export_obscore_snapshot.sh benchmarks/tap-compare/corpus
benchmarks/tap-compare/scaling/run.sh
Environment: see environment.json (git f79672d9, seed 424242, corpus 9f9da00f45f1…).