DDR QoS Modification Impact on Cyclictest Latencies

Headline:

DDR QoS mods consistently lower BOTH Core0 and Core1 max in customer tests. All 8 Customer -i1000 pairs improve on max0 (-10 to -257 µs) AND on max1 (-3 to -133 µs). Mean reductions: max0 ≈ -134 µs, max1 ≈ -39 µs.
DDR QoS mods do not produce a consistent improvement in SDK tests. 8 SDK pairs split: 3 show max1 improvement, 4 show max1 regression, 1 tied. Magnitudes are small (±40 µs).
DDR mods is one of the two effective levers for reducing customer-test Core1 max; the other being core isolation OFF, covered in its own report.
The largest customer-test improvement (260522_3 → 260525_2) cuts max0 by 249 µs and max1 by 133 µs on the same condition, just by enabling DDR mods.

What is being compared

"DDR mods" refers to a DDR QoS register configuration that elevates the priority of A53 reads above the priority of A53 writes. The register values are set with devmem2 and are visible in the test output for each DDR-mods run. Further detail on the DDR QoS configuration is provided in a separate document.

The customer test command intentionally exercises DDR from Core 1 (stress-ng --vm 1 --vm-bytes 80% generates large memory traffic on the same core that cyclictest measures); the SDK test stress command (stress-ng --cpu-method=all -c 4) does not include this Core 1 DDR access pattern.

SDK test pairs (8)

Aggregate Δmax (B − A) where A=no DDR mods, B=DDR mods, so negative = DDR mods better:
Core0: mean Δmax0 = +1.8 µs | DDR mods better in 2/8 pairs, worse in 5, tied in 1.
Core1: mean Δmax1 = +1.0 µs | DDR mods better in 3/8 pairs, worse in 4, tied in 1.

condition (fixed)	no DDR file	max0	max1	DDR file	max0	max1	Δmax0	Δmax1	plot
col=SDK test, fs=base, optee=oob, iso=n, ddr=n	260522_2	51	122	260524_1	72	85	+21	-37	histograms ↓
col=SDK test, fs=base, optee=oob, iso=Y, ddr=n	260530_1	108	102	260524_3	93	84	-15	-18	histograms ↓
col=SDK test, fs=base, optee=prng, iso=n, ddr=n	260522_4	54	63	260523_3	54	83	0	+20	histograms ↓
col=SDK test, fs=base, optee=prng, iso=Y, ddr=n	260531_1	62	37	260527_1	67	37	+5	0	histograms ↓
col=SDK test, fs=default, optee=oob, iso=n, ddr=n	260518_1	126	66	260526_4	104	107	-22	+41	histograms ↓
col=SDK test, fs=default, optee=oob, iso=Y, ddr=n	260530_4	101	111	260526_2	104	99	+3	-12	histograms ↓
col=SDK test, fs=default, optee=prng, iso=n, ddr=n	260519_1	62	55	260531_4	81	68	+19	+13	histograms ↓
col=SDK test, fs=default, optee=prng, iso=Y, ddr=n	260530_2	61	36	260524_2	64	37	+3	+1	histograms ↓

Customer test (-i1000) pairs (8)

Aggregate Δmax (B − A) where A=no DDR mods, B=DDR mods, so negative = DDR mods better:
Core0: mean Δmax0 = -133.9 µs | DDR mods better in 8/8 pairs, worse in 0, tied in 0.
Core1: mean Δmax1 = -39.0 µs | DDR mods better in 8/8 pairs, worse in 0, tied in 0.

condition (fixed)	no DDR file	max0	max1	DDR file	max0	max1	Δmax0	Δmax1	plot
col=Customer -i1000, fs=base, optee=oob, iso=n, ddr=n	260529_1	460	72	260528_3	203	69	-257	-3	histograms ↓
col=Customer -i1000, fs=base, optee=oob, iso=Y, ddr=n	260529_4	250	180	260525_1	240	113	-10	-67	histograms ↓
col=Customer -i1000, fs=base, optee=prng, iso=n, ddr=n	260531_3	198	73	260528_2	83	49	-115	-24	histograms ↓
col=Customer -i1000, fs=base, optee=prng, iso=Y, ddr=n	260530_3	175	149	260526_1	72	116	-103	-33	histograms ↓
col=Customer -i1000, fs=default, optee=oob, iso=n, ddr=n	260529_2	269	78	260528_4	251	59	-18	-19	histograms ↓
col=Customer -i1000, fs=default, optee=oob, iso=Y, ddr=n	260522_3	475	266	260525_2	226	133	-249	-133	histograms ↓
col=Customer -i1000, fs=default, optee=prng, iso=n, ddr=n	260531_2	252	72	260528_1	83	63	-169	-9	histograms ↓
col=Customer -i1000, fs=default, optee=prng, iso=Y, ddr=n	260529_3	251	159	260523_2	101	135	-150	-24	histograms ↓

Customer test (-i200 -M) pairs (2)

Aggregate Δmax (B − A) where A=no DDR mods, B=DDR mods, so negative = DDR mods better:
Core0: mean Δmax0 = -50.0 µs | DDR mods better in 2/2 pairs, worse in 0, tied in 0.
Core1: mean Δmax1 = -46.0 µs | DDR mods better in 2/2 pairs, worse in 0, tied in 0.

condition (fixed)	no DDR file	max0	max1	DDR file	max0	max1	Δmax0	Δmax1	plot
col=Customer -i200 -M, fs=default, optee=oob, iso=Y, ddr=n	260520_2	127	113	260525_4	126	98	-1	-15	histograms ↓
col=Customer -i200 -M, fs=default, optee=prng, iso=Y, ddr=n	260521_2	152	147	260523_1	53	70	-99	-77	histograms ↓

Aggregate trends across multiple conditions (Customer -i1000)

Splitting all 16 customer-test runs by DDR-mods state (without controlling for any other factor):

No DDR mods, Core1 max: 72, 73, 78, 72, 180, 149, 266, 159 µs. Median ≈ 113 µs.
DDR mods, Core1 max: 49, 63, 69, 59, 113, 116, 133, 135 µs. Median ≈ 91 µs.

The two distributions overlap near the bottom (when isolation is off, Core1 max is already low and DDR mods only nudge it down a bit), and tighten substantially in the isolated cases (266 → 133, 159 → 135, 180 → 113, 149 → 116).

Conclusions

Always enable DDR mods for the customer test. 8/8 pairs improve on both Core0 and Core1 max. No counter-evidence in the data.
DDR mods do not meaningfully change the SDK test.
Stacking DDR mods with isolation OFF gives the lowest customer-test Core1 max seen in the data. The two no-iso DDR-mods PRNG runs are 260528_1 (max1=63) and 260528_2 (max1=49) — the two lowest customer-test Core1 maxes in the entire data set.
Best-known customer-test configuration in this data: Default FS, OPTEE-PRNG, no isolation, DDR mods → 260528_1, max1 = 63 µs. The base FS counterpart (260528_2) is 49 µs, but the FS difference is within the run-to-run swings seen elsewhere in the data, so the FS contribution alone cannot be confirmed from a single pair.

Caveats

Only 2 DDR pairs exist in the -i200 -M column, so the directional agreement there should be treated as supporting evidence, not independent confirmation.
The exact DDR register values used per run are listed in the test output; further DDR QoS background is available in a separate document.

Appendix — per-pair histogram overlays

Reading the plots: filled circles are the baseline (A) run, "x" markers are the changed (B) run. The top panel is Core 0, the bottom panel is Core 1, both on a log-y count axis. A bump in the A curve that vanishes in the B curve is direct visual evidence that the factor under test produced or removed a latency cluster.

SDK test

fs=base, OPTEE=oob, iso=n, DDR=n | no DDR=260522_2 (max0=51, max1=122) vs DDR mods=260524_1 (max0=72, max1=85) ↑ back to top

fs=base, OPTEE=oob, iso=Y, DDR=n | no DDR=260530_1 (max0=108, max1=102) vs DDR mods=260524_3 (max0=93, max1=84) ↑ back to top

fs=base, OPTEE=prng, iso=n, DDR=n | no DDR=260522_4 (max0=54, max1=63) vs DDR mods=260523_3 (max0=54, max1=83) ↑ back to top

fs=base, OPTEE=prng, iso=Y, DDR=n | no DDR=260531_1 (max0=62, max1=37) vs DDR mods=260527_1 (max0=67, max1=37) ↑ back to top

fs=default, OPTEE=oob, iso=n, DDR=n | no DDR=260518_1 (max0=126, max1=66) vs DDR mods=260526_4 (max0=104, max1=107) ↑ back to top

fs=default, OPTEE=oob, iso=Y, DDR=n | no DDR=260530_4 (max0=101, max1=111) vs DDR mods=260526_2 (max0=104, max1=99) ↑ back to top

fs=default, OPTEE=prng, iso=n, DDR=n | no DDR=260519_1 (max0=62, max1=55) vs DDR mods=260531_4 (max0=81, max1=68) ↑ back to top

fs=default, OPTEE=prng, iso=Y, DDR=n | no DDR=260530_2 (max0=61, max1=36) vs DDR mods=260524_2 (max0=64, max1=37) ↑ back to top

Customer test (-i1000)

fs=base, OPTEE=oob, iso=n, DDR=n | no DDR=260529_1 (max0=460, max1=72) vs DDR mods=260528_3 (max0=203, max1=69) ↑ back to top

fs=base, OPTEE=oob, iso=Y, DDR=n | no DDR=260529_4 (max0=250, max1=180) vs DDR mods=260525_1 (max0=240, max1=113) ↑ back to top

fs=base, OPTEE=prng, iso=n, DDR=n | no DDR=260531_3 (max0=198, max1=73) vs DDR mods=260528_2 (max0=83, max1=49) ↑ back to top

fs=base, OPTEE=prng, iso=Y, DDR=n | no DDR=260530_3 (max0=175, max1=149) vs DDR mods=260526_1 (max0=72, max1=116) ↑ back to top

fs=default, OPTEE=oob, iso=n, DDR=n | no DDR=260529_2 (max0=269, max1=78) vs DDR mods=260528_4 (max0=251, max1=59) ↑ back to top

fs=default, OPTEE=oob, iso=Y, DDR=n | no DDR=260522_3 (max0=475, max1=266) vs DDR mods=260525_2 (max0=226, max1=133) ↑ back to top

fs=default, OPTEE=prng, iso=n, DDR=n | no DDR=260531_2 (max0=252, max1=72) vs DDR mods=260528_1 (max0=83, max1=63) ↑ back to top

fs=default, OPTEE=prng, iso=Y, DDR=n | no DDR=260529_3 (max0=251, max1=159) vs DDR mods=260523_2 (max0=101, max1=135) ↑ back to top

Customer test (-i200 -M)

fs=default, OPTEE=oob, iso=Y, DDR=n | no DDR=260520_2 (max0=127, max1=113) vs DDR mods=260525_4 (max0=126, max1=98) ↑ back to top

fs=default, OPTEE=prng, iso=Y, DDR=n | no DDR=260521_2 (max0=152, max1=147) vs DDR mods=260523_1 (max0=53, max1=70) ↑ back to top