Methodology

NO MEASURABLE CHANGE, and specifically NOT an improvement.

Measured on the full 1319-item gsm8k set on both sides: 0.3821 +/- 0.0134 baseline against 0.4094 +/- 0.0135 optimized.

The gap is 1.43 sigma, inside the noise band, so the two builds are statistically indistinguishable on this instrument.

The ratio reads as 107 percent recovery and that is noise: a quantized build does not gain reasoning ability, and any recovery above 100 percent is reported as indistinguishable rather than as a gain.

INSTRUMENT NOTE,

because it changes how this number should be read: lm-eval's own filters are not valid for a model whose chat template injects the thinking opener into the prompt.

Its strict-match filter returns a structural 0.0, and its flexible-extract filter scores the last number in the response, which grades answer formatting rather than correctness.

Measured example from this very comparison: on a problem whose answer is 18, both builds computed 18, but the BF16 response trailed off with '16' and scored wrong while FP8 ended on '18' and scored right.

Under that filter the same two runs show a +0.0902 gap;

swapping only the extractor removed 70 percent of it.

The figures above use a deliberation-aware extractor on both sides, with an unparsed rate of 0.000 baseline and 0.002 optimized, so extraction succeeded on essentially every sample.

Protocol:

seed 1234, zero-shot, chat template applied, 14000 generation budget, identical settings and the same H100 class on both sides, optimized side bound to the shipped artifact sha256 37cfa0aa.

The gate verdict covers this gsm8k metric;

ifeval, tool calling and safety refusal delta are recorded separately in accuracy-merged.json and all three returned a gate-eligible state with no exemption.