Previously the vanilla reference was a Gemma4ForCausalLM shell wrapping
the language_model (the same construction used inside
gemma4_flex_inference.main for LoRA / state-dict hashing convenience).
That is not plain HF: `Gemma4Model.forward` uses
`create_masks_for_generate(..., mm_token_type_ids, pixel_values)` to
build attention masks, while the shell calls `Gemma4TextModel.forward`
directly, which builds its own per-regime masks via `create_causal_mask`
+ `create_sliding_window_causal_mask`. Both are correct for text-only
input but their mask-bias precision differs enough to produce a
measurable drift.
The script now keeps both references alive and reports three diffs:
shell vs raw, flex vs raw, flex vs shell. On unsloth/gemma-4-E2B-it
bf16 with a 6-token prompt:
shell vs raw max 6.9e-01 mean 3.0e-01 argmax=yes top-10=10/10
flex vs raw max 6.3e-01 mean 2.2e-01 argmax=yes top-10=10/10
flex vs shell max 3.8e-01 mean 8.0e-02 argmax=yes top-10=10/10
Flex is actually closer to raw HF than the shell is. About 0.30 mean of
the flex-vs-shell-and-vs-raw gap comes from the shell's mask
construction alone, not the flex kernel. Either way the bf16 drift
band matches Qwen3 (0.3 / 0.09) and Llama-3.2 (0.13 / 0.02), and
semantic top-1 + top-10 are exact.
b294fbd3dc