c800e89206
# Conflicts: # studio/backend/core/inference/diffusion.py # studio/frontend/src/features/images/images-page.tsx
23 lines
No EOL
4.1 KiB
Markdown
23 lines
No EOL
4.1 KiB
Markdown
- Decision: shipped compile path uses `transformer.compile_repeated_blocks(fullgraph = True, dynamic = True)`, which forwards to `torch.compile`; effective settings are `fullgraph=True`, `dynamic=True`, and default `mode`. This was chosen because Studio can vary resolution/batch/steps without excessive recompiles.
|
|
- Decision: CUDA graphs are not enabled in the shipped path. Per `https://docs.pytorch.org/docs/stable/generated/torch.compile.html`, CUDA graphs come from `mode="reduce-overhead"` or `mode="max-autotune"`, while default mode gives Inductor fusion without graph capture.
|
|
- Decision: `dynamic=True` conflicts with CUDA graphs because graph capture expects static shapes. Switching to CUDA graphs would require `dynamic=False` and recompilation for new shapes.
|
|
- Decision: after ablation, `default` mode + `dynamic=True` remained the correct shipped choice because it was essentially as fast as autotune, much faster to compile, robust to shape changes, and avoided CUDA-graph runtime crashes.
|
|
- Decision: `max-autotune-no-cudagraphs` worked but was not worth enabling as default: `0.78 s` vs `0.80 s` was within run-to-run noise, while compile time increased from `5.7 s` to `66.2 s` and required `dynamic=False`.
|
|
- Decision: CUDA graph modes were rejected because they crashed in this pipeline, not merely because they were theoretically fragile.
|
|
- Clarification made: since `fullgraph=True` succeeds, there is no Dynamo graph break in the block; GGUF dequant is compile-traceable. The remaining inefficiency is Inductor declining codegen for Z-Image complex-valued ops: `"does not support code generation for complex operators"`.
|
|
- File context mentioned but not edited in this span: `diffusion_speed.py` contains the shipped compile call.
|
|
- File context mentioned but not edited in this span: `compile_probe.py` already has a `--mode` flag and was used for mode ablation.
|
|
- Command run: `LOG=logs/compile_modes_$(date +%Y%m%d_%H%M%S).log; echo "COMPILE MODES LOG: $LOG"; { for spec in "default:--mode default --dynamic" "maxautotune-nocg:--mode max-autotune-no-cudagraphs" "reduce-overhead:--mode reduce-overhead" "max-autotune:--mode max-autotune"; do ...` to ablate torch compile modes including CUDA graphs on GPU 6.
|
|
- Command status: background task `b01iys6fy` completed with exit code `0`.
|
|
- Command output file: `/home/ubuntu/CLAUDE_CODE_TMPDIR/claude-1000/-mnt-disks-unslothai-ubuntu-workspace-81/8723b9e3-2ab1-49e9-a4b1-d030954eb94a/tasks/b01iys6fy.output`.
|
|
- Command run: `cat "$(ls -t /mnt/disks/unslothai/ubuntu/workspace_81/unsloth/logs/compile_modes_*.log | head -1)"` to read the latest compile-mode ablation log.
|
|
- Key result: `default` with `--mode default --dynamic` gave eager `1.82s/gen`, compiled `0.80s/gen`, `+56.3% vs eager`, PSNR `37.7 dB`, compile time `5.7s`, verdict `COMPILE-WORKS FASTER`.
|
|
- Key result: `max-autotune-no-cudagraphs` with `dynamic=False` gave eager `1.83s/gen`, compiled `0.78 s`, `+57.2%`, PSNR `37.3 dB`, compile time `66.2 s`, and worked.
|
|
- Key result: `reduce-overhead` with CUDA graphs and `dynamic=False` failed.
|
|
- Key result: `max-autotune` with CUDA graphs and `dynamic=False` failed.
|
|
- Error encountered: both CUDA graph modes raised `RuntimeError: accessing tensor output of CUDAGraphs that has been overwritten by a subsequent run` at `transformer_z_image.py:271, in forward`.
|
|
- Error analysis: likely caused by regional compile plus CUDA graph output buffer reuse; denoiser calls the same compiled block repeatedly and Z-Image carries an output across calls, so the static CUDA graph output buffer is overwritten before a later read.
|
|
- Error resolution: resolved by keeping CUDA graphs disabled; no code changes were made to try manual `cudagraph_mark_step_begin()` or output cloning.
|
|
- Completed: answered why CUDA graphs were not being used and measured `default`, `max-autotune-no-cudagraphs`, `reduce-overhead`, and `max-autotune`.
|
|
- Completed: reported mode-by-mode speed, PSNR, compile time, and failure status.
|
|
- Pending: optional follow-up offered to wire `mode="max-autotune-no-cudagraphs"` into the opt-in `max` tier as a gated one-line change, leaving default unchanged. |