Addresses three follow-ups flagged on the first cut of this PR by static
reviewers and parallel reviewer runs:
1. Numbered plan-only stalls were treated as completed answers. A
response like `Here's my plan:\n1. Search the web\n2. Summarise`
matched both `_INTENT_SIGNAL` and the numbered-list branch of
`_HAS_ANSWER_ARTIFACT`, so the tool-forcing re-prompt was skipped.
That contradicted the PR's stated invariant that plan-only stalls
still re-prompt. The list now has to be paired with no plan framing
(no `Here's my plan` / `plan:` / `approach:`, no intent phrase
followed by a tool-action verb) to count as an artifact.
2. Closed code fences with non-alpha info strings (`python3`, `c++`,
`c#`, `objective-c`, `ts-node`, `bash-session`, `python linenums="1"`)
were not recognised by the `[a-zA-Z]*` info-string class. Complete
answers in those languages still re-prompted and could be wiped.
The info-string class is now `[^\r\n]{0,200}` and the closing fence
may be indented.
3. Bare `<!doctype` or `<html` text was treated as an artifact. A
plan-only response that mentions `<html>` in prose now no longer
bypasses the re-prompt; the HTML branch requires a closing
`</html>` (doctype prefix optional).
All `[\s\S]{...}?` runs are length-bounded so ReDoS-style adversarial
input stays linear. ReDoS guard tests cover CRLF spam and repeated
`<html ` openings without close.
Two robustness fixes for the `_HAS_ANSWER_ARTIFACT` regex from the
parent commit, both caught by a thorough simulation suite covering
Linux/Mac/Windows line-ending portability and adversarial inputs.
1. **CRLF line endings.** The original `\n` literals missed Windows-
authored or CRLF-converted content (model echoing a pasted prompt,
etc.). Replaced with `\r?\n` everywhere a newline is required, so
closed code fences, numbered lists, and end-to-end re-prompt
decisions all work on `\r\n` as well as `\n`.
2. **Catastrophic backtracking on whitespace spam.** The numbered-list
alternative `(?:^|\r?\n)\s*\d+\.\s+\S.*?\r?\n\s*\d+\.` was
O(n^2) on long whitespace runs: `\s*` greedy + `\d+` failing +
`\s` matching `\r\n` led to repeated backtracking through the
newline characters. Measured at ~630ms for 10KB of `\r\n` repeats.
Fix: restrict the post-newline indent to `[ \t]*` (spaces / tabs
only). After `\r?\n` we are at column 0 and only spaces / tabs
are a sensible leading indent for a list item; greedy whitespace
was never needed. New worst case on the same input: <1ms (1000x
speedup).
Added 5 in-tree tests:
- test_artifact_regex_handles_crlf_code_fence
- test_artifact_regex_handles_crlf_numbered_list
- test_artifact_regex_handles_mixed_lf_crlf
- test_no_backtrack_on_crlf_spam (asserts <50ms on 10KB \r\n)
- test_no_reprompt_on_crlf_complete_python_game
All 18 reprompt-guard tests pass. All 253 llama_cpp-related tests pass.
Out-of-tree simulation suite (84 tests) passes on both Python 3.12 and
Python 3.13 inside isolated uv venvs.
The plan-without-action re-prompt at
`studio/backend/core/inference/llama_cpp.py` fires when the model
emits intent-only language ("first I'll ...", "let me ...") without
calling a tool. Previously the heuristic only checked an intent regex
and a 2000-char length cap. The same intent words occur in long
explanations that accompany REAL code or markup, so a complete reply
like "First, let me set up pygame. ```python ... ```" still tripped
the re-prompt, and the synthetic follow-up ("STOP. Do NOT write code
or explain.") wiped the user-visible answer.
Reproduced at scale in a 900-run sweep across 15 Qwen3.5/3.6 GGUF
configs: prompts that emit code or markup (Create a Python game,
Create a Flappy Bird game, weather dashboard HTML, sloth SVG)
landed empty `final_text` for the majority of seeds even on the
strongest configs.
Fix adds a `_HAS_ANSWER_ARTIFACT` regex covering:
- closed code fences (```...```)
- HTML pages (<!doctype, <html)
- complete SVG (<svg...</svg>)
- 2+ item numbered lists
and a `and not _HAS_ANSWER_ARTIFACT.search(_stripped)` guard on the
re-prompt condition. Plan-only stalls still re-prompt; complete
responses no longer do.
13 new unit tests in `test_llama_cpp_reprompt_guard.py` pin both
directions (artifact present -> no re-prompt; plan-only -> still
re-prompts).
