ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: harden re-prompt artifact regex for CRLF + catastrophic backtracking

Browse source 
Two robustness fixes for the `_HAS_ANSWER_ARTIFACT` regex from the
parent commit, both caught by a thorough simulation suite covering
Linux/Mac/Windows line-ending portability and adversarial inputs.

1. **CRLF line endings.** The original `\n` literals missed Windows-
   authored or CRLF-converted content (model echoing a pasted prompt,
   etc.). Replaced with `\r?\n` everywhere a newline is required, so
   closed code fences, numbered lists, and end-to-end re-prompt
   decisions all work on `\r\n` as well as `\n`.

2. **Catastrophic backtracking on whitespace spam.** The numbered-list
   alternative `(?:^|\r?\n)\s*\d+\.\s+\S.*?\r?\n\s*\d+\.` was
   O(n^2) on long whitespace runs: `\s*` greedy + `\d+` failing +
   `\s` matching `\r\n` led to repeated backtracking through the
   newline characters. Measured at ~630ms for 10KB of `\r\n` repeats.
   Fix: restrict the post-newline indent to `[ \t]*` (spaces / tabs
   only). After `\r?\n` we are at column 0 and only spaces / tabs
   are a sensible leading indent for a list item; greedy whitespace
   was never needed. New worst case on the same input: <1ms (1000x
   speedup).

Added 5 in-tree tests:
  - test_artifact_regex_handles_crlf_code_fence
  - test_artifact_regex_handles_crlf_numbered_list
  - test_artifact_regex_handles_mixed_lf_crlf
  - test_no_backtrack_on_crlf_spam (asserts <50ms on 10KB \r\n)
  - test_no_reprompt_on_crlf_complete_python_game

All 18 reprompt-guard tests pass. All 253 llama_cpp-related tests pass.
Out-of-tree simulation suite (84 tests) passes on both Python 3.12 and
Python 3.13 inside isolated uv venvs.
This commit is contained in:
Daniel Han 2026-05-23 02:35:13 +00:00 • committed by danielhanchen
commit 2db8b81854
2 changed files with 65 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

11
studio/backend/core/inference/llama_cpp.py
View file

@ -79,12 +79,19 @@ _MAX_REPROMPTS = 3
# re-prompt and the next user-visible message wipes the code. We
# require ALL of (intent signal, length < _REPROMPT_MAX_CHARS, no
# answer artifact) to fire.
#
# `\r?\n` is used everywhere a newline is required so Windows-authored or
# CRLF-converted content still matches. The numbered-list indent uses
# `[ \t]*` (spaces / tabs only) rather than `\s*` so the regex stays
# linear on long whitespace runs -- greedy `\s*` + failing `\d+` caused
# O(n^2) backtracking through embedded `\r\n` characters on adversarial
# inputs.
_HAS_ANSWER_ARTIFACT = re.compile(
r"```[a-zA-Z]*\n[\s\S]+?\n```" # closed code fence
r"```[a-zA-Z]*\r?\n[\s\S]+?\r?\n```" # closed code fence
r"|<!doctype\b" # HTML page
r"|<html\b"
r"|<svg\b[\s\S]*?</svg>" # complete SVG
r"|(?:^|\n)\s*\d+\.\s+\S.*?\n\s*\d+\.", # 2+ numbered list items
r"|(?:^|\r?\n)[ \t]*\d+\.[ \t]+\S.*?\r?\n[ \t]*\d+\.", # 2+ numbered list items
re.IGNORECASE,
)