unsloth/studio/backend
Nilay 22493242a3
Studio: Don't re-prompt finished answers in the tool loop (#7505)
* don't re-prompt finished answers in the tool loop

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* keep a separate post-tool reprompt budget and tighten the intent regexes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reset the repeat guard after a tool runs and suppress 'I should call ...' forced stalls

* Cover 'must' in forced-retry suppression, keep appended answers, and count RAG autoinject as a prior tool run

* Anchor obligation suppression to sentence starts and wire the repeat guard into the safetensors loop

* Keep deletions out of restatement and nudge pronoun-free first-step plans

* Tighten repeat similarity, anchor subjectless plans, and restore first-step plan forms

* Keep first-person plan framing and punctuation-bearing terms out of repeat detection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep leading term punctuation, accept colon-delimited first steps, and drop invoke/query from suppression

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments on the plan-without-action re-prompt guards

* Compare plans by token sequence, suppress subjectless modals, and accept dash-delimited first steps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: narrow the first-step plan match and make repeat detection content-based

Restrict the bare "First, <word>" intent alternative to a pronoun, an explicit
plan, or an investigative verb, so ordinal prose ("First place went to Alice")
and user-facing advice ("First, install the package") no longer count as a plan
without action.

Keep punctuation-only tokens in the repeat comparison, so "the value is 5" and
"the value is < 5" stay distinct, and compare content-word sequences instead of
a similarity ratio: any ratio is length-dependent, so one corrected token in a
54-token plan still scored 0.98 and cost the model its remaining nudge.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten comments in the plan-without-action re-prompt path

* studio: keep a forced retry that pivots from a plan to an answer

The obligation-plan branch discarded the whole turn, so a retry such as
"I should call web_search, but the answer is Tokyo." reached the user as
nothing at all. Suppress the plan only when nothing follows it: a pivot
after the match keeps the output, and _FINAL_ANSWER_SIGNAL now recognises
"the answer is" and "to summarise" alongside "answer:".

Leaking a plan sentence is cosmetic, dropping an answer is not, so the
doubtful case now resolves towards shipping the turn.

* studio: keep articles in repeat comparison and exclude missing-answer phrasing

Articles are not filler: dropping them made "search for The Who" and
"search for Who" compare equal, so a corrected target ended the nudge.

_FINAL_ANSWER_SIGNAL matched "the answer is not in the provided context",
which announces a missing answer, so the plan behind it shipped as the final
response instead of being suppressed. Negated forms are now excluded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten the pivot and final-answer signals, drop filler-insensitive repeats

The purpose clause in "call web_search to summarize the results" matched the
final-answer signal, so the plan shipped instead of being suppressed; that
alternative is gone. A pivot word now has to carry text of its own, since
"I should call web_search, though." answers nothing.

Repeat detection no longer ignores filler words. No word is reliably filler:
dropping them to absorb rewording also absorbed the target ("OK Go" became
"Go"). A missed repeat costs one nudge out of the cap; a false one strands the
plan unexecuted.

* studio: exempt offers of help, and add a measured accuracy floor

Offering to help hands control back exactly like the existing "let me know"
exemption. On a corpus of real model turns, "I'll do my best to help" and
"allow me to assist" close a clarification request and never precede a tool
call, but they were read as intent and re-prompted. "help you" keeps its plan
reading when an action verb follows it.

The new test scores the classifier against 300 turns captured from three local
GGUF models, each one a finished answer: the turn called no tool, and three
regenerations behind the production nudge produced no tool call either. Over
those turns, wasted nudges go from 36 (12.0%) on main to 5 (1.7%), and retries
whose text would be discarded from 60 (20.2%) to 1 (0.3%).

Until now these patterns were tuned on hand-written example sentences, which
cannot show how often the classifier is right on real output.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-29 02:38:17 -07:00
..
assets feat(studio): add DoRA support to studio (#7315) 2026-07-24 03:24:16 -07:00
auth Studio: reset-password rotates the credential in place instead of deleting auth.db (#7573) 2026-07-29 01:40:12 -07:00
core Studio: Don't re-prompt finished answers in the tool loop (#7505) 2026-07-29 02:38:17 -07:00
hub Studio: read model text as utf-8 so umlauts survive on Windows (#7467) 2026-07-28 21:27:27 -07:00
loggers Studio: read model text as utf-8 so umlauts survive on Windows (#7467) 2026-07-28 21:27:27 -07:00
models feat(studio): adjustable llama-server parallel slots from the web UI (#7447) 2026-07-28 18:03:28 -07:00
picker Studio: add configurable model download location (#7274) 2026-07-23 01:34:38 -07:00
plugins Studio: read model text as utf-8 so umlauts survive on Windows (#7467) 2026-07-28 21:27:27 -07:00
requirements Installer: stop requiring a developer toolchain on the consumer path (#7547) 2026-07-28 18:50:38 -07:00
routes Studio: reset-password rotates the credential in place instead of deleting auth.db (#7573) 2026-07-29 01:40:12 -07:00
state feat(studio): run chats in parallel in the Chat tab (#7455) 2026-07-28 04:40:38 -07:00
storage Studio: add Deep Research (#7219) 2026-07-26 23:36:02 -07:00
tests Studio: Don't re-prompt finished answers in the tool loop (#7505) 2026-07-29 02:38:17 -07:00
utils Studio: read model text as utf-8 so umlauts survive on Windows (#7467) 2026-07-28 21:27:27 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
cloudflare_tunnel.py Studio: read model text as utf-8 so umlauts survive on Windows (#7467) 2026-07-28 21:27:27 -07:00
colab.py Let a decode failure degrade instead of escaping a fail-closed helper (#7487) 2026-07-27 03:26:08 -07:00
main.py Show release notes in the update popup, sourced from CHANGELOG.md (#7432) 2026-07-28 21:26:43 -07:00
mcp_server.py Replace standalone Studio wording with Unsloth (#7221) 2026-07-19 00:47:04 -07:00
run.py Studio: Stop every running Unsloth server, not just the last one recorded (#7577) 2026-07-29 01:56:13 -07:00
startup_banner.py Replace standalone Studio wording with Unsloth (#7221) 2026-07-19 00:47:04 -07:00