dataprep: detect vllm 0.19 server readiness (renamed log line, stderr)

SyntheticDataKit.from_pretrained waits for 'Starting vLLM API server on'
in the child's stdout before declaring the server up. vLLM 0.19 renamed
the line to 'Starting vLLM server on ...', so the regex never matched,
the 1200 s readiness timeout expired with a perfectly healthy server,
and the launcher tore it down; every downstream synthetic-data-kit step
then failed on missing files. Accept both wordings, watch stderr too
(vLLM has moved its logging between pipes across versions), and bail
out of the wait early if the child exits.

With this plus the ninja-build and flashinfer-jit-cache image fixes the
Meta synthetic data notebook goes from a 21 min timeout-and-fail to a
3 min pass inside the container.
This commit is contained in:
Daniel Han 2026-06-12 12:15:29 +00:00
commit 487ea4f5b9

View file

@ -267,7 +267,10 @@ class SyntheticDataKit:
stderr = subprocess.PIPE,
start_new_session = True,
)
ready_re = re.compile(r"Starting vLLM API server(?:\s+\d+)?\s+on\b")
# vLLM <= 0.18 logs "Starting vLLM API server on ..."; 0.19 renamed it
# to "Starting vLLM server on ...". Accept both, with the optional
# server index some versions insert before "on".
ready_re = re.compile(r"Starting vLLM(?:\s+API)?\s+server(?:\s+\d+)?\s+on\b")
self.vllm_process = vllm_process
self.stdout_capture = PipeCapture(
vllm_process.stdout,
@ -282,12 +285,24 @@ class SyntheticDataKit:
keep_lines = 2000,
echo = False,
name = "vLLM STDERR",
ready_regex = None,
# vLLM >= 0.19 emits "Starting vLLM API server ... on ..." (and
# the uvicorn startup lines) through the logging module, which
# writes to STDERR. Watching stdout alone makes a healthy server
# look like a startup timeout, after which we kill it.
ready_regex = ready_re,
text = False,
)
# we don't print stderr to console but self.stderr_capture.tail(200) will print the last 200 lines
ready = self.stdout_capture.wait_for_ready(timeout = timeout)
ready = False
deadline = time.monotonic() + (timeout or 1200)
while time.monotonic() < deadline:
if self.stdout_capture.wait_for_ready(timeout = 1) or \
self.stderr_capture.wait_for_ready(timeout = 0):
ready = True
break
if self.vllm_process.poll() is not None:
break
if not ready:
if self.stdout_capture.has_closed() or self.vllm_process.poll() is not None:
print("Stdout stream ended before readiness message detected.")