ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

perf(studio): remove quadratic region scan in LaTeX preprocessing (#7538)

Browse source 
findCodeBlockRegions scanned every region found so far for each inline code
match, and accepted inline spans were appended to the same array, making it
quadratic in the number of inline spans. preprocessLaTeX runs on the full
message text every animation frame while streaming and calls it twice.

Fenced and inline matches are both ascending and non-overlapping, so walk the
fenced list with a cursor instead. Only fenced regions can contain an inline
span, so previously accepted inline regions never needed checking.

34,670 chars with 2,100 inline spans: 5.51ms per call to 0.12ms.

Co-authored-by: shimmyshimmer <info@unsloth.ai>
This commit is contained in:
Michael Han 2026-07-28 05:47:48 -07:00 • committed by GitHub
commit 2989b178e1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 9 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

17
studio/frontend/src/lib/latex.ts
View file

@ -33,19 +33,20 @@ function findCodeBlockRegions(content: string): Array<[number, number]> {
regions.push([match.index, match.index + match[0].length]);
}
// Inline code: `...` (skip spans inside fenced blocks, filtered below)
// Inline code: `...`, skipped when inside a fenced block. Both loops yield
// ascending matches, so walk the fenced list with a cursor rather than
// rescanning it per match (was quadratic on code-heavy text).
const fencedCount = regions.length;
const inlineRe = /`[^`\n]+`/g;
let fencedIndex = 0;
while ((match = inlineRe.exec(content)) !== null) {
const start = match.index;
const end = start + match[0].length;
let inside = false;
for (const [rs, re] of regions) {
if (start >= rs && end <= re) {
inside = true;
break;
}
while (fencedIndex < fencedCount && regions[fencedIndex][1] <= start) {
fencedIndex += 1;
}
if (!inside) {
const fenced = fencedIndex < fencedCount ? regions[fencedIndex] : null;
if (!(fenced && start >= fenced[0] && end <= fenced[1])) {
regions.push([start, end]);
}
}