Show release notes in the update popup, sourced from CHANGELOG.md (#7432)
* Show release notes in the update popup, sourced from CHANGELOG.md The update banner only linked out to the online changelog, so there was no way to see what an update contains before taking it. Add CHANGELOG.md at the repo root as the source of release notes. Studio reads it from the default branch, so editing the file updates the popup without a release or rebuild, and falls back to the copy bundled in the install when the repo is unreachable. Notes are matched to one exact version. The popup asks for the version it is offering and gets that section or nothing, so an older release's notes can never appear next to a newer update. When there is no match the popup links out to the online changelog instead. The collapsed popup previews the top bullets with the leading sentence highlighted; "Show release notes" expands the full notes in a scrollable panel. Applies to both the browser and desktop banners, and the desktop updater's own release body is used when CHANGELOG.md has no matching section. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: fence matching, nested bullets, BOM, updater notes field Track the opening fence marker and length so a ``` sample inside a ```` block does not close it early and let the sample's heading be indexed as a real release. Preserve list indentation in the preview and take only top-level bullets, so nested detail no longer consumes the four headline slots. Strip a UTF-8 BOM before parsing. An editor on Windows can leave one on the first line, which hid a section whose heading started the file. Read `notes`/`pub_date` from latest.json in the manual Linux updater path, with aliases for the older `body`/`date`. The workflow publishes Tauri's field names, so the manual path's release body was always empty. Also loop the preview tag strip until stable for CodeQL js/incomplete-multi-character -sanitization; the value renders as text, so this is defence in depth. * Address review: bare fence closers, HTML comments, underscores, notes URL A closing fence must carry nothing after the delimiter, so a ```` line with trailing text inside a ```` block is content rather than the end of it. Both the parser and the preview extractor follow that rule now. Skip headings inside HTML comments. A commented-out section is not rendered by Markdown, so it must not be indexed as a release. Strip only paired emphasis and park code spans first, so identifiers keep their underscores: UNSLOTH_DISABLE_UPDATE_CHECK was previewing as UNSLOTHDISABLEUPDATECHECK. Prefer the caller's release URL over the API's generic changelog link, so the desktop fallback points at the release page for the version being offered. Look at the repo-root CHANGELOG.md before the packaging snapshot, and remove the snapshot after build.sh, so an edited root file is never shadowed by a stale copy. Also nudge the notes container radius from 16px to 14px. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: comparison operators, hidden comments, remote failures Require a name character after "<" when stripping tags. A bullet reading "Support Python <3.15 and >3.9" previewed as "Support Python 3.9", because the operators were consumed as if they were a tag. Track HTML comments while collecting preview lines. A commented-out bullet was previewed as a published change even though Markdown never renders it. Report a remote lookup failure whenever nothing matched. The bundled changelog cannot know a version newer than the install, so discarding the error made an offline lookup read as "no notes were published". The hook now treats a reported failure as its retryable error state. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: code-span delimiters, stale notes, retry past cached failures Treat an HTML comment delimiter inside inline code as literal. A note reading "Type `<!--` to begin a comment" put the parser into comment state, so every release below it was swallowed into the entry above and became unfindable. Applied to the preview extractor too. Return no notes while the offered version differs from the fetched one. On the render where the version changes, the hook still held the previous release's notes, which the panel would show for a frame. Let retry bypass a cached remote failure via a refresh flag on the endpoint. Failures are cached for five minutes, so the visible Retry action could not recover until the TTL expired. A cached success is still reused, so retries cannot hammer the remote. * Address review: CommonMark indentation, desktop release notes link Allow up to three leading spaces on release headings and fences, and treat four as indented code. An indented heading was unreachable and its notes were appended to the release above, while an indented backtick line opened a fence that swallowed later headings. Link desktop release notes to the release page for the offered version on every platform. The existing URL is built only in manual Linux package mode, so in-app updates on macOS, Windows and AppImage fell back to the generic changelog. The install button keeps using the manual URL. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: wrapped prose, autolinks, abbreviations in the preview Accumulate contiguous prose lines into one preview item. A paragraph wrapped across source lines renders as one block but previewed as three fragments, which also ate the four-item limit. Keep Markdown autolinks. <https://example.com/notes> was stripped as if it were a tag, so "See <https://example.com/notes> for details" previewed as "See for details". Do not split the lead sentence at an abbreviation. "Supports several formats, e.g. GGUF and Safetensors." highlighted only up to "e.g." and dimmed the actual change; known abbreviations and single initials are skipped now. * Address review: park code spans first, skip indented code blocks Park code spans before any other inline transformation. Tags, links, images and emphasis inside a span are literal, but the strips ran first, so "Use `<button>` for actions" previewed as "Use for actions". Skip lines inside an indented code block when collecting bullets. A "- pip install ..." line in a four-space-indented block became the headline and pushed out the real prose, though Markdown renders it as code. Continuation lines of an open bullet are unaffected. * Studio: skip raw HTML blocks when reading release notes A <pre>, <script>, <style> or <textarea> block renders literally, so a sample '## 9.9.9' heading inside one was indexed as a release and cut the real section's body short. The preview had the same gap and listed sample bullets as notes. Both readers now track type 1 HTML blocks and skip their contents. Blocks open only at the start of a line, so a tag named mid-sentence stays inline text, and <details> is type 6 so its Markdown still parses. * Studio: read HTML blocks the way CommonMark renders them A fence inside a <pre> block was treated as a real fence, so the block's closing tag was swallowed and every release below it disappeared. Raw HTML state is now checked before fences, in both readers. Type 6 and 7 blocks (<details>, <div>, a bare tag on its own line) run to the next blank line, so a heading pressed against the opening tag is not a release either. Type 7 cannot interrupt a paragraph, so prose followed by a bare tag is unaffected. Checked against a CommonMark reference: 20000 generated well-formed changelogs now agree exactly on which headings are releases, and every previewed note is text the renderer really shows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: restore preview types dropped in the scanner refactor The previous commit's refactor removed the Bullet and preview item interfaces, so tsc -b failed and every job that builds the frontend stopped there. * Studio: fix release-notes preview and packaging review findings Preview: a code span now closes on a run of the same length, so a note containing backticks keeps them; thematic breaks no longer take a preview slot; a quoted list is example output, so it stays out of the headline bullets and is only used when a section has none of its own. Popup: a failed lookup keeps the changelog link beside Retry, which the web banner always offered before, and the desktop popup waits briefly for the auto-auth token instead of recording a failure the user has to clear. Packaging: the changelog snapshot is made by the build backend, so python -m build, pip install . and sdist builds all ship the offline copy, not only build.sh. * Studio: scope the changelog fallback and hide staged sections Installed, the levels above studio/ are site-packages, so a stray CHANGELOG.md left there by another package outranked the bundled snapshot. Those levels are now searched only when a checkout marker (pyproject.toml or .git) is present, so a source checkout still serves the editable file. A section staged as only an HTML comment renders as nothing but was reported as matched, leaving an empty notes surface. Notes that render nothing now read as unpublished, so the popup links out instead. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: cover the remaining raw block forms and repository links Parser and preview: processing instructions, declarations and CDATA are literal like <pre>, so a sample heading or bullet inside one is no longer read as a release. ATX headings now need a space or tab after the hashes, matching CommonMark, so a pasted non-breaking space no longer truncates the release above it. Popup: the notes region follows the viewport and the card scrolls as a backstop, so a window under about 430px high no longer pushes the title and dismiss control off screen. Relative links in the notes resolve against the repository instead of Studio's origin, where the renderer blocked them. * Studio: reference-style images, empty previews and version queries Reference definitions now resolve against the raw host when the label is used as an image, so ![alt][arch] loads the file instead of its HTML page on GitHub. Labels are matched the way CommonMark compares them, and a reference written inside a fenced block does not count. Notes that preview as nothing, such as a lone command block, no longer leave an empty muted strip in the collapsed popup; expanding still shows them. A version query that cannot parse is rejected up front rather than looked up and reported as no notes. * Studio: Markdown scanning fixes across the release notes path Code spans are now scanned rather than matched by pattern, so a run of backticks closes only on a run of the same length. The preview and the link resolver share that scanner, so a link inside `a``b [x](y.md)` stays literal in both. Also: a closing fence may carry only spaces or tabs, so a delimiter with a non-breaking space after it stays code in all three scanners; escaped parentheses in a link target resolve to the literal path instead of being mangled; the collapsed preview decodes entities the way the expanded view renders them, while code spans stay literal; and release notes are fetched through authFetch so an expired access token is refreshed and retried. * Changelog: real 2026.7.5 notes, led by the AMD release Fills the section the popup reads with the actual headline changes, so the collapsed preview shows real content instead of placeholder notes. Leads with AMD support and covers the 23 July update: RDNA2 and Gorgon Halo, Strix Halo detection, RDNA4 and ROCm failure recovery, 2x faster unified memory loading, whisper.cpp dictation, and rollback environment cleanup. * Studio: fix release-notes text handling found by adversarial testing Line endings are normalised first: a CRLF body from the desktop updater no longer hides fences, so a code sample cannot become a headline bullet, and lone CR text splits into bullets. Preview: reference links and images render as their text, a definition line renders as nothing, parentheses in a destination no longer truncate the sentence, escaped punctuation stays literal, and a fence indented into a list item is treated as the block it is. Links: a badge resolves both its image and its outer link, indented code and code spans that cross a line are left alone, a definition cannot interrupt a paragraph, and image alt text no longer decides a label's host. Also: an escaped backtick cannot open a code span, park sentinels in the source cannot swap content, two in-flight requests for one version resolve in order, and repeated bullets no longer share a React key. Comment scanning no longer rescans code spans per delimiter and span lookup is a binary search: the worst inputs measured drop from 96ms to 1ms at the 20k cap, and from 544ms to 15ms at 200k. * Studio: parser and fetch fixes found by adversarial testing A comment marker written in prose no longer swallows the rest of the file. Only a comment that starts a line opens a block; one written mid-sentence is inline HTML and hides its own line at most. This was the worst case found: a single stray marker made every release below it unreachable and served their notes under the newer version's heading. Also in the parser: a closing delimiter takes its whole line, so a heading glued after it is not a release; an exact heading is never shadowed by a zero-padded one; setext headings are release boundaries; any heading, rule or definition ends a paragraph; and the code-span guard is a linear scan rather than a backtracking pattern, so 20k backticks parse in a millisecond instead of over a minute. Fetching: one deadline for the whole response with chunked reads, so a trickling server cannot hold a worker for minutes, waiters give up instead of queueing behind a stalled fetch, and identity encoding is requested so a compressing proxy cannot produce mojibake notes. Truncated notes close an open fence. UI: images and the renderer's own link dialog are held inside the card, which the shared preview's blanket max-width reset had let escape, and only the notes region scrolls so the dismiss control stays reachable on a short viewport. The developer update override no longer beats the documented opt-out, and its value has to parse as a version. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: CommonMark paragraph and block rules across the notes path Setext detection now requires plain paragraph text above the underline. A list item followed by --- is a list and a rule, not a heading: reading it as one discarded the bullet and every note after it. A backtick fence whose info string holds a backtick is not a fence, so such a line no longer swallows the releases below it in the parser, the preview and the link resolver. Preview: only an ordered list starting at 1 interrupts a paragraph, an unresolved reference keeps its brackets, a comment written mid-sentence hides its own line at most instead of the rest of the document, a raw block closer takes its whole line, and a code span closer after a backslash still closes, since escapes do not apply inside a span. Links: raw HTML blocks are literal, an escaped opener is not a link, and a definition under a heading is a definition. The overlay stack is capped to the viewport and both overlays can give up height, so a long download list no longer pushes the update card's title and dismiss control off screen. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: desktop notes by backend version, desktop stack cap, fetch budget latest.json now publishes the backend release the desktop build pins, and both desktop paths carry it: the manual metadata check through Rust, and the in-app updater through the raw metadata it already exposes. The popup looks release notes up by that version, so desktop stops asking CHANGELOG.md for an app SemVer it never contains and falling back to the generic installer text. Metadata without the field still parses and behaves as before. The desktop overlay stack is capped to the viewport like the browser one, since the download panel shares it and the card's own cap cannot see a sibling. The fetch budget now bounds each read, not just the gap between reads. Slow headers followed by a slow body held a worker for 5.6s against a 3s budget; it is 3.0s now, and a timeout is reported as one. * Studio: keep list-nested headings out of the release index A `## <version>` heading indented to a list item's content column is inside that item in CommonMark, not a release boundary. Reading it as one truncated the real release and indexed a version that does not exist. parse_changelog now tracks the open list items by the column their content starts at, and only counts a heading left of that column. Supporting rules, each checked against markdown-it (commonmark preset): a marker needs whitespace after it, so `2.0` stays a setext version; an item interrupts a paragraph only when it has content, and an ordered one only when it starts at 1; an empty item takes one blank line; a dedented fence, break or heading closes the item; and `- ## 2.0` is a heading inside the item. * Studio: whole-paragraph setext headings, uppercase declarations, escaped marks Three CommonMark conformance fixes on the notes path, each checked against markdown-it (commonmark preset). A setext heading is the whole paragraph above the underline, so a heading that wraps kept its version only on the first line while the parser read the last: `2026.7.5 - Release` over `July 25` left that release unindexed and its notes unreachable. The parser now tracks every line of the open paragraph, including lazy continuations, and stops at whatever really interrupts it: a quote marker, a bullet, or an ordered marker starting at 1. A type 4 HTML block needs an uppercase letter after `<!`, so prose mentioning `<!note` was hiding every release below it until the next `>`. In the link resolver, `\![alt][label]` renders as a link, so its definition resolves to the file's page on GitHub rather than the raw-content host. * Studio: the preview needs the uppercase declaration rule too The backend parser stopped treating `<!note` as an HTML block, but the collapsed preview still did, so prose mentioning one emptied the preview of every bullet below it while the expanded notes rendered them. A shipped test now pins the two to the same rule. * Treat an empty HTML comment as closed and always release the changelog fetch flag <!--> and <!---> are complete comments in CommonMark: the closer overlaps the opener, so searching for --> past the opener never found it and the scanner stayed in comment state for the rest of the file. An empty comment used as a section marker hid every release below it, in both the backend parser and the frontend preview. get_remote_changelog cleared its single-flight flag only after except Exception, so a BaseException stranded it and every later caller waited out the full deadline for the life of the process. Move the release into a finally. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Compare resolved changelog paths instead of a hardcoded checkout name The ordering assertion matched the string suffix /unsloth/CHANGELOG.md, so it raised StopIteration in any checkout not literally named unsloth, and on Windows the separator is a backslash so the suffix never matched there either. Both are unrelated to the ordering under test. Verified failing on ubuntu-24.04, macos-14-arm64 and windows-2025 alike, and passing after. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scan backtick runs once instead of rescanning the suffix per opener Every unmatched opener rescanned the rest of the line and the outer loop then advanced by a single run, so a line of runs of 1, 2, 3 ... backticks was quadratic: 321 KB took 7.688s, and release notes are reparsed on every popup request, so one malformed remote changelog could tie up backend workers across installed clients. Collect the runs in one pass and walk a cursor per run length, since a length that runs out of partners stays out. Same 321 KB now takes 0.013s and 5 MB takes 0.205s. Verified identical output against the old implementation on 30000 randomized lines. * Read type 6 and 7 HTML containers in the link resolver too The resolver masked only type 1 blocks (pre, script, style, textarea), while the backend parser and the collapsed preview already apply the type 6 and 7 rules, so the three disagreed on the same notes. A <details> or <div> with no blank line inside is a type 6 block whose contents render verbatim, so two things went wrong there: a relative link was rewritten into text the reader sees literally, and a fence inside the block was taken for a real fence, which silently stopped every link below it from resolving. A blank line, not the closing tag, ends these blocks, so the common '<div align="center">' followed by a blank line still holds Markdown and still resolves. * Mask comments before fences, split only on Markdown line endings, stage the snapshot Three separate reports, all confirmed against head. The link resolver tracked no comment state, so a fence delimiter hidden inside an HTML comment was read as a real fence. The fence then stayed open and every visible line below was classified as code, so none of its links resolved: one commented-out draft containing a stray backtick run silently broke the rest of the notes. Comments are masked now, but only outside a fence, since fenced content is literal and a comment opener in it is not one. Commented ranges join the code spans, so a link the reader cannot see is not rewritten either. Verified with 9 cases under node; 2 fail on the previous file. str.splitlines also breaks on U+2028, U+2029, NEL, vertical tab and form feed, none of which end a line in CommonMark. A separator sitting in prose ahead of "## 9.9.9" made the parser index a release that renders nowhere and truncate the notes above it: measured, the version list went from 2.0, 9.9.9, 1.0 to 2.0, 1.0 and the 2.0 body stopped being cut at the separator. The build wrote the snapshot beside the checked-in sources, so a PEP 517 build against an immutable checkout (Nix, Bazel, a read-only container mount) raised PermissionError before build_py started and produced no wheel at all. The source-tree copy is best effort now and the wheel takes its copy from the staging directory. Reproduced both ways against a read-only package dir. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use the backend's heading and quote marker rules in the preview An ATX heading needs an ASCII space or tab after the marker, which is exactly what _HEADING_PATTERN requires. The \s class also matches a non-breaking space, so prose beginning "## Important change" with one was classified as a heading and discarded by collectBullets, and a prose-only release then had no collapsed preview at all rather than a wrong one. A blockquote marker takes at most three leading spaces, like every other marker in this file. Accepting any run let an indented code sample containing "> - sample output" shed its indentation and enter the collector, so a release with no real bullets showed code as its summary. Both reproduced under node against the real module: the two cases fail on the previous file and pass now, with a real heading, a real quoted bullet and an ordinary bullet unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Collect preview reference labels only from lines that can be definitions A definition-shaped line inside an indented code block or a deep fence is literal text, so CommonMark leaves a later "[Beta] support" unresolved with its brackets showing. The pre-scan ran over every line regardless, so the label was recorded and toPlainText stripped the brackets: the collapsed preview claimed a resolved reference the expanded notes do not have. It now skips the same code the collector pass skips. A real definition takes at most three spaces of indentation, so the indent test cannot reject one, which the second case checks. Reproduced under node: the indented-code definition resolved "Beta support" before and keeps its brackets now. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Let a document-level HTML block close an open list item CommonMark HTML blocks of types 1 to 6 interrupt a paragraph, so a "<div>" to the left of an open list item closes it and a following one-to-three-space indented "## 2.0" is a real document heading. Two things stopped that: the block opener was blanked before the list tracker saw it, so it read as a blank line, and _may_be_lazy treated it as ordinary text that could continue the item's paragraph. The item therefore stayed open and the release below the block was swallowed entirely. The opener's indentation is now taken before it is hidden, the way a fence opener's already was, and an HTML block opener is no longer a candidate for lazy continuation. Type 7 cannot interrupt a paragraph and is deliberately excluded, since after_paragraph is the only state this helper is asked about. Measured on the reported shape: the version list went from 3.0, 1.0 to 3.0, 2.0, 1.0. The test also pins the two cases that must not change, an indented heading genuinely nested in an item and an ordinary lazy continuation, both of which still suppress the heading. * Let the download panel shrink inside the capped overlay stack The bottom-right stack is capped to the viewport, but a flex item defaults to min-height:auto, so the download panel's outer wrapper could not shrink below its own content. min-h-0 had been added to the nested panel and not to this wrapper, so on a short viewport the cap was absorbed by the update card, whose header and actions are fixed, instead of by the download list, which scrolls. Only the shared-stack branch takes it. Standalone is positioned fixed and is not a flex item at all. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten release notes comments Shorten the comments and docs added with the update popup release notes so each explains its line in as few words as possible. Comments only, no behaviour change. * Measure release-notes indentation from the container CommonMark measures a block's indentation from its container, not from the left margin (spec 0.31.2 sections 4.4 and 5.2). The three changelog scanners measured from the margin in different places, so they disagreed with the renderer and with each other. Under "- Details:" the content column is 2, so a four-space line is two columns in: a paragraph holding a link. The link resolver read it as an indented code block and left the destination relative, so it resolved against Studio's own origin instead of the repository. At document level the same four spaces really are code, and a top-level bullet is not indented enough to continue the block. The preview promoted an indented line that looked like a fence opener to a list-contained fence, so with no later closer every bullet below it was skipped and the collapsed popup lost its summary. A fence is scoped to its container too: with no closing line it runs to the end of the containing block, not the end of the document (section 4.5). A dedented "## 2.0" closes the list item the fence sits in, so it is a real release heading. Document-wide fence state kept the block open, so one missing closing line hid every release below it. Both frontend scanners now read their list columns from one module ported from the backend's own tracker, which keeps the three in step. Two smaller fixes ride along. A release body written as a GFM table rendered as a grid but previewed as its raw "| Change | Detail | | --- | --- |" delimiters, so table rows are now dropped from the collapsed summary the way a code block already is. The comment scanner restarted its code-span search at the first span for every opener, so a line of N spans and N openers cost N squared: a 203 KiB line, well inside the 2 MiB the fetcher accepts, took 10.9s and now takes 41ms. Differential fuzzing against a CommonMark reference implementation puts the parser's heading mismatches at 11 of 14275 documents, down from 617, and the link resolver's at 147 of 6000, down from 217. * Keep Retry reachable when the release notes fetch fails The panel took fallbackMarkdown for every response that did not match, error included, so markdown was always truthy on desktop and the error branch that carries the Retry button was unreachable. The fallback there is the updater's static install blurb, not this release's notes, so a transient failure showed "Download the Apple Silicon .dmg" where the notes should be, with no way to ask again until the cache expired. The hook already separates the two: a reported failure is error and retryable, "no section for this version" is ready and is not. The fallback now applies only to the second, which is the case its prop documents. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope an unclosed comment to its block and end a release on a bare ## Two CommonMark rules the changelog scanners read too strictly. An HTML block only opens when the line itself begins with a comment marker (spec 0.31.2 section 4.6, type 2). One written mid-sentence is inline raw HTML and, unclosed, is ordinary text. The link resolver carried the open state to every line below instead, so a note reading "- Type <!-- to begin a comment" masked the relative links under it and they resolved against Studio's own origin rather than the repository. maskComments now separates the block form from the inline one and skips an opener sitting inside a code span, the way _strip_comments and stripCommentSpans already do. The spans are scanned only once an opener turns up, so a line without one costs what it did before. An ATX heading's opening sequence may also be followed by the end of the line (section 4.2), so a bare ## is an empty level-two heading. Both heading patterns required whitespace after the hashes, so everything below such a line stayed inside the release above it and the popup could show unrelated notes under that version. An empty heading carries no version, so it ends the release without indexing one of its own. Differential runs against markdown-it-py: section bodies 7769 to 0 mismatches over 36069 generated documents, comment-heavy link resolution 705 to 53 over 6000, and previews leaking a bare marker as headline text 22484 to 0 over 40000. The residual link cases are all one shape, a comment block opened inside a list item that outlives the item, which the fence tracker scopes and the comment tracker does not, in all three scanners alike. * Give a hidden comment its own column and balance link destinations A comment is an HTML block, so one written at the margin under a bullet is not indented enough to continue that item and closes the list. All three scanners blanked the line before list tracking saw it, which reads as a blank line and leaves the item open, so a release heading below it looked like nested item content and the new release merged into the one above. A hidden line now keeps its own column through _hidden_structure and hiddenStructure, and only its column, since the text a comment or a raw block hides is not Markdown and must not open a list of its own. A line inside a block already open is that block's content and still keeps nothing. A link destination may hold parentheses while they balance, so [x]((draft).md) points at (draft).md. The resolver stopped at the first paren, matched an empty destination and left the markdown alone, so the link resolved against Studio's own origin. The balanced form counts only while a closing paren or a title still ends the link, so the stray paren in [x](a(b.md) stays the closer the way CommonMark reads it rather than being swallowed into a link across lines. * Scope paragraph state to the container a line is written in Two lines the parser read as block starts are lazy paragraph text, so the list they were written under closed early and the heading indented to the item's content column was indexed as a release the renderer never shows. A setext underline may never be a lazy continuation line (spec 0.31.2 section 4.3), so `===` written left of an open item is more of that item's paragraph. Rejecting every underline-shaped line ended the list there. A row of three dashes is still a thematic break, which does end it. Lazy continuation runs the other way too: a marker written outside a blockquote is not text of the quote's paragraph, so `2. item` under `> quote` opens a list even though an ordered marker past 1 may not interrupt a paragraph. Paragraph state is now scoped to its container: a quote line leaves open only the quote's own paragraph, an underline needs one in its own container, a definition ends one only when there is none to continue, and a line four columns past its container is code, which may not interrupt. The frontend pair reads the same tracker, so both scanners now carry the quote state and a fence inside a list item ends with the item in the preview the way it already did on the backend. Measured against markdown-it-py (CommonMark 0.31.2) over 264k generated documents: 3368 sections now match the renderer, none regressed, and every list and quote corpus is exact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read a fence and an HTML block from the container it opens in A block is measured from its container and not from the left margin (spec 0.31.2 sections 4.5 and 5.2), but the link resolver's fence, raw HTML and type 6 expressions all started at the margin, so a fence behind a quote marker and one three columns under a nested bullet opened nothing. The sample inside was then read as prose, and a relative link written in a code block or a details body was rewritten into text the reader is shown verbatim. Matching runs of backticks hid some of it by accident, since the code span scanner pairs them across lines, but a tilde fence, a closer of a different length and every HTML block went through. Each line is now read from the container it is written in, which the list tracker already knew, and a block is scoped to that container the way a fence inside an item already was: a line to the left of the item, or outside the quote, ends the block along with it, and a bare quote marker is the blank line that ends a type 6 block. A destination holds parentheses while they balance, and a path may nest them, so [x](((draft)).md) points at ((draft)).md. One nesting level was all the expression allowed, so anything deeper fell through to the plain form, matched an empty destination and left the link resolving against Studio's own origin. The pairs are unrolled to the 32 levels cmark counts, and the balanced form is still gated on a closer following it, so the stray paren in [x](a(b.md) stays the closer the way CommonMark reads it rather than inventing a link across lines. Measured against markdown-it-py (CommonMark 0.31.2) over 66k generated documents, comparing the rendered HTML rather than the destinations alone: 7286 documents in the parenthesis corpus and 313 in the container corpus now match the renderer, and the link and definition corpora are unchanged. One container document regresses, where closing the HTML block correctly exposes an unrelated gap of its own: a link reference definition still leaves a paragraph open, so the indented line below it reads as prose rather than as code. The list tracker still matches the backend on every step, the repo's own CHANGELOG resolves identically, and the pathological inputs measure the same. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read a block from the item its marker opens, and let a comment reach its paragraph Four things the three changelog scanners read differently from a renderer. A fence written straight after a list marker is the item's own first content, measured from the column that content starts, so "- ```md" opens one. All three scanners matched the whole line and saw nothing, so the code sample below it was prose: the resolver rewrote a destination the reader sees verbatim, and the preview offered the info string as a headline bullet. A shared itemContent / _item_content reads past a marker that really opens an item, capping the padding the way the list tracker caps it so an over-indented line is still indented code. An HTML block opener is read the same way, and its marker survives into the structural line so the item it opens is still tracked. An HTML block holds no lazy continuation line, so one opened on an item's continuation line ends where the item does, exactly as a fence there already did. The backend and the preview ended it only on a blank line, so it ran past the item and swallowed the next release heading, which made those notes unreachable and dropped every bullet below it from the collapsed popup. A raw block inside an item ends on a blank line too, which is where cmark puts it. A comment written mid-sentence is inline raw HTML belonging to the paragraph around it, so its "-->" may arrive on a later line of that same paragraph. Ending it at its own line left a backtick inside it pairing with a real one below, which hid a following link from the resolver, and left the preview quoting text the popup body does not show. A shared commentClosesBelow answers whether the closer arrives before the paragraph breaks; where it does not, the opener stays the ordinary text a renderer shows, so a note that merely mentions "<!--" still hides nothing. Only ASCII punctuation is escapable, so the backslash in "docs\alpha.md" is a character of the path. Dropping every backslash rewrote it to a path that does not exist, and a URL parser reads what survives as a separator, so a Windows or namespaced path pointed at the wrong file either way. The destination expression now escapes only punctuation, which also means a space still ends a destination: "[x](a b.md)" and "[x](a(b.md)" are not links, so their paths are left alone rather than half-rewritten. A destination that runs out of line still resolves, since its closer is on the line below. Fuzzed against markdown-it (CommonMark 0.31.2) over 20k-document corpora, with the whole rewritten document rendered and compared, not just its destinations. Release headings: 117 to 16 on containers, 88 to 10 on markers, 17 to 12, nothing new anywhere. Link destinations: 8823 to 104 on markers, 114 to 98 on comments, nothing new. Whole-document renders: 9271 to 220, 5116 to 245, 1265 to 671. The Python and TypeScript list trackers still agree over 26861 steps, and itemContent and hiddenStructure agree over another 6335. 321 KB of unmatched backticks still measures the same. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Let a definition follow a definition, and read a comment from the item it opens in Three CommonMark conformance fixes in the changelog scanners. A link reference definition is a block of its own that may not interrupt a paragraph, so it opens none either: definitions are allowed to run consecutively (spec 0.31.2 section 4.7). The link resolver counted one as paragraph text, so every definition after the first fell outside the set of lines a definition may start on and kept its relative destination, which then resolved against Studio's own origin. The backend already read the line this way. The guard asking whether a `-->` is reachable from an opener read any line whose first character was punctuation as the start of a new block. A `-->` written on a line of its own is how a multiline comment is ordinarily closed, and a wrapped line may open with emphasis, so neither counted as more of the paragraph carrying the comment. The comment never closed and the collapsed popup showed the author's internal note to the reader. It now tests for a block that may actually interrupt a paragraph. A comment is an HTML block too (section 4.6, type 2), so one written as a list item's first content opens inside that item exactly as a fence written there does. All three scanners looked for the opener at the margin of the line as written, so a marker in front of it hid the block: the resolver rewrote a destination inside raw HTML, which Streamdown then shows the reader as a literal URL, and the preview quoted the hidden note back at them as though the bullet were Markdown. The opener is now read from the item's content, the marker survives into the structural line so the item it opens is still tracked, and the block is scoped to that item the way a fence there is. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the release notes comments without losing the reasons they record --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>
This commit is contained in:
Michael Han
GitHub
parent
7b068090b2
commit
d74d03d350
25 changed files with 5874 additions and 35 deletions
1906
tests/studio/test_update_release_notes.py
Normal file
1906
tests/studio/test_update_release_notes.py
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue