d8cd2f8693
Adds a first-party Data Designer seed reader that scrapes GitHub issues, pull requests, and commits from one or more repositories via the GraphQL API, and a learning recipe (GitHub Support Bot) that turns those rows into synthetic support Q&A pairs for fine-tuning. Backend (new plugin studio/backend/plugins/data-designer-github-repo-seed): * GitHubRepoSeedSource config: repos, token (falls back to GH_TOKEN / GITHUB_TOKEN env var), item_types (issues / pulls / commits), per-resource limit (0 means all), max_comments_per_item. * Rate-limit-aware GraphQL client (GitHubClient + RepoScraper) shared across repos; flattens each item into a uniform row with columns item_type, repo, number, title, body, state, author, created_at, closed_at, url, labels, comments. * Registered via the data_designer.plugins entry point. Frontend: * New seed_github block variant so the seed node card shows "GitHub repositories" instead of the generic "Document file" placeholder, with its own icon and inline summary (repo count + item-type list). * Rewritten seed dialog github_repo form: repos textarea pre-filled with unslothai/unsloth + unslothai/unsloth-zoo, password input for the GH token, items-per-repo number with an "All" toggle, and the noisier options (item types, max comments, include comments) tucked under an Advanced collapsible. * Local model auto-load on Run: if a recipe uses an is_local provider and the inference server is not already serving that model, the executions hook calls /api/inference/load first. Removes the "open /chat to load a model" prerequisite that users kept tripping on. * Honor the recipe's run.rows value in the Run dialog (previously the store reset to 5 regardless of what the template shipped). Recipe (studio/frontend/src/features/data-recipes/learning-recipes/ github-support-bot.json): * Defaults to the Local Model provider + unsloth/gemma-4-E2B-it-GGUF. * Scrapes unslothai/unsloth and unslothai/unsloth-zoo, issues and pulls, up to 100 items per resource. * Two LLM blocks: normalized_question (llm-text) rewrites each thread into a clean support question, support_answer (llm-structured) produces JSON with answer / diagnosis_questions / cites / confidence. * Run defaults to 10 rows for a quick smoke test. Verified end-to-end on a running Studio: card renders, source-data dialog is pre-populated, All toggle disables the limit input, the recipe executes and produces rows against a loaded local GGUF. |
||
|---|---|---|
| .. | ||
| src/data_designer_github_repo_seed | ||
| pyproject.toml | ||
| README.md | ||
data-designer-github-repo-seed
A Data Designer seed-reader plugin for Unsloth Studio that scrapes real GitHub data (issues, pull requests, commits) from one or more repositories and hands it to the recipe pipeline as a seed dataset.
Designed to ship with Studio as a default seed source so any user with a GitHub token can build training datasets straight from live repos.
What it does
Given a list of owner/name repos, a GitHub token, and a per-resource
limit, the plugin uses GitHub's GraphQL API to fetch issues, pull
requests, and/or commits, with labels, state, authors, and the first N
comments of each item, and materialises a single JSONL with uniform
columns so the rest of the recipe (LLM text / LLM structured / processors)
can treat it like any other seed table.
| Column | Description |
|---|---|
item_type |
issue / pull / commit |
repo |
owner/name |
number |
Issue/PR number, or commit SHA |
title |
Title (or commit message headline) |
body |
Issue/PR body (or full commit message) |
state |
OPEN / CLOSED / MERGED (empty for commit) |
author |
GitHub login of the author |
created_at |
ISO8601 |
closed_at |
ISO8601 (empty for commits) |
url |
Permalink |
labels |
List of label names |
comments |
First N comments concatenated |
Usage in a recipe
{
"seed_config": {
"source": {
"seed_type": "github_repo",
"repos": ["unslothai/unsloth", "unslothai/unsloth-zoo"],
"token": "",
"item_types": ["issues", "pulls"],
"limit": 100,
"include_comments": true,
"max_comments_per_item": 30
},
"sampling_strategy": "shuffle",
"selection_strategy": null
}
}
Leave token empty to fall back to the server's GH_TOKEN / GITHUB_TOKEN
environment variable, useful when the recipe is published and shouldn't
carry a secret.
Auth
A GitHub personal access token with public_repo scope is enough for public
repositories; repo scope is required for private ones. GraphQL requests
are rate-limit aware: the client inspects x-ratelimit-* headers and
sleeps until reset when the budget drops below a safety threshold.
Install
Shipped as a default Studio plugin. For development:
pip install -e .
Registered automatically via the data_designer.plugins entry point.