6d8c18cd1a* Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error.
73 lines
2.7 KiB
Markdown
73 lines
2.7 KiB
Markdown
# data-designer-github-repo-seed
|
|
|
|
A Data Designer seed-reader plugin for **Unsloth Studio** that scrapes real
|
|
GitHub data (issues, pull requests, commits) from one or more repositories
|
|
and hands it to the recipe pipeline as a seed dataset.
|
|
|
|
Designed to ship with Unsloth as a default seed source so any user with a
|
|
GitHub token can build training datasets straight from live repos.
|
|
|
|
## What it does
|
|
|
|
Given a list of `owner/name` repos, a GitHub token, and a per-resource
|
|
`limit`, the plugin uses GitHub's GraphQL API to fetch issues, pull
|
|
requests, and/or commits, with labels, state, authors, and the first N
|
|
comments of each item, and materialises a single JSONL with uniform
|
|
columns so the rest of the recipe (LLM text / LLM structured / processors)
|
|
can treat it like any other seed table.
|
|
|
|
| Column | Description |
|
|
|---------------|------------------------------------------------|
|
|
| `item_type` | `issue` / `pull` / `commit` |
|
|
| `repo` | `owner/name` |
|
|
| `number` | Issue/PR number, or commit SHA |
|
|
| `title` | Title (or commit message headline) |
|
|
| `body` | Issue/PR body (or full commit message) |
|
|
| `state` | `OPEN` / `CLOSED` / `MERGED` (empty for commit)|
|
|
| `author` | GitHub login of the author |
|
|
| `created_at` | ISO8601 |
|
|
| `closed_at` | ISO8601 (empty for commits) |
|
|
| `url` | Permalink |
|
|
| `labels` | List of label names |
|
|
| `comments` | First N comments concatenated |
|
|
|
|
## Usage in a recipe
|
|
|
|
```json
|
|
{
|
|
"seed_config": {
|
|
"source": {
|
|
"seed_type": "github_repo",
|
|
"repos": ["unslothai/unsloth", "unslothai/unsloth-zoo"],
|
|
"token": "",
|
|
"item_types": ["issues", "pulls"],
|
|
"limit": 100,
|
|
"include_comments": true,
|
|
"max_comments_per_item": 30
|
|
},
|
|
"sampling_strategy": "shuffle",
|
|
"selection_strategy": null
|
|
}
|
|
}
|
|
```
|
|
|
|
Leave `token` empty to fall back to the server's `GH_TOKEN` / `GITHUB_TOKEN`
|
|
environment variable, useful when the recipe is published and shouldn't
|
|
carry a secret.
|
|
|
|
## Auth
|
|
|
|
A GitHub personal access token with `public_repo` scope is enough for public
|
|
repositories; `repo` scope is required for private ones. GraphQL requests
|
|
are rate-limit aware: the client inspects `x-ratelimit-*` headers and
|
|
sleeps until reset when the budget drops below a safety threshold.
|
|
|
|
## Install
|
|
|
|
Shipped as a default Unsloth plugin. For development:
|
|
|
|
```bash
|
|
pip install -e .
|
|
```
|
|
|
|
Registered automatically via the `data_designer.plugins` entry point.
|