Commit graph

14 commits

Author SHA1 Message Date
Roland Tannous
c87fdf079c feat: add embedding model training support
Add end-to-end embedding/sentence-transformer training pipeline using
FastSentenceTransformer, SentenceTransformerTrainer, and
MultipleNegativesRankingLoss with BatchSamplers.NO_DUPLICATES.

Backend:
- Add is_embedding_model() detection via HF tags + pipeline_tag
- Add /check-embedding/ API route and EmbeddingCheckResponse
- Extend derive_model_type() to return "embeddings"
- Add _run_embedding_training() in worker.py with progress callbacks,
  stop handling, LoRA (task_type=FEATURE_EXTRACTION), and model saving
- Add is_embedding field to TrainingStartRequest and ModelDetails
- Add YAML configs for 5 models: all-MiniLM-L6-v2, bge-m3,
  embeddinggemma-300m, gte-modernbert-base, Qwen3-Embedding-0.6B

Frontend:
- Wire isEmbeddingModel flag through store, API types, and mappers
- Force packing=false, train_on_completions=false, warmup_ratio=0.03
- Hide packing and train_on_completions checkboxes for embedding models
- Auto-set modelType to "embeddings" from backend model_type response
2026-03-10 18:10:09 +00:00
Roland Tannous
a0f03d3080 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Wasim Yousef Said
a5ad6ff12f Merge branch 'nightly' into feature/canvas-lab 2026-02-20 01:23:44 -08:00
Shine1i
a2cf89214e feat: add schemas for local model discovery and listing 2026-02-17 21:53:42 +01:00
Roland Tannous
8a239dc83e refactor: move checkpoint scanning to utils/models and /checkpoints endpoint to models router 2026-02-16 09:32:11 +00:00
Manan17
eae183504e Fixing the get checkpoint api 2026-02-16 04:47:28 +00:00
Shine1i
ba09bbfaba merge nightly 2026-02-15 14:52:07 +01:00
Shine1i
265f6350c8 feat: add Data Recipe core functionality with job manager, API routes, and validation services 2026-02-15 13:43:46 +01:00
sshah229
46cc71f310 added the pydantic models and routes for export 2026-02-11 18:34:12 -07:00
Roland Tannous
5ae20f6099 move inline pydantic models - fix existing models routes integration 2026-02-11 12:39:58 +00:00
sshah229
6e5cd50c34 fixed the errors- renamed jwt to authentication, used raw jwt, and removed search route 2026-02-07 03:14:30 -07:00
sshah229
f9f5dd4dcb refactored the code for username/password and added pydantic models and routes for the same 2026-02-06 03:15:30 -07:00
sshah229
fc673227b7 Refactored the training and model routes and added the jwt authentication 2026-02-06 03:15:30 -07:00
Roland Tannous
3463311bca fix: restore models directory files deleted during restructure 2026-02-02 19:36:30 +00:00