34d24bcf97
Late chunking (the Jina single-pass technique) is a non-default embedding path (chunking_strategy defaults to 'standard'). It adds a second full-document forward pass with windowed token pooling for a marginal long-document gain that the standard per-chunk encoder already covers at R@5 = 1.0 on the gold set. - remove late_chunk_encode / _windowed_late_chunk_encode / _pool_spans / _encode_tokens from embeddings.py (205 lines) - remove the chunking_strategy == 'late' branch and _run_late_chunking from ingestion.py (75 lines); 'late' now degrades to standard chunking 42 RAG tests pass. |
||
|---|---|---|
| .. | ||
| backend | ||
| bench | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| RAG_FAST_INDEXING.md | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||