diff --git a/Home.md b/Home.md index 1e94ab5..90d0548 100644 --- a/Home.md +++ b/Home.md @@ -997,39 +997,4 @@ services: build: context: . dockerfile: Dockerfile -``` - -### -We are hiring - -*** - -[Updated 10th November 2024] Want to work on cool Triton kernels, optimizations and maths algorithms to make LLMs and AI more accessible? Come join us! We currently have over [2.5 million](https://huggingface.co/unsloth) monthly Hugging Face model downloads and collaborate with Meta, Google, Hugging Face on open models. We fixed dozens of bugs in [Gemma](https://x.com/danielhanchen/status/1765446273661075609), [Llama & Phi](https://unsloth.ai/blog/phi3), [Mistral](https://unsloth.ai/blog/mistral-nemo), helped fix a [gradient accumulation bug](https://x.com/danielhanchen/status/1846235913443262891), showed how [gradient checkpointing](https://unsloth.ai/blog/long-context) can be improved to reduce VRAM and more! - -We value engineers who are proactive, independent and who ship features and ideas quickly - if stuff breaks, that's fine with us! Internships are 3 monthly renewable roles ($100K - $120K USD pa) SF focused or remote. Full time roles ($150K - $250K USD pa with equity) SF only. We're [Y Combinator S24](https://www.ycombinator.com/companies/unsloth-ai) alumni & backed by Github! - -**!! On our criteria for being considered for an internship / full time role !!** - -1. Debug, solve urgent issues / bugs and make 3 merged PRs for interns / 6 merged PRs for a full time engineer -2. **OR** Create a high quality accepted PR on 1 item below. 📚=Software Eng 🔢=Kernels 🛠️=Infra: - * 📚Unoptimized or optimized (extra points) Apple Silicon / Metal support LoRA - MLX, Core ML Tools etc support. - * 📚Utilities to export Unsloth finetunes to vLLM, SGLang & Ollama - LoRA adapters only. Provide a standalone serving interface to vLLM, SGLang - * 🔢Add float8 + QLoRA finetuning support via Torch AO into Unsloth. - * 🔢Bitsandbytes 4bit QLoRA dequant Triton kernel - must be faster than CUDA version. - * 📚Add TPU (maybe JAX?) & AMD support into Unsloth. TPU - Colab & Kaggle notebooks. AMD - Runpod equivalent. - * 🔢Use torch AO and add MXFP4 support in preparation for Blackwell. Show experiments on loss curves matching. - * 🔢Add fully optimized Deepseek finetuning support - investigate [Scatter MoE](https://github.com/shawntan/scattermoe) - confirm loss curves match. - * 🔢Add full finetuning / pretraining support in Unsloth - Triton kernels for all. - * 🔢FSDPv2 + QLoRA (maybe via Torch AO) + `torch.compile`. Investigate PyTorch native Pipeline, Sequence & Tensor parallelism. - * 🛠️Using spot instances to train models with a checkpoint recovery mechanism (like SkyPilot) - * 🛠️Modal but using spot instances - * 🔢Use binary tensor cores for fast Hamming distances. Show this works in approximate nearest neighbors - * 🔢Make `torch.compile` work on gradient checkpointing with compiled autograd and removing `torch._dynamo.disable` - * 🔢Port Flex Attention to Unsloth for all models (Llama, Mistral, Gemma etc). Must be torch compilable and faster than naive SDPA. - -**For for details, email me, or ask me questions on Discord! For more information about us, see our:** -1. CUDA / GPU Mode lecture Talk. [Youtube link](https://www.youtube.com/watch?v=hfb_AIhDYnA) -2. Low Level Technicals of LLMs. [Youtube link](https://www.youtube.com/watch?v=pRM_P6UfdIc) -3. Fixing bugs in Llama, Mistral, Gemma. [Youtube link](https://www.youtube.com/watch?v=TKmfBnW0mQA) -4. PyTorch Conference Mini Talk. [Youtube link](https://www.youtube.com/watch?v=PdtKkc5jB4g) -5. PyTorch Engineers Meeting Talk. [Youtube link](https://www.youtube.com/watch?v=MQwryfkydc0) -6. Hugging Face Collab Blog. [Blog link](https://huggingface.co/blog/unsloth-trl) \ No newline at end of file +``` \ No newline at end of file