* Prefer loading model from pretrained instead of config
* Fixup FP8 forward pass and inference
* [WIP] Fix lora forwards
* Infer block size from weight shapes
* reconstruct weights from fp8 quants for lora matmul
* Return weight transpose and fix dtype
* Refactor FP8 operations
* Fix naming :)
* Saner compile
* do not depend on transformers
* [WIP] fix training
* Update comment
* fixup training
* use dequant kernel from deepseek
* Differentiate between fp8 and fbgemmfp8
* fixup differentiation b/w fp8 and fbgemm_fp8
* make inputs contiguous if required
* Improve dequant
* More robust handling
* Fixup backward pass for fbgemm_fp8
* refactor and use bf16 for dequant
* Use torch fp8 block matmul
* Disable torch block matmul for now
* safer import and cosmetics
* more cosmectics
* add torchao operations
* Spaceeeeeee
eb65ff9b2f