So I'm running UD-IQ2_XXS through the code in pr like they say to and the model is generating thinking tokens (which chat template doesn't emit but I can see them because I'm running --verbose w/llamacpp) and it's sitting there saying ?????????????????????????????????????????????????????????????????
Unsloth AIYou can now run Inkling-Small, a new 276B model by Thinking Machines. Inkling-Small is the strongest open model for its size and runs local on 128GB RAM. Apache-2.0 Licensed, it has image, audio + 1M context support. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Inkl...