Never used Claude, but GPT-5.6 Sol annoyed me so much I actively avoid it and just use Terra on max. Sol is trained to favor convoluted San Francisco jargon that projects any trivial idea into some sort of an ingenious business maneuver.
You don’t need to worry about the split in strix halo, it runs a unified memory architecture so Linux will dynamically allocate VRAM on demand up to maximum possible (which would be around 120GB I think)
I’m on an nvidia system so can’t answer the rest of your questions other than to say that in my experience ollama gave me a much worse tokens per second speed compared to running the same model using llama.cpp, so if you get something slower than you expect it might be ollama caused.
Good luck with your £250 system 😉
I’ve been testing this model since yesterday through the Qwen API in opencode and deepseek harness. I actually prefer it over GLM-5.3-Flash! it’s super good at agentic coding and comfortably fast, now my goal is to be able to one day run it locally 😳 I think a Strix Halo with 128GB and a fast nvme should be able to run it well at near loseless quant
llama.cpp support for it got merged a few hours ago 🏆
This is better than GLM-5.2 which just 2 months ago was my go to high tier model!
Also in OpenRouter this model is listed to have a maximum context of 1.31M tokens! this is the first high-end model to my knowledge that supports more than 1M tokens context 🤯
You might want to try running Ling-3.0-Flash (124B-A5.1B) it might be perfect for your M4 and should perform close to the level of DeepSeek V4 Flash preview or MiniMax M2.7
Yeah I tried Kat Coder when it came out, I don’t remember the exact issue I ran into but I identified a definite regression in my specific use cases compared to using vanilla. Ornith seems to be equivalent or better so far 👍
My previous experience with finetunes is bad…
Today I tried Ornith-1.5-35B-A3B, and I’m quite positively surprised by it… I compared it to both Qwen3.7-Flash and MiMo-V2.5 (via OpenRouter) in a few coding tasks and Ornith seems to be better than both! Thinks much more but the result definitely speaks for itself.
Hoping for Qwen3.8-35B soon 🤞
I’ll add my vote to keeping the 32+8 system as that has the most ram.
I also have an 8GB RTX 3070 Ti and 32GB RAM that I run Qwen3.6-35B-A3B perfectly on (30+ t/s) and looking forward to a possible Qwen3.8 35B update.
Recently bought a used 8GB RTX 3070 for a bargain, planning to combine it with the 3070 Ti and hopefully run a lower quant Qwen3.8-27B which is crazy good for its size.
Cheers for the new local GOAT!
Also on ModelScope in case you-know-who bribes huggingface takedown because model is too dangerous.
free to use if you have the hardware. For this model because of its size, the main problem is the hardware availability/cost. But in general there are 3 ways to run an open weights model:
Update: GLM-5.3 dropped today, open weights release in ~2 weeks!
Everything runs well if one is patient enough 😂
But yeah, with a smaller context and carefully offloading some of the layers to the CPU, it’ll run just not very fast. And definitely using Q4 quants.
No you don’t need a data center, this can run on any current gaming PC with a GPU that has 16GB+ VRAM.
Although to get good token generation speed and full context you need at least 24GB VRAM.
Appears they removed the branch and changesets
I only have 8gb vram and can run 35b-a3b perfectly (30+ tps), have you tried 35b?
Wow, got about 9 repos of mine there and a few from my startup’s, only public ones are scraped so I say fair enough. Happy to have switched to running my own git infra a year ago
I hope their decision to opensource Qwen3.8-Max means this version is going to be another open-weights frontier contender 👍
I’ve had this silly fantasy of being able to clone myself into N clones each works on a different project because I often have so much more on my plate than I can ever really handle and context-switching kills my productivity instantly. Agentic coding seems to have gotten close to making my silly fantasy a reality (but only recently, current LLMs are awesome).
I spent the last 2 weeks reviving and completing old incomplete hobby project that I’ve abandoned over the years knowing I’ll never really have time to go back and get them to a working state. Most turned out not to be as usable/fun as I thought when I had the idea, but a few were so good I might actually use them in my current and future work.
I also build bespoke tools only useful for specific projects, but instead of spending weeks or months getting side-tracked, it takes mere hours and the tool is immediately usable. An example of this, building a level editor for a specific game I’m working on. It might not be great and the code might be sloppy, but doesn’t matter if it lets me compose content faster and better than anything else for this specific game.
Lately I’ve had an idea to really pair up with an agentic coder where the agent codes the supporting libraries and frameworks I want on demand while I build the actual thing.
The possibilities are exciting and endless.