This is an abstract curiosity. Let’s say I want to use an old laptop to run a LLM AI. I assume I would still need pytorch, transformers, etc. What is the absolute minimum system configuration required to avoid overhead such as schedulers, kernel threads, virtual memory, etc. Are there options to expose the bare metal and use a networked machine to manage overhead? Maybe a way to connect the extra machine as if it is an extra CPU socket or NUMA module? Basically, I want to turn an entire system into a dedicated AI compute module.
It’s not. Like the commenter above said: It’s a fraction of the task at hand. Especially when you design the rest of the system to run only if necessary. Context Switches are what? like 50 CPU Cycles? Store Registers, Store TCB, Load other TCB and load other register states jump back to PC. Maybe some other OS Shenanigans, but that’s basically it.
Now Imagine complex calculations on a 25-Dimensional Matrix.