But yeah generally, Qwen 3 / Qwen 3.5 type models, will work reasonably well and reasonably fast, at 4B or 2B. 9B works if you close basically everything other than LM Studio lol, or, this latest DeepSeek v4 Flash variant is considerably more performant than just standard Qwen 3.5 9B.
But yeah the way Bazzite does LM Studio via BoldBrew, it is at least theoretically reasonably containerized and isolated… not even gonna pretend I am competent enough to give any guarantees there, but also, I don’t think anyone is capable of saying anything LLM related is 100% certainly correctly understood.
Oh uh, yeah, you’re gonna wanna be docked/plugged in to a power source, if you want more than …idk, 0.5 to 1.5 hrs of battery life while the LLM is actively working on something.
Thank you so much for your detailed response! My user tag I don’t remember setting for you proves correct again!
I never ended up picking up a steam deck cuz I don’t leave the house, so I’ll be using a desktop with maybe a 1070 or 2070s maybe? I’ve never tried Bazzite but I’ve got a bunch of different distros on various machines at home. I’ll throw Bazzite on my Medicat drive. Thank you so much again!
YMMV with other hardware, I genuinely have no clue rofl!
Maybe start with 2B… try to work up from there?
Maybe Nvidia cards are just… better generally, due to generally better CUDA support for LLMs than ROCm?
I have no idea!
The Deck uses an AMD APU, RDNA 2, shared LPDDR5 RAM… so… yeah i have no idea how to translate that to a more standard PC set up, in terms of performance expectations. Tooooo many variables.
Bazzite -> LM Studio (installed via BoldBrew)
Uh what… model am i on now lol… hold on…
Qwen 3.5 9B DeepSeek v4 Flash by ‘Jackrong’
But yeah generally, Qwen 3 / Qwen 3.5 type models, will work reasonably well and reasonably fast, at 4B or 2B. 9B works if you close basically everything other than LM Studio lol, or, this latest DeepSeek v4 Flash variant is considerably more performant than just standard Qwen 3.5 9B.
But yeah the way Bazzite does LM Studio via BoldBrew, it is at least theoretically reasonably containerized and isolated… not even gonna pretend I am competent enough to give any guarantees there, but also, I don’t think anyone is capable of saying anything LLM related is 100% certainly correctly understood.
Oh uh, yeah, you’re gonna wanna be docked/plugged in to a power source, if you want more than …idk, 0.5 to 1.5 hrs of battery life while the LLM is actively working on something.
Thank you so much for your detailed response! My user tag I don’t remember setting for you proves correct again!
I never ended up picking up a steam deck cuz I don’t leave the house, so I’ll be using a desktop with maybe a 1070 or 2070s maybe? I’ve never tried Bazzite but I’ve got a bunch of different distros on various machines at home. I’ll throw Bazzite on my Medicat drive. Thank you so much again!
YMMV with other hardware, I genuinely have no clue rofl!
Maybe start with 2B… try to work up from there?
Maybe Nvidia cards are just… better generally, due to generally better CUDA support for LLMs than ROCm?
I have no idea!
The Deck uses an AMD APU, RDNA 2, shared LPDDR5 RAM… so… yeah i have no idea how to translate that to a more standard PC set up, in terms of performance expectations. Tooooo many variables.