YMMV with other hardware, I genuinely have no clue rofl!
Maybe start with 2B… try to work up from there?
Maybe Nvidia cards are just… better generally, due to generally better CUDA support for LLMs than ROCm?
I have no idea!
The Deck uses an AMD APU, RDNA 2, shared LPDDR5 RAM… so… yeah i have no idea how to translate that to a more standard PC set up, in terms of performance expectations. Tooooo many variables.
YMMV with other hardware, I genuinely have no clue rofl!
Maybe start with 2B… try to work up from there?
Maybe Nvidia cards are just… better generally, due to generally better CUDA support for LLMs than ROCm?
I have no idea!
The Deck uses an AMD APU, RDNA 2, shared LPDDR5 RAM… so… yeah i have no idea how to translate that to a more standard PC set up, in terms of performance expectations. Tooooo many variables.