Do you really need 64gb though? Do you want a PCIe in a low wattage, low air circulation mini PC? Are you running a docker swarm of VMs within the same system? Why?
What’s your use case for this?
You won’t get Local LLMs if the PSU is minuscule. Best get a big old workstation with good cooling. Fried motherboards overloading them with local LLMs. That stuff is mean on hardware.
16gb is still fine for most people.
Yeah an appliance distro would be a different consideration. For me Ubuntu is really flexible and a great docker host. I feel it’s safer to tinker with since you get those extra layers of containment and sandboxing.
If I was to build an appliance I might also start with Debian, but so far I have used FreeBSD. Works great on my NAS, where I use XigmaNAS. You can transfer a surprisingly lot of skills. Learning what was common to FreeBSD and Linux really helped me with cli.
I’m curious to try making stuff with Ubuntu Core. I think it has potential, but that’s a future deep dive and project. lol
I wonder how viable it would be to do the opposite of quantum. If we remove every at risk feature and strip it down to the basics: no scripts, no multi-user, no public sharing etc…
Project could be pretty easy to maintain indefinitely?
I remember reading earlier this year that the project was essentially “complete” and would only get security updates.
But like he says: “people keep making PRs for new features. Features that I do really think would benefit the project.”
I really struggle with folks’ sense of gratitude sometimes. Like there’s so many people making cool things for us… for free… When they could be doing the same work for money.
But yeah, sucks nobody could really keep the project up and secure. I’m not looking for a fork that will add anything to the project.
I’ll read some of the bug tickets to see what the current state is like. Figured this thing could just work indefinitely without much maintenance… its so simple?
Ubuntu’s “reputation” is nonsense. It’s a great distro for servers. Security updates are really fast and using snaps in a server environment is really great. I recommend running things like Docker or Tailscale off snap. They will revert to a previous image if an update fails. Extra stability for a server as you run automatic updates. If you run everything that is not in the base install off a snap, you have an extra layer of sandboxing against any supply chain attack for dependencies.
Ubuntu will have more recent packages, that doesnt really matter in a server environment. Debian is otherwise fine, I just dislike it’s installer and prefer Ubuntu’s. I found it easier to sync third party drivers on Ubuntu.
Ubuntu Pro gets you livepatches for kernal issues, which may be worthwhile. Also done without rebooting.
Ubuntu also gives you better apparmour default profiles on more things, whereas on debian, its up to you to set them up.
Debian will be exactly the same, its just more conservative and has less of those extras I mentioned. May be marginally lighter on resources for that. Either are fine.
Well, they do share one thing in common and that they are all massive users of or suppliers for datacenters. This makes sense since Linux is used in all kinds of servers and appliances. The possible bias of these companies is going to more towards the Cloud and SAAS rather than anything else. Likewise, generative ai is marketed nearly solely as a cloud technology, when it can be hosted locally. We are like a year or two off the average person’s web ai usage being possible to replace by ewaste gpus or entry level consumer hardware.
Nevertheless, I don’t see how the Linux foundation can really be steered towards that bias in a way that is concrete. Its just the kernal. All the userspace development is done by other parties.
We should celebrate these kinds of changes. Like its great that they’ve already achieved so much and not just that, but left behind a group of folks just as committed in the future of the project.
I always feel that posterity of your work beyond your own contribution is the real achievement and a sign of humility.
Yeah, that’s the neat part about servers, they don’t necessarily need to be snappy. Most of what it does is background work and it doesn’t have anything else to do. DDR3 and DDR4 get the job done just as well, and save you a buck.
My NAS has 16gb of ddr3 ecc, it has 10tb of ZFS pools and it runs an embedded OS. No problems with that. Not going to use it to host Gitlab lol, but it gets the job done for what it has to do.