Local models are still controlled by a small group of people, in the sense of deciding how and with what to train the thing. And given the cost of doing so, you can’t just easily fork it and make adjustments yourself.
How about going SETI@home with LLM training? Everyone contributing their own computing power to train better, more transparent local models. Using properly licensed, publically available datasets. Not sure how viable this approach would be, but that would make forking of open-source models possible, maybe?
Olmo exposes all of its training code and training data, including post-training etc.
I’ve trained a tiny GPT model on my computer before. It’s shitty, certainly, very shitty actually, but essentially anyone can train an LLM, if you got either enough time or calculation power.
Local models are still controlled by a small group of people, in the sense of deciding how and with what to train the thing. And given the cost of doing so, you can’t just easily fork it and make adjustments yourself.
How about going SETI@home with LLM training? Everyone contributing their own computing power to train better, more transparent local models. Using properly licensed, publically available datasets. Not sure how viable this approach would be, but that would make forking of open-source models possible, maybe?
Olmo exposes all of its training code and training data, including post-training etc.
Olmo: https://huggingface.co/allenai/Olmo-3-1125-32B
Also Apertus: https://www.apertus-ai.org/
I’ve trained a tiny GPT model on my computer before. It’s shitty, certainly, very shitty actually, but essentially anyone can train an LLM, if you got either enough time or calculation power.
Which is very different from compilers, especially for large languages with existing open source code. Those you just change willy nilly, as you want.