I’m having difficulty with getting off the ground with these. Primarily I don’t trust the companies or individuals involved. I’m hoping for open source, local, with a GUI for desktop use and an API for automation.
What model do you use? And in what kind of framework?
Huggingface lists thousands of open source models. Each one has a page telling you what base model it’s based on, what other models are merged into it, what data its fine-tuned on, etc.
You can search by number of parameters, you can find quantized versions, you can find datasets to fine-tune your own model on.
I don’t know about GUI, but I’m sure there are some out there. Definitely options for API too
R1 last i checked seems to be decent enough for a local model. customizable. but that was a while ago. its release temporarily crashed Nvidia stock because they showed how smart software design trumps mass spending on cutting edge hardware.
at the end of the day its all of our data. we should own the means, especially if we built it by simply existing on the internet. without consent.
if we wish to do this, its crucial that we do everything in our power to dismantle the “profit” structure and investment hype. sooner or later someone will leak the data, and we will have access to locally run versions we can train ourselves. as long as we dont allow them to monopolize hardware, we can have the brain, and the body of it run local.
thats the only time it will be remotely ethical to use, unless its the persuit of attaining these goals.
I’m having difficulty with getting off the ground with these. Primarily I don’t trust the companies or individuals involved. I’m hoping for open source, local, with a GUI for desktop use and an API for automation.
What model do you use? And in what kind of framework?
Huggingface lists thousands of open source models. Each one has a page telling you what base model it’s based on, what other models are merged into it, what data its fine-tuned on, etc.
You can search by number of parameters, you can find quantized versions, you can find datasets to fine-tune your own model on.
I don’t know about GUI, but I’m sure there are some out there. Definitely options for API too
I use the Apertus model on the LM Studio software. It’s all open source:
https://github.com/swiss-ai/apertus-tech-report/blob/main/Apertus_Tech_Report.pdf
R1 last i checked seems to be decent enough for a local model. customizable. but that was a while ago. its release temporarily crashed Nvidia stock because they showed how smart software design trumps mass spending on cutting edge hardware.
at the end of the day its all of our data. we should own the means, especially if we built it by simply existing on the internet. without consent.
if we wish to do this, its crucial that we do everything in our power to dismantle the “profit” structure and investment hype. sooner or later someone will leak the data, and we will have access to locally run versions we can train ourselves. as long as we dont allow them to monopolize hardware, we can have the brain, and the body of it run local.
thats the only time it will be remotely ethical to use, unless its the persuit of attaining these goals.
Self-hosting is already an option, go have a look around huggingface
No need to leak the data, it’s open source. https://arxiv.org/abs/2211.15533