About once a week I’ll come across a broad question that I think an LLM can answer without hallucination. I’ve recently switched to Mistral’s “vibe” app for that and find it good.
It’s the only moderately sized European AI company so worth your consideration at least.


I really would like to use them but they are still so far behind.
source: https://artificialanalysis.ai/
I hope they can catch up in the future, but for now if you want to buy european and want quality its better to use a chinese model on a european host.
To give them some chance of credit, Mistral claims that they have optimized their model to work much better with their own harness. I don’t know about that, but most benchmarks use a common harness for all AIs so any such optimizations don’t get applied.
Benchmarks don’t tell the whole story though, I’m sure they’re doing their best to be representative and measure something that is in reality pretty random, but I also find they’re pretty synthetic and not totally reliable. As a non-synthetic human I actually find a lot of Mistral’s answers pretty decent and models derived from it tend to score pretty highly on my personal “I find myself using them a lot because I like their output” benchmark. Take that anecdote for what it’s worth (very little) and YMMV, but don’t always completely base your choices on benchmarks either.
That is true.
Arena.ai is supposed to factor for that.
And you still have to scroll down a long way.
It really depends on the task, asking general questions hasn’t changed much since ye olde days of GPT-4.
But for coding or reasoning tasks the difference is very noticeable.
Ah I think it depends on your needs. As I say I only ask about one question a week and they tend to be broad ones (often historical) that I think I’m unlikely to get a hallucination from and I feel like more usage is likely to get them more funding. I’ve removed ChatGPT as a result.
Interesting graph though, no question. Maybe they’re the Lemmy of AI :)