Good direction, but very futile considering that guardrails on LLMs are about as real as LLMs are intelligent
The nice thing about laws is that they don’t give a shit. If they say you can’t do something, it’s your problem to make sure you don’t do it.
It might not work, but it would serve to increase awareness that you absolutely cannot trust these machines to give you correct answers, ever.
Limiting the medical knowledge people can get is a good direction? really?
Limiting medical knowledge from reliable information resources would be terrible. I’d say that limiting the spread of plausible-sounding medical misinformation generated by a fancy autosuggest tool is good, though.
AI chatbots don’t actually “know” anything. They just generate the most statistically probable appearance of a response to each prompt based on their training data. They don’t tell you when something wasn’t in their training data or when there’s limited research on a subject, just give you the most statistically probable response like always, which leads to a “hallucination” that sounds true but isn’t. A tool that’s trained on MCAT testbanks and “aces” the MCAT won’t necessarily give reliable medical advice to real patient questions. Even if you gave an AI chatbot all the accurate training data in the world, it would still hallucinate information occasionally, and it wouldn’t give you any indication of when it did so. If your doctor screws you over, they can lose their license, but if ChatGPT screws you over, there’s currently very little to stop OpenAI from screwing over more people. OpenAI can just stand behind the disclaimer that says ChatGPT can make mistakes, never mind that users are treating it like a licensed medical professional. There are AI tools that are useful in medical research, but they’re a VERY different kind of AI from the AI chatbots this bill is concerned with.
If you want medical knowledge, there are tons of medical journals, news sources, blogs, and podcasts from doctors/scientists who don’t hallucinate BS when they don’t know something.
Instead of banning it then teach people how to use it and not to trust it 100% (who the hell is doing this to begin with?)
If you want medical knowledge, there are tons of medical journals, news sources, blogs, and podcasts from doctors/scientists who don’t hallucinate BS when they don’t know something.
I can ask an LLM to help me sourcing these journals and informations
Expecting people who have already taken the easy route of asking an AI to summarize medical information to do additional due diligence by reading the sources that the AI spits out is naive as fuck.
There’s a reason it became a meme for someone to Google their symptoms, click on WebMD to read about it, and then come to the conclusion that they have cancer, all without ever having seen a medical professional in the meanwhile. Do you really think these sorts of people are going to be using AI responsibly for research purposes only?
There’s a reason it became a meme for someone to Google their symptoms, click on WebMD to read about it, and then come to the conclusion that they have cancer, all without ever having seen a medical professional in the meanwhile. Do you really think these sorts of people are going to be using AI responsibly for research purposes only?
If they come to the wrong conclusion at worst they go to the hospital and someone there tells they are wrong and not to worry. We haven’t ban googling your symptoms.
LLMs are NOT a reputable source of medical information. They regurgitate information from random sources, and can lead to disastrous effects when pulling from bad sources or incorrectly summarizing from a good source. At best it should exclusively provide links to sources (good or bad) and let the end user make the determination if that source can be trusted.
This isn’t limiting information at all. People can still access all the reputable sources that provide the information they are looking for. This is simply cutting out the unreliable middleman.
At best it should exclusively provide links to sources (good or bad) and let the end user make the determination if that source can be trusted.
You can ask AI to do exactly this and according to your own logic turning it out into a good at best medical tool.
Limiting the medical misinformation.
FTFY.
AI is a tool that can provide good and bad results. To people who can’t get access to medical care easily (half of the people on the planet) it can be an useful tool to get basic medical knowledge
The issue is that AI is extremely good at sounding right, but only okay-ish at being right.
When you get good sounding information that is wrong, it can be extremely problematic and often much worse than no info at all.
Imagine LLMs aiding people with health improvements, only to lead them into health decline (actually already happening, a lot) not to mention the LLMs that further convince people of their conspiracies.
There is a reason why health professionals go through a lot of training and are under a lot of scrutiny, you wouldn’t want just any rando who thinks he know better to be your doctor
When you get good sounding information that is wrong, it can be extremely problematic and often much worse than no info at all.
A partially good answer is better than no information at all
Imagine LLMs aiding people with health improvements, only to lead them into health decline (actually already happening, a lot)
Already happening where? How can AI lead you to health decline? It cannot prescribe you medicines or do surgery on you
not to mention the LLMs that further convince people of their conspiracies.
It can also help people understanding if conspiracies are true or false, for example had you ask an LLM about this post it would have highlight to you that there’s a good chance that it is propaganda.
Counterpoint:
I diagnosed myself with something pretty significant using the internet, and then I went to get evaluated, and my suspicions were confirmed.
This would prevent a lot of good, in addition to preventing some bad.
I diagnosed myself
Congrats, glad your life is hopefully changing for the better because of this.
I hate taking the wind out of your sails but this is in the area of anecdotal evidence. You are a single person saying this for your specific case. I am sure we will get another few people confirming the same here.
But what about how many people tried this and got nothing useful out? Or even a misdiagnose? Anyone enthusiastic about this possibility of AI diagnosing people wouldn’t really mention this. People like to mention the successes but it needs to be weighed against the failures too, and I don’t think there is a good record of that.
If there is, its probably a medical study and I’d venture a guess that AI did not perform as well as a trained doctor.
I hate taking the wind out of your sails but this is in the area of anecdotal evidence.
Oh, no, I’m sorry, but it’s not. And you’re not.
The thing I self-diagnosed with is actually something that doctors have noted that it’s very common for people to diagnose themselves with, and a majority of people with the particular thing I’ve been diagnosed with have actually never received a formal diagnosis. And we’re talking something that’s been around as far as we know throughout recorded history, and doctors have just been terrible at diagnosing it.
Not ONLY does access to AI enable people who have this particular issue to self-diagnose, but it enables the vast numbers of undiagnosed people with it to identify their actual issues and identify the need to get help.
In fact, several other people in my family independently came to the same conclusion and then got diagnosed with it within the last few years. And I’m talking a family which doesn’t regularly interact as a family, so we were all acting basically as people who don’t interact with each other.
But hey, keep on parroting that anecdotal line. Usually it’s probably true. But for me and my particular issue, your argument actually falls completely flat.
I can’t even count the literally millions of people who are getting correct diagnoses because they’ve been denied medical care in the United States for decades and didn’t have the ability to get diagnoses and then talked to AIs and were able to get diagnoses.
I diagnosed myself
using the internet
I’m going to assume you researched medical documents, studies, and journals to link things together, and not ask a chat bot “My foot hurts, what is wrong?”
You’re smart enough to self diagnose yourself with readily available information to the public, but not smart enough to not know the difference between that and a chat bot. That’s remarkable.
I diagnosed myself
Putting aside the ultra ignorance and/or untreated narcissism just dripping off of a statement like that… If you were curious of the presence of any absolutely blatent bias in your speech, just look at your use of “a lot” and “some” in your final sentence . Wow…
So you played digital Russian roulette and won. Congratulations.
So now you’ll have to pay $200 every time you have an important question you need answered? My PCP just tells me to use over the counter products anyways. Also is this NY state or city? Why is there a picture of Mamdani?
What about engineering? Food safety? Education? So many other fields could be added
Anything that requires a license is a good start.
I really think this is a bad idea these should really just be flagged, there’s many situations where asking these questions in necessary and there’s few if any other ways to get an answer
Same, especially for medical advices.
I am currently going through a very complex diagnosis of combination of SCDS and migraines, and neck joint / facial nerve issues through multiple doctors - ENT, neurologists and so on.
Having AI backing me up with providing a structural understanding from multiple disconnected sources is a huge blessing.
Especially when you die from migraine, and trying to understand are symptoms worth to go to emergency ward or not.
AI can’t back you up better than you can with your own research. Why are you even considering AI credible? What gives it credibility?
It’s not credible, but you can ask it for a credible source and read that, which is still incredibly helpful.
I guess the question is, why ask it for the source? What gives it credibility to be the best at doing such a thing?
It’s not, but a source can be validated by a human much easier than finding the source in the first place.
Use it like a fancy search engine.
What makes it fancy? Serious question.
A lot, it is much better at giving better results and while you can’t trust it it’s a great place to start for problems if you’re willing to verify, it has helped me find VERY obscure information that I have searched for literal hours on google before essentially instantly because it can search all the variations of what you mean and instantly check for relevant text. For obscure info it’s vastly better than google.
Maybe I’m just ignorant about this, but it feels a bit “gatekeepy” to outlaw answering questions about the law itself. Like… so I can’t use an LLM to help do taxes? Or if I’m served a confusing legal document, I have to pay an expensive attorney to understand the legalese? I can’t even just get a rough idea from an LLM first?
EDIT: Also I’ve literally had a professional accountant send me a ChatGPT conversation they had because they didn’t know tax law well enough. I feel like if anything that should be outlawed. Professionals should actually have to know shit and know how to research primary sources.
law in itself is very gatekeepy - you have to learn the “correct” way to interpret the words and then get certified for it.
More like you as an average person have no idea what it’s hallucinating and what is the truth and may end up in jail or the hospital like the cases that are starting to get reported on who did just that.
I disagree, knowledge should be easily accessed by everyone, they should only append it with a warning.
Some people can’t afford seeing many lawyers and doctors, so it is cheaper to cross refrence with ai and do some online search.
Except AI is an over engineered autocomplete operating completely off of statistical models. Knowledge is not statistical or random. I should get the same outputs to the same inputs.
Knowledge is not statistical or random. I should get the same outputs to the same inputs.
Where does the thought come from? If you write five articles about the topic or any topic, will they be the same?
They won’t look the same but they’ll convey the same information consistently.
Even about topics that you barely are familiar with?
If the prompt includes the older articles, or if the NN is updated once an article is written, wouldn’t AI be consistent, too?
Then I research from known references starting with Wikipedia and using the sources at the bottom of the page to expand my research out further.
The entirety of how LLMs work is token prediction based on the entire collective of human creation, that is not research, that is not understanding. that is an over engineered autocomplete. Guessing what the next token is in a string of characters is not the same knowledge of the topic. How much does the Epic of Gilgamesh and the complete works of Shakespeare and the writings of Adolf Hitler and the unhinged Tumblr posts of every Susie, Harry, and Jillian and all of the statistical models born from vectorizing all of that into a billion dimensional vectorspace have any impact on the final output, how does any of that have anything to do with any medical or legal inquiries?
Ive already explained multiple times how a glorified autocomplete predicting words is not knowledge. If you’re going to keep sealioning I’m done with this conversation.
Ive already explained multiple times how a glorified autocomplete predicting words is not knowledge.
Not on your last two comment pages. Do you have a link?
Guessing what the next token is in a string of characters is not the same knowledge of the topic
How do you write? Do you know your entire sentences and paragraphs before you write them down?
So you’re just gonna keep sealioning, exactly as I predicted. Got it, glad to establish this conversation is going to go nowhere. You can repeat my last handful of comments in this thread. I dont have time to waste entertaining your asinine questioning. You’ve been nothing but fake polite in order to waste all of our time.
New York State not New York City.
I don’t know why Mamdani is pictured. I think it is because the proposal was introduced in the State Senate by Kisten Gonzalez who is a DSA member.
I feel like finding and citing appropriate law cases would be a great way for AI to be used, especially because the bullshit legal-ese that surrounds everything in court cases is so spectacularly opaque and trivial that you have to pay out the nose just for someone to parse through all the bullshit… Like, lawyers shouldn’t even be a thing you need unless it’s for some really complicated case. The courtroom should be a place where an average citizen could speak for themselves.
Also, as someone who studied medicine… Do you know how often doctors will take a patients history, step outside the exam room, and then google shit? I don’t want to imply that they don’t know what they’re doing, like, you need to study medicine to know what to google, BUT… they still be googling shit. I think AI could be trained to ask the same patient history questions that a nurse asks. Then using the answers to those questions, then they can provide some feedback, with a big ol disclaimer that it’s NOT to be taken seriously, and you should consult with a physician.
When people say “AI” these days, they mean the LLM-derived generative AI such as ChatGPT, Google Gemini, Anthropic Claude, and Grok.
These AI systems are not capable of finding authoritative search results. They are good at generating fluent, plausible, context-appropriate text. In legal and medical situations, that’s the opposite of what you’d want.
Do you know how often doctors will take a patients history, step outside the exam room, and then google shit?
Think about how often the doctors land on some pseudo-science result. Now imagine that instead of a doctor it’s a patient and instead of asking “could these be related” the patient is told “these are related”.
a big ol disclaimer that it’s NOT to be taken seriously
This has worked very well for Fox News, which is not a news agency and is for entertainment purpose only. Also for dietary supplements, which are not medicine.
Specifically tuned LLMs that are trained on medical databases and less sycophant (E.g. OpenEvidence) are one of the killer applications of AI. Tons of frivolous AI nonsense but this could actually save lives. Access to broad knowledge and connections that a doctor can make use of.
We’re demanding and regulating the complete wrong shit about AI.
In that we aren’t regulating any of it?
What would be your demands for regulation?
Companies should be liable for what their LLM says.
That seems technically not possible.
How would that be technically impossible? Or difficult? Companies are currently liable for the actions of the human agents, and I don’t think the implementation needs to be different.
Maybe you mean socially impossible? Like, the major providers have a lot of money behind them and so why would they allow themselves to be liable for anything?
Absolutely not.
Ai has saved me from suicide multiple times. Without it I’d 100% be dead.
However, it’s a recursion engine and I have more objective self awareness than most so I understand I am a minority.
I wonder how many people have committed suicide because of chatbots.
Who else is gonna answer those questions nowadays, the medical “professionals” who completely lost to common sense in 2020 from start to finish, and who still believe you can be a million genders in this fake fantasy world of pretending unchangeable biological constants are suddenly not how real life works anymore? LOL, may as well ban YouTube next too so nobody can spread ‘evil false information’ which turns out to be 100% true, the TV will only state the truth trusttt
Let’s build all the massive data centers but then we’ll just make using them illegal
