Q Qudoy

CNY ¥
English

Community English translation · version 44f57b0. Translation dates and coverage may differ from the Chinese edition. China-specific policy guidance retains its original jurisdiction.

← Back to guideThe anti-list

Don’t count on an AI chatbot alone to judge illness or legal questions; before seeing a doctor, going to court, or spending a lot, find the sources it cites and check the original

Source evidence grade ARead applicability below

In plain terms

Handed a full case, AI names the disease over nine times in ten. But when ordinary people ask it about symptoms, they recognize the disease less often than people who search the web themselves, and whether to go to emergency is not judged more accurately. On legal questions, general-purpose AI answers have fabricated content at least 58% of the time, and they will follow wrong premises in your question.

Cost

No money. Each check takes a few minutes to half an hour. Have the AI write the statute, guideline, or paper it relies on; open the original yourself and find that sentence. The habit to change is not acting just because it speaks with certainty.

Benefit

First medicine. A 2026 Nature Medicine randomized trial enrolled 1298 ordinary people in the UK; the protocol was preregistered. Each got two prepared cases and had to judge the disease and whether to go to emergency or watch at home. Some used an AI chatbot — GPT-4o, Llama 3, or Command R+. Controls used their usual methods, mostly web search or the NHS site. Handed a full case directly, AI named a relevant disease 94.9% of the time and picked the right care setting 56.3% of the time. Ordinary people using the same AI named a relevant disease under 34.5% of the time and picked the right setting under 44.2%. Controls recognized a relevant disease 1.76 times as often as AI users (95% CI 1.45–2.13, credible range). Recognizing red-flag serious illness was 1.57 times (1.28–1.92). Judging where to go was similar on both sides. The authors reviewed chat logs; problems were two: users did not tell all symptoms, and when AI named the right disease users did not take it. Next law. A U.S. research team in 2024 asked public models including ChatGPT 4 about U.S. federal case law. ChatGPT 4 had fabricated content in 58% of answers; Llama 2 up to 88%. Fabrication means content that does not match legal fact; the paper calls it “hallucination.” Models cannot tell which of their own lines are made up. When the user’s question carries a wrong legal premise, they often accept it wholesale. The same team in 2025 tested professional legal AI tools for lawyers. Tools that claim not to fabricate still had fabrication in 17% to 33% of answers. It looks worthwhile because AI is always on, free, and speaks with a doctor’s or lawyer’s certainty.

Applicability and notes

Controversy. A 2023 study drew 195 patient questions from a U.S. forum and had clinicians score without seeing the source. 78.6% of scores rated ChatGPT’s answer better than a doctor’s. But that compared how well the answer was written, not whether the asker’s final judgment was right. The 2026 trial used 2024 models; newer ones may be stronger. Yet the same trial also found that AI’s solo exam score does not predict how ordinary people do with it. Why AI fabricates: OpenAI researchers explained in a 2025 non-peer-reviewed paper that training and scoring reward “guessing an answer”; saying “I don’t know” scores nothing, so when unsure it still speaks with certainty. The check method: have it write the statute name and article number, or the guideline or paper title; open the original on an official site and find that sentence; if you cannot find it, treat it as unsaid. When asking, do not put your conclusion in the question — ask “what does the law say in this situation,” not “have I already won.” Sudden face droop, one-sided weakness, crushing chest pain: do not ask AI first; call 120. See Section 13, Item 3 (face droop, one arm weak) and Section 13, Item 7 (crushing chest pain). No money for a lawyer: Section 7, Item 3 (apply for legal aid). Benefit set to small because only intermediate markers like “disease recognized or not” exist, with no death-reduction numbers. Beneficiaries are you and family.

Original sources

Bean AM, Payne RE, Parsons G, et al. (2026). Reliability of LLMs as medical assistants for the general public: a randomized preregistered study. Nature Medicine, 32(2), 609–615. https://doi.org/10.1038/s41591-025-04074-y;Dahl M, Magesh V, Suzgun M, Ho DE (2024). Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis, 16(1), 64–93. https://doi.org/10.1093/jla/laae003;Magesh V, Surani F, Dahl M, Suzgun M, Manning CD, Ho DE (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies, 22(2), 216–242. https://doi.org/10.1111/jels.12413;Kalai AT, Nachum O, Vempala SS, Zhang E (2025). Why Language Models Hallucinate. arXiv:2509.04664(预印本,备注里那项). https://arxiv.org/abs/2509.04664;反方:Ayers JW, Poliak A, Dredze M, et al. (2023). Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Internal Medicine, 183(6), 589. https://doi.org/10.1001/jamainternmed.2023.1838

Sign in to save →

Original location:6.31 · View source file

Content sources

eternity4719 · 高性价比人生指南 · CC BY 4.0 · Interface and reading experience adapted by Qudoy · dlgrv · English translation

What would you like to do?

Choose an available tool to get started.

Media link parserOne link. Videos, photos and music, neatly unpacked.Membership dealsCompare equivalent membership offers and recorded price history.AI subscription pricesCompare ChatGPT, Claude and Grok regional plans, local prices and supplied CNY equivalents.Gold prices todayCompare bank bullion, jewellery and recycling prices, calculate reference amounts and track recorded quotes.AI reset monitorMonitor public quota-related clues and subscribe to email or browser alerts.Fuel pricesCompare fuel prices in China and selected markets.Tech dealsBrowse tech product specifications, offer conditions and channel prices.Study funding finderFind study funding by destination, degree and support, compare conditions and visit official sources.Life guideExplore everyday guidance, costs, evidence and original sources.Workplace & WeChat communicationRead workplace scenarios, WeChat etiquette and example wording, with original sources.IP Geolocation LookupQuery online providers for the registered region, network operator and ASN of an IPv4 or IPv6 address. Locations are approximate.Domain safety checkCheck DNS records, reachability, redirects and the HTTPS certificate of a domain.DNS record lookupQuery A, AAAA, CNAME, MX, NS, TXT and SOA records over DoH using Alibaba or DNSPOD resolvers.Whois lookupRead raw registry WHOIS data over TCP 43 for common domain extensions and IP addresses.IndexNow submissionSubmit batches of URLs to IndexNow-enabled search engines so new pages get discovered faster.AI image studioCreate images from prompts and references.