There’s a lot of hype surrounding ChatGPT (chat.openai .com), currently the most widely used artificial intelligence text generator. Of course, I’m a progenitor of such hype. But when the hype comes from a journal of the American Medical Association, it lends a certain credibility. And if anything, the results of a recent study published in JAMA Internal Medicine actually exceed the hype.
The authors of the study wondered how ChatGPT’s answers to medical questions from individuals would compare to answers given by medical professionals. They selected 195 medical questions that had been answered in Reddit’s AskDocs forum by verified health-care professionals. They then asked ChatGPT to also answer each of those 195 questions.
Then they had a panel of three licensed healthcare professionals read the two answers to each question without knowing who answered them. These physicians were simply asked which of the two answers were best in terms of quality and empathy.
ChatGPT excelled on both counts. In terms of quality, the raters found that 79 percent of ChatGPT’s responses were good or very good compared to 22 percent of the responses from doctors. In terms of empathy, the raters found that 45 percent of ChatGPT’s responses were empathic or very empathic compared to 5 percent of the doctors’ responses.
This doesn’t mean you should use ChatGPT instead of a doctor. ChatGPT itself always cautions that it’s important for a person to consult a health-care professional or specialist.
What it does mean is that ChatGPT can do astonishing things.
A friend of mine recently mentioned how grateful he was that I had written about ChatGPT—and went on to tell me some extraordinary things that it’s done for him. Years ago, in his master’s thesis in geometry at the University of California Davis, he discussed a general theorem that he determined would be difficult to prove and would be well beyond the scope of his proof of a specific instance of invariant convexity. So recently, out of curiosity, he asked ChatGPT to prove this more general theorem. ChatGPT did so in seconds, and my friend was thrilled.
He was also excited about his use of it to translate articles on his Chinese friend’s website into English. He gave one of the translations to his friend, who is fluent in both languages, and she said the translation was perfect and better than a human could do.
Let’s remember, though, that everything ChatGPT does is gleaned from human texts. It’s not itself intelligent. It’s as if it has assimilated the world’s knowledge and can, on demand, apply it to your specific prompt.
Google is very afraid. It seems antiquated in comparison, and Google knows that unless it comes up with something revolutionary to compete with the new Large Language Model chatbots, its business could vanish. To that end, Google is obsessed with modernizing its search engine, and as I write this in early May, the company is planning to roll out some changes this month. Features they’re exploring include using a chatbot to search for music and the ability to interact with a chatbot while using their Chrome browser.
They’ve also rolled out their Bard chatbot to a limited number of users, but when I asked it to write a history of Fairfield, the result was bizarre. It said that Fairfield was located near the Chisholm Trail and that it was a “major center of the cattle trade and was home to several stockyards and packing plants.” Also, I haven’t been able to figure out how to see my previous conversations with Bard.
Microsoft, too, is desperate to compete, but their Bing Chat feature, which is gradually being made available, is inferior as I write this. There doesn’t seem to be any way to see its previous responses. That’s a key feature that ChatGPT does well. When I asked a specific medical question to ChatGPT-4, I was able to go back weeks later and ask follow-up questions in the conversation.
Microsoft is so desperate to take advantage of the current upheaval in search that they’ve begun offering rewards for using their Bing search engine. You can earn points every time you search with Bing, and then use those for gift cards or other prizes.
Meanwhile, ChatGPT-4 forges ahead, offering more features, such as plugins that increase its capabilities. ChatGPT-4 is also now available on smartphone apps. My favorite so far is ChatOn, which gives you access to ChatGPT-4 for $40 per year compared to $20 per month when you access it via your browser.
I’ve subscribed to both, of course. I don’t ever want to be without ChatGPT-4 nearby. But if you’re reluctant to pay, you can still use version 3.5 for free (chat.openai.com). It’s actually the version my friend used for his geometry proof and Chinese translation.
Find column archives at JimKarpen.com.