Can AI answer medical questions better than your doctor?
The potential for AI, like ChatGPT, to provide answers to patient questions in healthcare is intriguing. However, a study comparing responses from ChatGPT to those from physicians revealed some surprising results.
According to an article by Harvard Health, researchers collected questions and answers from an online platform and had a panel rate them for quality and empathy. ChatGPT outperformed physicians in terms of quality, with 78% of its responses rated as good or very good compared to 22% for physicians.
In terms of empathy, ChatGPT scored significantly higher at 45% compared to physicians' 4.6%. Notably, ChatGPT's responses were longer on average.
Despite these findings, the study had limitations. It did not assess the accuracy of the answers or whether patients would accept AI-generated responses.
Additionally, the evaluation criteria were subjective, and the evaluators were not always blinded to whether the response came from a physician or ChatGPT.
While AI may offer insights into empathy and collaboration with healthcare professionals, relying solely on AI for patient responses may be premature without evidence of accuracy and professional supervision.
Even ChatGPT acknowledges that it cannot answer medical questions better than a doctor.