metricas
covid
Buscar en
Actas Urológicas Españolas (English Edition)
Toda la web
Inicio Actas Urológicas Españolas (English Edition) Quality of information about urologic pathology in English and Spanish from Chat...
Journal Information
Vol. 48. Issue 5.
Pages 398-403 (June 2024)
Share
Share
Download PDF
More article options
Vol. 48. Issue 5.
Pages 398-403 (June 2024)
Resident section
Quality of information about urologic pathology in English and Spanish from ChatGPT, BARD, and Copilot
Calidad de información de ChatGPT, BARD y Copilot acerca de patología urológica en inglés y en español
J.J. Szczesniewskia,b,
Corresponding author
Juliusz.szcz@gmail.com

Corresponding author.
, A. Ramos Albac,d, P.M. Rodríguez Castroa, M.F. Lorenzo Gómezb,e, J. Sainz Gonzálezd, L. Llanes Gonzáleza,f
a Servicio de Urología, Hospital Universitario de Getafe, Getafe, Madrid, Spain
b Departamento de Cirugía, Facultad de Medicina, Universidad de Salamanca, Salamanca, Spain
c DXC Technology, Las Rozas, Madrid, Spain
d Departamento de Economía Aplicada I e Historia e Instituciones Económicas, Universidad Rey Juan Carlos, Madrid, Spain
e Servicio de Urología, Hospital Universitario de Salamanca, Salamanca, Spain
f Universidad Francisco de Vitoria, Madrid, Spain
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Tables (4)
Table 1. Scoring of responses provided by ChatGPT on DISCERN test and Likert scale for information appropriacy. Appropriacy is defined as no score <3 on the Likert scale.
Table 2. Scoring of responses provided by BARD on DISCERN test and Likert scale for information appropriacy. Appropriacy is defined as no score <3 on the Likert scale.
Table 3. Scoring of responses provided by Copilot on DISCERN test and Likert scale for information appropriacy. Appropriacy is defined as no score <3 on the Likert scale.
Table 4. Score on the informed consent questionnaire developed by Spatz et al. 15
Show moreShow less
Additional material (1)
Abstract
Introduction and objective

Generative artificial intelligence makes it possible to ask about medical pathologies in dialog boxes. Our objective was to analyze the quality of information about the most common urological pathologies provided by ChatGPT (OpenIA), BARD (Google), and Copilot (Microsoft).

Methods

We analyzed information on the following pathologies and their treatments as provided by AI: prostate cancer, kidney cancer, bladder cancer, urinary lithiasis, and benign prostatic hypertrophy (BPH). Questions in English and Spanish were posed in dialog boxes; the answers were collected and analyzed with DISCERN questionnaires and the overall appropriateness of the response. Surgical procedures were performed with an informed consent questionnaire.

Results

The responses from the three chatbots explained the pathology, detailed risk factors, and described treatments. The difference is that BARD and Copilot provide external information citations, which ChatGPT does not. The highest DISCERN scores, in absolute numbers, were obtained in Copilot; however, on the appropriacy scale it was noted that their responses were not the most appropriate. The best surgical treatment scores were obtained by BARD, followed by ChatGPT, and finally Copilot.

Conclusions

The answers obtained from generative AI on urological diseases depended on the formulation of the question. The information provided had significant biases, depending on pathology, language, and above all, the dialog box consulted.

Keywords:
Artificial intelligence
Information quality
ChatGPT
Copilot
BARD
Urology
Resumen
Introducción y objetivo

La inteligencia artificial (IA) generativa permite preguntar, a través de los cuadros de diálogo, sobre patologías médicas. Nuestro objetivo fue analizar la calidad de la información acerca de las patologías urológicas más comunes en los chatbots ChatGPT de OpenIA, BARD de Google y Copilot de Microsoft.

Material y método

Se realizó un análisis de la información aportada por IA sobre las siguientes patologías y sus tratamientos: cáncer de próstata, cáncer renal, cáncer de vejiga, litiasis urinarias e hipertrofia benigna de próstata (HBP). A través de cuadros de diálogo se formularon preguntas estructuradas en inglés y en español, recopilando las respuestas para analizarlas posteriormente con cuestionarios DISCERN y sobre la idoneidad global de la respuesta. Los tratamientos quirúrgicos se realizaron con cuestionario de consentimiento informado.

Resultados

Las respuestas obtenidas a través de los tres chatbots explicaron la patología, detallaron los factores de riesgo y describieron los tratamientos. La diferencia radica en que BARD y Copilot aportan citas de información externa, algo que ChatGPT no realiza. Las puntuaciones DISCERN más altas, en números absolutos, se obtuvieron en Copilot; sin embargo, en la escala de idoneidad se objetivó que sus respuestas no fueron las más apropiadas. Las mejores puntuaciones de tratamientos quirúrgicos fueron obtenidas por BARD, seguido de ChatGPT y finalmente de Copilot.

Conclusiones

Las respuestas obtenidas de la IA generativa sobre enfermedades urológicas dependieron de la formulación de la pregunta. La información proporcionada presentó sesgos importantes, dependiendo de la patología, del idioma y, sobre todo, del cuadro de diálogo consultado.

Palabras clave:
Inteligencia artificial
Calidad de información
ChatGPT
Copilot
BARD
Urología

Article

These are the options to access the full texts of the publication Actas Urológicas Españolas (English Edition)
Subscriber
Subscriber

If you already have your login data, please click here .

If you have forgotten your password you can you can recover it by clicking here and selecting the option “I have forgotten my password”
Subscribe
Subscribe to

Actas Urológicas Españolas (English Edition)

Purchase
Purchase article

Purchasing article the PDF version will be downloaded

Price 19.34 €

Purchase now
Contact
Phone for subscriptions and reporting of errors
From Monday to Friday from 9 a.m. to 6 p.m. (GMT + 1) except for the months of July and August which will be from 9 a.m. to 3 p.m.
Calls from Spain
932 415 960
Calls from outside Spain
+34 932 415 960
E-mail
Article options
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos