The analysis of ethical aspects in clinical research has always been a challenge and has required constant updates.
In short, research ethics is the set of specific principles, rules, and norms of behavior that a research community has decided are appropriate and fair under the premise that research must be valid, reliable, legitimate, and representative.
This non-systematic review brings some ethical concerns that should be considered within the scientific community. Many studies and the development of new artificial intelligence (AI) tools, especially in radiology, make it necessary for the radiology research community to promote debates and establish ethical standards for the practice and development of new AI tools.
El análisis de aspectos éticos en la investigación clínica siempre ha supuesto un reto que ha requerido de constantes actualizaciones.
En resumen, la ética en la investigación es el conjunto de principios, reglas y normas específicas del comportamiento que una comunidad investigadora ha decidido como apropiadas y justas bajo la premisa de que la investigación debe caracterizarse por ser válida, fiable, legítima y representativa.
Se presenta una revisión no sistemática que trata algunas de las preocupaciones éticas que la comunidad científica debe tener en cuenta. Numerosos estudios sobre el desarrollo de nuevas herramientas de inteligencia artificial (IA), especialmente aplicadas a la radiología, hacen necesario que la comunidad investigadora en esta área promueva debates y establezca unos principios éticos aplicables a la práctica y al desarrollo de las nuevas herramientas de IA.
In health research, ethics involves a set of principles and norms that must be constantly updated and reassessed to ensure that it is valid, reliable, legitimate, and representative.1
Digital technology is transforming how healthcare professionals work, including radiology, where AI tools are increasingly used to automate labor-intensive tasks like examining CT images. This shift requires radiologists to adapt and focus on more complex cognitive tasks.2–5
Although AI-based technology and its application in healthcare are expanding, the paucity of ethical aspects in AI research remains a significant issue. The current published literature lacks practical tools for testing and upholding ethical requirements across the lifecycle of AI-based technologies. Besides, its enforcement in public health raises ethical concerns related to privacy, trust, accountability, and numerous biases.6
In this article, by reviewing the existing literature, we aim to identify and discuss some ethical issues that arise with the development and use of AI algorithms in radiology, such as the ethics of data, algorithms, practice, and conflicts of interest, by analyzing current research and contributing to the ongoing discussion on AI ethics in healthcare and promoting the responsible use of AI technology in radiology.
The ethics of dataThe radiology domain contains a substantial amount of patient data. The ethics of data surrounds the acquisition, management, and assessment of data. Some of the most important areas of data ethics to consider include informed consent, ownership of data, transparency, objectivity, privacy/data protection, ensuring moral and meaningful access to data, and the resource to ensure data management.1–3,7 Indeed, researchers and radiologists have a moral obligation to utilize patient information to improve radiology practice and patient care.2,3,7
Unfortunately, there are ways that data can be unethically used, in particular for commercial purposes.2,3 In addition, well-labeled and high-quality data, required along and after algorithm training, are highly fetched, and their value is skyrocketing.8
Certainly, it is imperative to consider inquiries regarding patient data ownership.3,4 For instance, who owns the data that can lead to the creation of highly profitable intelligence products? Also, who owns the intellectual property of the analysis that emerges from aggregated data?2,4,7
Patient autonomy over their data is a crucial point of discussion. Most patients agree, through consent forms, to the retrospective use of data for research purposes. In fact, in Europe, the General Data Protection Regulation (GDPR) allows consent withdrawal at any time and requires patients' permission for information reuse.8 Data ownership definitions vary significantly among countries, limiting the answers about data's commercial use beneficiaries.8 Ultimately, patients own their data, so should they participate in the profits made by AI systems built through the given information? Or should the companies fully hold the interests once they buy the rights to medical data access? Foremost, could patients choose to use their data exclusively for academic ends? Or even pick the company they want to sell their information to?
Deeper discussions are needed to understand commercial and academic data practices to create policies that balance benefits with the greater good without harming patients.
Data collection and managementAI systems greatly depend on the quality and amount of data used to develop them.8 Although, there are significant barriers related to radiology data collection, annotation process, availability, and accessibility. This scenario leads to data scarcity and impairs deep learning (DL) -powered tools to subsidize radiologists' decisions.8–10
Representative and high-quality data require researchers' continuous monitoring along the data extraction steps.8,11 Concerning this remark, automated data extraction systems showed flawed extraction ability once the scarcity of standardized data and contextual and user-specific variability proposes challenges to its performance.8
Indeed, the radiology field is significantly prejudiced by technical acquisition factors bias due to divergences between different machines or acquisition methods.8 While radiologists are used to interpreting technical differences in imaging, such as slice thickness or scanner brand,
DL- systems could better execute this ability if they were exposed to these variables during the training phase.8
Bias can also arise when handling patient data.2–4,7 As aforementioned, data sharing is fundamental to diverse image datasets.9 Although, other issues with the implementation of widespread data sharing include data access policies, data quality and safety policies, intellectual property issues, and data protection.10
Besides, ethical impasses emerge due to imaging reconstruction technologies, especially regarding facial exams, which violate patients' privacy.9 Nowadays, facial recognition to 3D reconstruction can build models from unidentified medical images, such as MRIs, thereby unraveling anonymization.8 However, the existing re-identification prevention software is limited to specific DL systems and data types, hindering its application to most algorithms.8
The ethics of algorithmsDecision-making is part and parcel of medicine and healthcare. It involves selecting a course of action from different alternatives.3 People make decisions using their beliefs, knowledge, preferences, and values.3 AI makes a decision depending on the features of input data.3,4 Human values, beliefs, and preferences will often be transferred to AI, yet it is the source of human bias.2,3 Although AI products are not human, they are envisioned, built, and evaluated by them.2,3 Therefore, human concepts are responsible for equality and fairness.3 In other words, humans can misuse AI models.3,4 Therefore, it is imperative to ensure transparency in how decisions are made to promote provider and patient trust in AI.2,3,5,12
Furthermore, AI use promotes the "automation bias," meaning that humans start to rely entirely on the work of a machine instead of applying their critical judgment and scrutiny. Therefore, patients become more vulnerable to AI's mistakes if health decision-making gets based on physicians' trust in unverified AI conclusions.9
Particularly in insurance-based countries, AI systems could prejudicate multiple health system users.8 Therefore, AI practical application requires constant and assertive analysis to prevent algorithms' decisions from being accepted over doctors' moral and knowledge-guided intuition.8
The ethics of practiceAI in radiology is complex because it combines clinical care, business, economics, technology, and mathematics.3 Nevertheless, moral behavior is intellectually uncertain.3 Moreover, there are instances where innovations unintentionally cause harm and engage in unprincipled activities.7 Therefore, there is a need to engage in moral and ethical values when deciding where to involve AI, define what responsible AI ought to be, and raise the alarm when AI behaves unethically.3
The sampling bias occurs when curated data needs to appropriately represent the population, primarily due to data collection barriers.8,11 This concern occurs when a single institution provides the data to develop and train the DL algorithms, resulting in discrimination of underrepresented subsets of other institutions' populations.8
Unarguably, DL- models demand financial and scientific investments, primarily available to economically prosperous countries.8 Additionally, low- and middle-income countries' (LMICs) scarcity of infrastructure generates additional challenges, mainly concerning their ability to inform patients, communicate uncertainty, administer consent, and generate robust data.6 Consequently, these nations are forced to use DL- models trained on data from developed countries, which are different from the reality of rural areas.10 This scenario aggravates social inequalities regarding healthcare and highlights the use limitations of DL- algorithms in the regions that would benefit the most, such as those with insufficient resources, specialists, and technology.
Moreover, the lack of analyzed data promotes harm to underrepresented groups based on their gender, sexual orientation, ethnicity, comorbidities, social status, or economic factors, among others.1–3,5,7,8,11 Analytical healthcare studies demonstrate significant differences in underdiagnosis rates depending on the above-mentioned variables.8 For instance, if training datasets fail to present "rare" conditions, AI algorithms will fail to identify structures resembling inherent traits from underrepresented groups.8,11 Therefore, all potential sources of bias must be considered to reduce their impact on AI's decisions.3
Envisioning a solution for sampling bias, the external validation method guarantees the generalization of AI systems by using representative data from other institutions. Unfortunately, despite its relevance, only 6% of recent medical DL papers included validation on independent external data.8
Reporting guidelinesSome ethical concerns included safety, transparency, and value alignment.1,3,7 AI systems must be verifiable and secure to ensure safety.3,7,12 Although the algorithm's actions may be discernible, understanding its decision-making rationale may be challenging, underscoring the importance of transparency.3,7,8,12 Value alignment is about optimizing AI's work for the patient's benefit, bearing this responsibility to researchers and radiologists.3,7
New analysis procedures tailored to the nature of algorithms are needed in addition to standard root-cause analysis.3,7,12 In addition, the expanding literature on AI in medical imaging requires transparent and systematic reporting of research.13
FUTURE-AI (Fairness Universality Traceability Usability Robustness Explainability-AI), published in 2021, proposed broad principles for AI development in medical imaging, encompassing research, design, and deployment. Unlike prior guidelines, which focused on manuscript structure, FUTURE-AI emphasizes AI systems' fairness, usability, robustness, and explainability.
It introduces novel topics, including "clinical conception," "end-user requirement gathering," and "AI deployment and monitoring," aiming for equitable and minimally biased systems.13
The DECIDE-AI (Developmental and Exploratory Clinical Investigations of Decision support systems driven by AI ) guideline, published in 2022, aims to ensure transparent reporting of clinical studies evaluating AI systems and address human influence on clinical AI performance. This framework enabled and uniformed the assessment of complex interventions by approaching a stage of development, the early clinical trials, instead of a type of study. It provides a checklist for live evaluation to analyze clinical utility and safety, assess users' learning curves, and prepare the algorithm for larger-scale evaluations.14
Currently, the AI - guidelines under development include STARD-AI and TRIPOD-AI. The first standardize diagnostic accuracy reports, while the latter evaluates prediction model studies.13,14
Technical validationMany AI systems for high-stakes decisions are developed daily, risking adverse outcomes.8 For AI's radiology performance to be safe and effective, validation criteria such as robustness, reproducibility, and generalizability must be met. Unfortunately, designing a technical validation study is challenging since most studies use the same dataset for algorithm development, optimization, and validation. It leads to a need for more generalizability and robustness verification, besides possible data leakage. It is essential to specify the expected accuracy of the algorithm before proceeding to the next verification steps. Technical approval must be followed by real clinical validation to ensure patient safety. The algorithm's performance must be evaluated before implementing it in a routine setting.9
The implementation of DL-powered systems must ensure safety, efficacy, and equity. However, the current form of DL explainability techniques could be more suitable for fields in which patients' lives are at stake, such as radiology.8
Regulatory frameworksIn deploying DL systems, ethical standards are ensured through different methods in each country. Europe requires the CE mark, while the USA mandates clearance from regulatory agencies like the FDA and local IRBs.8
Approval from regulatory frameworks is crucial for adopting AI systems in medical practice. However, the current protocols have limitations and need improvement to evaluate diagnostic AI algorithms comprehensively.15
The International Medical Device Regulators Forum (IMDRF) sets the standards followed by most medical software regulators, such as the FDA and the frameworks from the European Union (EU). The regulatory bodies broadly explore AI systems' safety, effectiveness, and performance.15 However, they stumble on crucial points such as the conflation of the diagnostic task and diagnostic algorithm, simple definition of the diagnostic task, absence of mechanisms to compare similar algorithms directly, poor definition of safety and performance elements, and lack of resources to access performance at each installed site.15
A discrete evaluation process enhances the ability to address software issues and allows for comparison. The process involves defining the diagnostic task, testing in a controlled environment, evaluating real-world effectiveness, assessing durability over time, and setting internal developer benchmarks. Guidelines based on this approach improve control over the longitudinal implantation of AI systems.15
Conventional regulatory frameworks must ensure excellence at every medical software application site.15 To address this, third-party evaluators, such as clinical research
organizations, research laboratories, or organizations that develop and maintain reference standard data sets, could be utilized. This approach is already implemented in drug studies under the supervision of regulatory agencies.15
Conflict of interestIn nascent radiology AI markets, radiologists involved in patient care may also have positions in AI startups or established commercial entities.1–3 Similar to drug investigators with financial interests in drug success, COI related to AI products may be managed through remedies like public disclosure, divestment, or oversight. Medical software manufacturers funding and publishing evaluations of their products may also create COI.15
Stakeholders responsible for sharing patient data, procuring AI agents, or implementing models in clinical workflows should carefully manage their conflicts of interest when dealing with AI in healthcare. In some cases, they may need to recuse themselves from such activities.3
The emergence of AI tools for text productionWe must consider that, with the emergence of AI tools for text production, this becomes more worrying to be considered related to research.16 For example, Large Language Models (LLMs) qualified by AI can generate increasingly complex phrases to distinguish from the text written by people. At the same time, ChatGPT and Oher LLMS increase repair in education and scientific audiences due to their ability to write text for evaders, reports, examinations, and scientific papers.16
ConclusionIn the Big Data era, the seven research ethics requirements proposed by Emanuel, E. J. (2000) must be revised for AI-related practice and research. Therefore, frequent discussions and debates are necessary.1,3 In conclusion, the incorporation of AI into radiology has enhanced the efficiency and precision of radiologists. However, resolving ethical concerns surrounding patient data, algorithms, and conflicts of interest is essential to ensure patient safety and privacy. Developing and enforcing regulatory frameworks and ethical principles can assist in mitigating potential moral concerns and maximizing the benefits of AI in radiology.
Study informationThis study was performed at the Neuroradiology Department of Hospital Antonio Prudente, Fortaleza - CE, Brazil.
Funding sourceThis research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
