The Problem
Artificial intelligence (AI) is transforming healthcare faster than the systems designed to evaluate and govern it. A sophisticated pattern-recognition tool that learns from the data on which it is trained, AI can reinforce inequities if it is built on biased or incomplete data and implemented in ways that fail to meet the needs of diverse patients.
What History Teaches Us
Science-fiction writer William Gibson famously observed, “The future is already here – it’s just not evenly distributed.” Every disruptive technology follows a similar pattern: those with the greatest resources benefit first, while equitable access requires intentional action.
When electricity first arrived, it primarily served wealthy urban communities until policies like the Rural Electrification Act of 1936 dramatically expanded access to rural communities. Dr. Kedar Mate, Founder and Chief Medical Officer for Qualified Health AI, argues that artificial intelligence poses a similar challenge. Like electricity, the equitable distribution of AI will depend on the choices we make about how it is designed, governed, and distributed.
The Approach
Health systems, researchers, and medical societies must move beyond asking whether AI works and instead pose difficult questions about for whom it is working: Who is represented in the data? Who benefits from the technology? Who has access to it? And who might be left behind?
Dr. Leo Celi, Senior Research Scientist at Massachusetts Institute of Technology (MIT), recommends using the LTARC Framework to evaluate the ethical deployment of AI:
- Local: Evaluate AI in the health system where it will actually be used. A model that performs well in one hospital or patient population may not produce the same results elsewhere.
- Task-specific: Test AI on the real-world task it is intended to perform, not abstract measures of intelligence. For example, an AI model’s performance on the U.S. Medical Licensing Examination says little about how well it will support patient care.
- Agile: Be prepared to change the evaluation framework frequently based on new information.
- Reflective and Reflexive: Monitor AI’s impact on health equity. If a tool widens disparities or produces unintended harm, its use should be paused and reassessed.
- Community-partnered: Perhaps most importantly, give patients and communities a seat at the table. Ask community members to identify clinical pain points, negotiate data-sharing agreements that respect community ownership, include community participants on the design team itself, and establish community advisory boards that retain veto power over model updates based on community-driven equity dashboards.
Ultimately, Dr. Celi argues, implementing AI so that it doesn’t deepen healthcare inequalities requires a “mindset revolution.” “We need to change and question the way we know, the way we think, the way we relate,” he says. “It’s about being open to disruption.”
Call to Action for Medical Societies
1. Use AI responsibly in your work. Become familiar with the technology, its strengths, and its limitations. “The future of this technology is going to be written by the people that are using it,” says Dr. Mate.
2. Join your institution’s governance systems. Share best practices like the LATRC framework and encourage meaningful community input.
3. Require independent, locally validated, subgroup-stratified evidence before endorsing any AI tool or citing it in guidelines.
4. Write AI evaluation and equity auditing into clinical practice guidelines. Make asking “for whom does this work” a professional obligation. Work with certifying boards to advocate for incorporating these concepts into board certification and maintenance of certification (MOC) processes.
5. Apply strategies from the Encoding Equity Alliance Artificial Intelligence Implementation Toolkit.